Gender Effects in Complex Problem Solving

I am really excited about my most recent experiment on gender effects in complex problem solving (CPS). Complex problems represent the type of problems that managers and politicians face on an everyday basis: A complex and dynamic (changing on its own over time) system needs to be transformed from a current state into an ill-defined goal state; the system is networked (i. e., tweaking at one screw will lead to unanticipated changes in other parts of the system) and multiple, possibly conflicting goals need to be pursued. The ability to solve such complex problems is tested with so-called "Microworlds", complex computer-simulations that place the gamer in a semantically framed complex problem scenario: A company needs to be saved from bankruptcy, a system needs to be steered within certain parameters, a forest needs to be catered for, an eco-system must be maintained. These microworlds run over a simulated period of time (usually several months), many variables can be tweaked, and decisions taken at an early step influence the further cause of the game. A bit like SimCity if you will. CPS performance is largely determined by certain factors of intelligence and by knowledge on the system in question. This knowledge is usually obtained during the problem solving process itself. Thus, the ability to identify connections, to understand systems, and to learn quickly is a key determinant of CPS.

Since complex problem solving is considered to be a core managerial competence, microworlds are frequently employed in assessment centers by large corporations such as banks and business consultants.

However, I came accross two studies that bothered to examine microworld performanc scores seperately for male and female experiment participants. All other studies I came accross did not report individual findings for the two sexes. I have an idea why: The two abovementioned studies reported gender effects in the direction that men outperform women. Those two studies (both from the 90s) explained the gender effect with higher intelligence levels of male participants and with higher levels of computer experience among male participants. If these artifacts were controlled for, the statistical difference between male and female complex problem solvers would vanish. That was not the case in the experiment I conducted in my PhD-thesis. I found severe gender differences in CPS performance, even after controlling for several variables: Intelligence, learning, computer experience, and economic knowledge. These variables were unable to explain the gender differences I found.

Now, one wouldn't say that women are poorer managers than men. At the same time, if my results hold true, the use of microworlds in assessment centers favors male applicants over female applicants. This sounded like an important issue to me and I decided to pursue the matter further.

My brilliant colleague Carmen Lebherz suggested the concept of stereotype threat to me when I told her about my odd findings. Wikipedia:

"Stereotype threat is the fear that one's behavior will confirm an existing stereotype of a group with which one identifies. This fear may lead to an impairment of performance."

In my case, the either explicit or implicit stereotype that women are poor in CPS (or in "computer-related stuff") may have impaired the performance of my female experiment participants. I designed an experiment in order to test this assumption. We employed a 2 x 2 x 3 between-subjects design: gender (male / female) x stereotype threat (yes / no) x microworld (Taylorshop / FSYS / ColorSim). Stereotype threat was altered by the instructions that the experiment participants received. In the stereotype threat condition, participants were told that we would measure their ability to solve complex problems with a complex problem solving microworld. We told them of the role microworlds play in assessment centers and asked them to do their best. In the non-stereotype-threat condition, we told them that we would like them to play a kind of computer game and that we would be interested in the emotions that this game would create (which we measured with Marx & Stapel's 2006 questionnaire).

The result: Over all three employed scenarios, female experiment participants exhibited much poorer performance under the stereotype threat condition than under the non-stereotype-threat condition, as the graph below illustrates (standardized CPS performance is indicated on the y-axis over all three microworlds).

The weird thing is that this happens both to women who think that men do better in microworlds and to women that do not think so, i.e. the effect of stereotype occurs regardless of the salience of the stereotype.

Further analyses of covariance will hopefully shed more light on the conditioning factors of these effects (we measured motivation, frustration, anxiety, intelligence, experience with computer-simulations only to name a few). However, this is a compelling example for the role of the situation and setting on human performance.

I will try to write up a paper on our findings as soon as I finish data analysis. In the meantime, I would like to thank my collaborators and the people who enabled this experiment: Heinz Gutscher for the generous funding and the tremendous working conditions at his group, Jürgen Boss for adapting ColorSim for use in my expriment (during his xmas holidays!), Annette Kluge for providing me with Jürgen's taylor-made version of ColorSim, Dietrich Wagener for providing a copy of FSYS, my students Jeanine Grütter, Marisa Oertig, and Rahel Schuler for their great efforts in conducting the experiments (179 participants in the lab in six weeks!), and finally our great and willing participants.

Copyright for the first two above images obtained from www.istockphoto.com. Reproduction is prohibited.

Implicit knowledge in complex system control

What does it take to be a successful manager or politician? In Psychology, tasks that these groups face on an everyday basis are called complex problems. Complex problems are characterized by their dynamics (the system that is being altered during problem solving is constantly changing on its own), by the fact that multiple goals have to be achieved at the same time and by their network structure: changes to one variable usually lead to changes of other variables as well.
Experimental Psychology simulates complex problems with computer-simulated dynamic scenarios, also referred to as microworlds; computer games that place the subject into the role of a manager of a factory or a mayor of a small city. Over a simulated period of time, the subject has to improve some target value (e.g. factory profit or item sales) and has a variety of variables that he can change for influencing the system (e.g. number of machines, number of employees, wages, welfare, advertising budget etc).
The current state of research argues that it does not require a special skill for complex problem solving, but that complex problem solving ability is determined by a person's intelligence, especially his/her ability to process information, and by a person's knowledge of the problem domain, i.e. structure of the system. However, research has also shown that with longer experience of system control, subjects tend to show better problem solving abilities without showing an increase in verbalizable knowledge. The question whether this was a sign for implicit knowledge or just some artifact has been heavily debated.
In my experiment, I devided my subjects in two groups. One group learned how to control a complex scenario from texts, the other group also and learned from texts and was able to try out the scenario during the learning period. Both groups scored equal in a post-learning test for verbalizable system knowledge, but the group that had experience with the szenario performed better during actual scenario control. This dissociation between verbalizable knowledge and increased scenario performance is a clear indication for the development of implicit knowledge.
On top of that, the group with experience learning showed a higher density in knowledge organization in my Association Structure Test (AST). This supports my theory that implicit knowledge correlates with a higher density in knowledge organization (which can be found here).