Gender Effects in Complex Problem Solving

I am really excited about my most recent experiment on gender effects in complex problem solving (CPS). Complex problems represent the type of problems that managers and politicians face on an everyday basis: A complex and dynamic (changing on its own over time) system needs to be transformed from a current state into an ill-defined goal state; the system is networked (i. e., tweaking at one screw will lead to unanticipated changes in other parts of the system) and multiple, possibly conflicting goals need to be pursued. The ability to solve such complex problems is tested with so-called "Microworlds", complex computer-simulations that place the gamer in a semantically framed complex problem scenario: A company needs to be saved from bankruptcy, a system needs to be steered within certain parameters, a forest needs to be catered for, an eco-system must be maintained. These microworlds run over a simulated period of time (usually several months), many variables can be tweaked, and decisions taken at an early step influence the further cause of the game. A bit like SimCity if you will. CPS performance is largely determined by certain factors of intelligence and by knowledge on the system in question. This knowledge is usually obtained during the problem solving process itself. Thus, the ability to identify connections, to understand systems, and to learn quickly is a key determinant of CPS.

Since complex problem solving is considered to be a core managerial competence, microworlds are frequently employed in assessment centers by large corporations such as banks and business consultants.

However, I came accross two studies that bothered to examine microworld performanc scores seperately for male and female experiment participants. All other studies I came accross did not report individual findings for the two sexes. I have an idea why: The two abovementioned studies reported gender effects in the direction that men outperform women. Those two studies (both from the 90s) explained the gender effect with higher intelligence levels of male participants and with higher levels of computer experience among male participants. If these artifacts were controlled for, the statistical difference between male and female complex problem solvers would vanish. That was not the case in the experiment I conducted in my PhD-thesis. I found severe gender differences in CPS performance, even after controlling for several variables: Intelligence, learning, computer experience, and economic knowledge. These variables were unable to explain the gender differences I found.

Now, one wouldn't say that women are poorer managers than men. At the same time, if my results hold true, the use of microworlds in assessment centers favors male applicants over female applicants. This sounded like an important issue to me and I decided to pursue the matter further.

My brilliant colleague Carmen Lebherz suggested the concept of stereotype threat to me when I told her about my odd findings. Wikipedia:

"Stereotype threat is the fear that one's behavior will confirm an existing stereotype of a group with which one identifies. This fear may lead to an impairment of performance."

In my case, the either explicit or implicit stereotype that women are poor in CPS (or in "computer-related stuff") may have impaired the performance of my female experiment participants. I designed an experiment in order to test this assumption. We employed a 2 x 2 x 3 between-subjects design: gender (male / female) x stereotype threat (yes / no) x microworld (Taylorshop / FSYS / ColorSim). Stereotype threat was altered by the instructions that the experiment participants received. In the stereotype threat condition, participants were told that we would measure their ability to solve complex problems with a complex problem solving microworld. We told them of the role microworlds play in assessment centers and asked them to do their best. In the non-stereotype-threat condition, we told them that we would like them to play a kind of computer game and that we would be interested in the emotions that this game would create (which we measured with Marx & Stapel's 2006 questionnaire).

The result: Over all three employed scenarios, female experiment participants exhibited much poorer performance under the stereotype threat condition than under the non-stereotype-threat condition, as the graph below illustrates (standardized CPS performance is indicated on the y-axis over all three microworlds).

The weird thing is that this happens both to women who think that men do better in microworlds and to women that do not think so, i.e. the effect of stereotype occurs regardless of the salience of the stereotype.

Further analyses of covariance will hopefully shed more light on the conditioning factors of these effects (we measured motivation, frustration, anxiety, intelligence, experience with computer-simulations only to name a few). However, this is a compelling example for the role of the situation and setting on human performance.

I will try to write up a paper on our findings as soon as I finish data analysis. In the meantime, I would like to thank my collaborators and the people who enabled this experiment: Heinz Gutscher for the generous funding and the tremendous working conditions at his group, Jürgen Boss for adapting ColorSim for use in my expriment (during his xmas holidays!), Annette Kluge for providing me with Jürgen's taylor-made version of ColorSim, Dietrich Wagener for providing a copy of FSYS, my students Jeanine Grütter, Marisa Oertig, and Rahel Schuler for their great efforts in conducting the experiments (179 participants in the lab in six weeks!), and finally our great and willing participants.

Copyright for the first two above images obtained from www.istockphoto.com. Reproduction is prohibited.