Diversity Mediocrity Illusion

I've often been involved in discussions about deliberately increasing the diversity of a group of people. The most common case in software is increasing the proportion of women. Two examples are in hiring and conference speaker rosters where we discuss trying to get the proportion of women to some level that's higher than usual. A common argument against pushing for greater diversity is that it will lower standards, raising the spectre of a diverse but mediocre group.

To understand why this is an illusionary concern, I like to consider a little thought experiment. Imagine a giant bucket that contains a hundred thousand marbles. You know that 10% of these marbles have a special sparkle that you can see when you carefully examine them. You also know that 80% of these marbles are blue and 20% pink, and that sparkles exist evenly across both colors [1]. If you were asked to pick out ten sparkly marbles, you know you could confidently go through some and pick them out. So now imagine you're told to pick out ten marbles such that five were blue and five were pink.

I don't think you would react by saying “that's impossible”. After all there are two thousand pink sparkly marbles in there, getting five of them is not beyond the wit of even a man. Similarly in software, there may be less women in the software business, but there are still enough good women to fit the roles a company or a conference needs.

The point of the marbles analogy, however, is to focus on the real consequence of the demand for 50:50 split. Yes it's possible to find the appropriate marbles, but the downside is that it takes longer. [2]

That notion applies to finding the right people too. Getting a better than base proportion of women isn't impossible, but it does require more work, often much more work. This extra effort reinforces the rarity, if people have difficulty finding good people as it is, it needs determined effort to spend the extra time to get a higher proportion of the minority group — even if you are only trying to raise the proportion of women up to 30%, rather than a full 50%.

In recent years we've made increasing our diversity a high priority at Thoughtworks. This has led to a lot of effort trying to go to where we are more likely to run into the talented women we are seeking: women's colleges, women-in-IT groups and conferences. We encourage our women to speak at conferences, which helps let other women know we value a diverse workforce.

When interviewing, we make a point of ensuring there are women involved. This gives women candidates someone to relate to, and someone to ask questions which are often difficult to ask men. It's also vital to have women interview men, since we've found that women often spot problematic behaviors that men miss as we just don't have the experiences of subtle discriminations. Getting a diverse group of people inside the company isn't just a matter of recruiting, it also means paying a lot of attention to the environment we have, to try to ensure we don't have the same AlienatingAtmosphere that much of the industry exhibits. [3]

One argument I've heard against this approach is that if everyone did this, then we would run out of pink, sparkly marbles. We'll know this is something to be worried about when women are paid significantly more than men for the same work.

One anecdote that stuck in my memory was from a large, traditional company who wanted to improve the number of women in senior management positions. They didn't impose a quota on appointing women to those positions, but they did impose a quota for women on the list of candidates. (Something like: "there must be at least three credible women candidates for each post".) This candidate quota forced the company to actively seek out women candidates. The interesting point was that just doing this, with no mandate to actually appoint these women, correlated with an increased proportion of women in those positions.

For conference planning it's a similar strategy: just putting out a call for papers and saying you'd like a diverse speaker lineup isn't enough. Neither are such things as blind review of proposals (and I'm not sure that's a good idea anyway). The important thing is to seek out women and encourage them to submit ideas. Organizing conferences is hard enough work as it is, so I can sympathize with those that don't want to add to the workload, but those that do can get there. FlowCon is a good example of a conference that made this an explicit goal and did far better than the industry average (and in case you were wondering, there was no difference between men's and women's evaluation scores).

So now that we recognize that getting greater diversity is a matter of application and effort, we can ask ourselves whether the benefit is worth the cost. In a broad professional sense, I've argued that it is, because our DiversityImbalance is reducing our ability to bring the talent we need into our profession, and reducing the influence our profession needs to have on society. In addition I believe there is a moral argument to push back against long-standing wrongs faced by HistoricallyDiscriminatedAgainst groups.

Conferences have an important role to play in correcting this imbalance. The roster of speakers is, at least subconsciously, a statement of what the profession should look like. If it's all white guys like me, then that adds to the AlienatingAtmosphere that pushes women out of the profession. Therefore I believe that conferences need to strive to get an increased proportion of historically-discriminated-against speakers. We, as a profession, need to push them to do this. It also means that women have an extra burden to become visible and act as part of that better direction for us. [4]

For companies, the choice is more personal. For me, Thoughtworks's efforts to improve its diversity are a major factor in why I've been an employee here for over a decade. I don't think it's a coincidence that Thoughtworks is also a company that has a greater open-mindedness, and a lack of political maneuvering, than most of the companies I've consulted with over the years. I consider those attributes to be a considerable competitive advantage in attracting talented people, and providing an environment where we can collaborate effectively to do our work.

But I'm not holding Thoughtworks up as an example of perfection. We've made a lot of progress over the decade I've been here, but we still have a long way to go. In particular we are very short of senior technical women. We've introduced a number of programs around networks, and leadership development, to help grow women to fill those gaps. But these things take time - all you have to do is look at our Technical Advisory Board to see that we are a long way from the ratio we seek.

Despite my knowledge of how far we still have to climb, I can glimpse the summit ahead. At a recent AwayDay in Atlanta I was delighted to see how many younger technical women we've managed to bring into the company. While struggling to keep my head above water as the sole male during a late night game of Dominion, I enjoyed a great feeling of hope for our future.