Diversity data: what self-reported figures can and cannot tell you

In Women in the Workplace 2024, the tenth anniversary report from McKinsey & Company and LeanIn.Org, the bar for "women overall" reads 81, the bar for Black women 54 and the bar for Asian women 99: same year, same step up to manager (McKinsey & Company and LeanIn.Org, 2024, p. 8). If you are handed a gender diversity dashboard, this post asks three things. What does the combined figure hide? Who reported the numbers, and what did they report? How far do a company's stated intentions match behaviour? Reading the page-8 chart as a chart is covered in five questions to ask before you trust a chart and in the chart-critique case.

What does the 81 hide once you split it by race?

It hides the distance between groups of women. At the step up to manager in 2024 the bars read 99 for Asian women, 89 for white women, 65 for Latinas and 54 for Black women, against 81 for women overall; each bar is a ratio to 100 men promoted (McKinsey & Company and LeanIn.Org, 2024, p. 8).

Source: (McKinsey & Company and LeanIn.Org, 2024, p. 5; McKinsey & Company and LeanIn.Org, 2024, p. 8). "Not printed" and "not split" mean the page has no such figure.

Page 6 adds that Asian men's share of the C-suite is more than double Asian women's, although the two groups' shares at entry level are virtually identical (McKinsey & Company and LeanIn.Org, 2024, p. 6). So the 99 describes one rung. Page 8 says Asian women, alongside Latinas, have the lowest promotion rates from director to VP (McKinsey & Company and LeanIn.Org, 2024, p. 8).

What did Crenshaw find when combined and split figures disagree?

That a combined figure can be accurate and still leave out the smallest group. Crenshaw (1989) argues that the "single-axis" view in antidiscrimination law erases Black women, because it looks only at the experience of the "otherwise-privileged" members of each group, and that the intersectional experience "is greater than the sum of racism and sexism" (Crenshaw, 1989, p. 140).

In DeGraffenreid v General Motors, GM had not hired Black women before 1964, and the Black women hired after 1970 all lost their jobs in a seniority-based layoff (Crenshaw, 1989, p. 141). The court reasoned that GM had hired women, albeit white women, in that period, so there was no sex discrimination for the seniority system to perpetuate (Crenshaw, 1989, p. 142). As I read it, a figure saying "GM hired women" was true at the time and did not answer the plaintiffs' question.

Moore v Hughes Helicopter shows the opposite problem: the split group is too small. In first-level supervisory jobs between 1976 and 1979, white women held 1.8 to 3.3 per cent and Black women 0 to 2.2 per cent (Crenshaw, 1989, p. 143). The court said that promoting just two Black women would have met the expected distribution, and in Crenshaw's account the sample was too small for the plaintiff to show disparate impact (Crenshaw, 1989, p. 146). The 2024 report reports individual race groups for only three groups, citing small samples (McKinsey & Company and LeanIn.Org, 2024, p. 62).

Who reported these figures, and what do they count?

Three sources, three different questions. Companies submit headcounts and promotions by gender and race, and inconsistent data were excluded as needed (McKinsey & Company and LeanIn.Org, 2024, p. 58). HR professionals answer for their company about policies and priorities (McKinsey & Company and LeanIn.Org, 2024, p. 57). More than 15,000 employees from 27 organisations took an optional survey (McKinsey & Company and LeanIn.Org, 2024, p. 60). As I classify them, only the first counts outcomes, and employees with unreported race are left out of the race tables (McKinsey & Company and LeanIn.Org, 2024, p. 62).

The third source records perception. Among Asian women, 39 per cent say their race has played a role in missing out on a raise, promotion or chance to get ahead, and the question begins "Have you ever felt…" (McKinsey & Company and LeanIn.Org, 2024, p. 31; McKinsey & Company and LeanIn.Org, 2024, p. 65). Beside the 99 for the same group, these are two different pictures: an outcome at one rung, and accumulated experience.

The methodology I read does not say how companies determine employees' race. Green and Hand classified the race of 6,931 S&P 500 executives from photographs and names, and say that no approach other than each executive's self-reported identification would achieve perfect accuracy (Green and Hand, 2024, p. 13).

How closely do a company's intentions match its managers' behaviour?

The report shows a gap but does not measure it directly. In 2024, 95 per cent of companies said they instructed managers to encourage a respectful and inclusive culture, while 54 per cent of employees said their managers encouraged respectful and inclusive behaviour on their team, meaning they had done so "consistently in the past year" (McKinsey & Company and LeanIn.Org, 2024, p. 18; McKinsey & Company and LeanIn.Org, 2024, p. 63). The respondents differ, so the gap is a hint, not a measurement.

Bias training is a second example: 86 per cent of companies offered it in 2024, yet the share of men who recognise microaggressions against women is 11 per cent in both 2019 and 2024 (McKinsey & Company and LeanIn.Org, 2024, p. 21). In 2019 the question asked whether people had heard or seen biased behaviour towards women; in 2024 it asked about "microaggressions" (McKinsey & Company and LeanIn.Org, 2024, p. 64). The line between the two years joins two different questions.

Webb and Sheeran (2006) pooled 47 randomised experimental tests and found that a medium-to-large change in intention (d = 0.66) led to a small-to-medium change in behaviour (d = 0.36) (Webb and Sheeran, 2006). That is research on individuals, not companies.

Does diversity raise profit, according to Green and Hand?

They found no evidence of it in their data, and they tested only the racial and ethnic diversity of executive teams. McKinsey's 2015 study reported that 61 per cent of US firms in the most diverse quartile outperformed on EBIT margin, against 41 per cent in the least diverse (Green and Hand, 2024, p. 7). Applying the same test to 497 S&P 500 firms, Green and Hand got 54.0 per cent against 51.2 per cent, p = 0.65 (Green and Hand, 2024, p. 8; Green and Hand, 2024, p. 10).

They also raise the direction of causality. McKinsey measured financial performance over the four or five years before the year it judged executives' race, so, in Green and Hand's reading, the default direction in the correlation is that good performance leads to diversity (Green and Hand, 2024, p. 25; Green and Hand, 2024, p. 26). The library's evidence check for "doing good pays" claims asks exactly this at its second gate: has reverse causation been controlled?

Their own limits: US firms only, S&P 500 firms are not a random sample, and race inferred from photographs and names is likely to undercount non-white executives (Green and Hand, 2024, p. 27). Their results do not speak to the social or moral contributions of diversity (Green and Hand, 2024, p. 28; Green and Hand, 2024, p. 29). This post does not conclude whether diversity does or does not raise profit.

What should you ask of a diversity dashboard?

Ask for the counts behind each ratio, which groups were merged or dropped, who reported, and whether a figure measures intention or outcome. The promotion rate, per page 58, is the promotions of one gender into a level divided by the number of that gender at the level below at the start of the year (McKinsey & Company and LeanIn.Org, 2024, p. 58). A hypothetical, with numbers I invented: a company has 300 non-managers, 200 men and 100 women, and last year 10 men and 4 women became managers. Men's rate is 5.0 per cent, women's 4.0 per cent, a ratio of 80.

Split the women by site. At head office, 60 women and 3 promoted give a ratio of 100; at provincial branches, 40 women and 1 promoted give 2.5 per cent, a ratio of 50. That 50 rests on exactly one person: one more promotion makes it 100. The split figure is a question, not a conclusion.

The last row is what Castilla (2015) did with almost 9,000 employees of a large company. Before the company introduced accountability and transparency procedures, women, ethnic minorities and non-US-born employees received lower performance-based rewards than US-born white men with the same evaluation scores, in the same job and work unit, with the same manager; afterwards the gap narrowed (Castilla, 2015). Dobbin and Kalev describe publishing each unit's average pay rise by race and gender, after which the raise gap between Black and white employees "all but disappeared" (Dobbin and Kalev, 2016).

The questions suit you if you hold promotion records person by person. They do not suit a company of a few dozen people: group rates then swing with each individual, so read the counts rather than the ratio; pooling several years is my suggestion, not something in the sources.

Going deeper: what did I read, and what did I not?

Women in the Workplace 2024 runs to 68 pages. I read the PDF pages 2 to 11, 13, 14, 16, 18, 19, 21, 23, 25, 26, 28, 31 and 46, plus the methodology and endnotes on pages 57 to 65; I did not read the rest closely. Page numbers are printed page numbers. McKinsey & Company wrote the report with LeanIn.Org and also authored the studies Green and Hand re-test; this post does not assess the report's conclusions, only how to read it.

For Crenshaw (1989) I read pages 139 to 151 closely, in the open PDF from Chicago Unbound; pages 152 to 167 I did not read closely. I read Green and Hand (2024) in full in the open PDF, apart from the appendix. For Webb and Sheeran (2006) and Castilla (2015) I had only the abstracts; I read Dobbin and Kalev (2016) in the full web version, not their underlying papers. I have no public source on Vietnamese companies' diversity data.