---
title: "Five questions to ask before you trust a chart"
description: "Five questions to check a chart before you use it: what is measured, who is in the data, does the picture match the numbers, how big, does it apply to you."
url: "https://staging.khoi.co.uk/en/blog/nam-cau-hoi-doc-bieu-do"
lang: en-GB
author: "Lê Quốc Khôi"
date: 2026-11-02
updated: 2026-11-02
topics: ["Data & analytics"]
tags: ["Data literacy", "Chart critique", "Data visualisation", "Base rate", "Selection bias"]
---

# Five questions to ask before you trust a chart, tried on a real report

## Key takeaways

- Five questions: what the chart measures and over what period; who is in the data; whether the picture matches the numbers; how big the number is and how sure we can be; and whether it supports the conclusion drawn and applies to you.
- Questions 1, 4 and 5 adapt Gigerenzer and colleagues' questions about health risks: risk of what, over what time, how big, does it apply to me (Gigerenzer et al., 2007, p. 58). Question 2 draws on the selection errors Jansen and colleagues describe (Jansen et al., 2022); question 3 on Tufte's lie factor (Tufte, 2001).
- Tried on the chart on page 8 of Women in the Workplace 2024: "81 women promoted per 100 men" reads, with the methodology on page 58, as a ratio of two promotion rates, computed as if both groups were the same size, and the page gives neither the underlying rates nor the numbers behind each bar (McKinsey & Company and LeanIn.Org, 2024, p. 8).
- The bars in that chart start from a common baseline and their heights match the printed numbers. Its weaknesses lie elsewhere: where the sample comes from, and how certain the figures are.
- The 92-versus-95 satisfaction score example is my own invention.

Page 8 of the Women in the Workplace 2024 report has a bar chart anyone can remember in one number: 81. For every 100 men promoted to manager, 81 women are. Suppose you head HR, tomorrow's meeting is about promotion policy, and that number is about to go on a slide. What should you ask first? This post offers five questions and tries each one on that chart. The report was produced by McKinsey & Company and LeanIn.Org (McKinsey & Company and LeanIn.Org, 2024); see also [the case on this report](https://staging.khoi.co.uk/en/case/mckinsey-leanin-women-in-the-workplace-2024). I am not assessing the report's conclusions, only asking whether the chart supports them.

## Which five questions, and where do they come from?

They come from the library's [chart critique and redesign framework](https://staging.khoi.co.uk/en/framework/chart-critique-and-redesign) and its [checklist of places where numbers mislead](https://staging.khoi.co.uk/en/framework/data-lie-checklist), which I have folded into five questions that are easy to remember.

> **Figure:** (steps) Five questions for reading a chart, combined by me from the library framework and the sources in this post.

```json
{
  "type": "steps",
  "phases": [
    {
      "label": "Five questions before you trust a chart",
      "steps": [
        { "title": "1. Measured what, when?", "body": "Metric name, unit, data period." },
        { "title": "2. Who is in the data?", "body": "Where the sample comes from, who is missing, how groups are pooled." },
        { "title": "3. Does the picture match?", "body": "Axes, scale, how it is drawn." },
        { "title": "4. How big, how sure?", "body": "Baseline rate, group sizes, uncertainty." },
        { "title": "5. Proves what, for whom?", "body": "Correlation or cause, and whether it resembles you." }
      ]
    }
  ]
}
```

In an interview about his book, Cairo says a data visualisation is an argument rather than an illustration, and that a chart shows only what it shows (Cairo, 2019).

## Question 1: what is measured, and over what period?

This is Gigerenzer and colleagues' "risk of what?" and "time frame?" (Gigerenzer et al., 2007, p. 58). For the "81" bar, the chart says it is the ratio of promotions to manager for women and men, 2018 to 2024, assuming equal numbers of each group (McKinsey & Company and LeanIn.Org, 2024, p. 8). The methodology defines a promotion rate as the promotions of one gender into a level, divided by the number of that gender at the level below at the start of the year (McKinsey & Company and LeanIn.Org, 2024, p. 58). So, in my reading, 81 is not 81 people but a ratio: women's promotion rate is 81 per cent of men's.

On timing, an endnote says each report's pipeline data come from the previous year, and that "2024" is the year of publication (McKinsey & Company and LeanIn.Org, 2024, p. 62). It adds that in the 2023 report some historical "broken rung" data were labelled by the year the data described rather than the year of publication (McKinsey & Company and LeanIn.Org, 2024, p. 62). Even a year label deserves a second look.

## Question 2: who is in the data, and who is not?

Jansen and colleagues put selection bias among the errors made when collecting data about people (Jansen et al., 2022). For this chart, the methodology says:

- The report is based on 281 companies in the US and Canada, which joined in response to invitations from McKinsey and LeanIn.Org or by showing interest through the public website (McKinsey & Company and LeanIn.Org, 2024, p. 57). The methodology also says that not all participating companies submitted talent pipeline data (McKinsey & Company and LeanIn.Org, 2024, p. 60).
- Each company's results were averaged by industry, then weighted by the 2023 Fortune 500 industry mix (McKinsey & Company and LeanIn.Org, 2024, p. 58). Page 58 describes the aim as to "better estimate trends over time based on each year's sample of companies" (McKinsey & Company and LeanIn.Org, 2024, p. 58).
- In its survey-data section, the report acknowledges that participating companies differ each year, so year-to-year comparisons may reflect sample characteristics or random variation (McKinsey & Company and LeanIn.Org, 2024, p. 60).

My reading of this: 81 is a weighted average over companies that opted in, so it cannot tell you how many companies sit near 81 and how many are far from it. Jansen and colleagues also list the "flaw of averages" among errors in analysis (Jansen et al., 2022).

The radial chart at the top of the page needs the same question. Its three rings read 59 per cent of bachelor's degrees, 51 per cent of the population and 48 per cent of entry-level employees (McKinsey & Company and LeanIn.Org, 2024, p. 8). Endnote 9 cites US Department of Education and Census Bureau statistics for the chart (McKinsey & Company and LeanIn.Org, 2024, p. 62). It does not cover the entry-level figure, which I infer comes from the participating companies' pipeline data in the US and Canada (McKinsey & Company and LeanIn.Org, 2024, p. 57). So the rings measure three different things, drawn as one sequence.

## Question 3: does the picture match the numbers?

Tufte calls the gap between picture and number the lie factor: the size of the effect shown in the graphic divided by the size of the effect in the data (Tufte, 2001). A hypothetical example, built by me: customer satisfaction scores of 92 at branch A and 95 at branch B, drawn as bars on an axis that starts at 90. Bar A is 2 units tall and bar B is 5, so B looks 150 per cent taller while the data differ by about 3.3 per cent. The lie factor is roughly 46.

The bar chart on page 8 passes this test. I measured bar heights in pixels on the PDF page image, taking the men's bar as 100. The labelled bar of each group (the last one, 2024) comes out within about one point of its printed value: 81, 89, 99, 54 and 65 for all women, white women, Asian women, Black women and Latinas. The Black women's 54 bar is about half the men's height. Two other things are worth noting. Only the "all women" group carries year labels, so readers must assume the other groups follow the same order; and there is no vertical axis, so you rely on the printed numbers.

The radial chart is harder to compare. By my rough check of the arc end points on the page image, each ring sweeps close to its percentage of a full turn (59, 51 and 48 per cent). But the rings have different radii, so arc length depends on the radius as well as the percentage. Read the printed labels, not the arc lengths.

## Question 4: how big is the number, and how sure are we?

Gigerenzer and colleagues advise expressing size in absolute terms, or relative to a more familiar risk (Gigerenzer et al., 2007, p. 58). Page 8 prints only the ratio, not the underlying promotion rates. A hypothetical test, not the report's figures: if 10 in every 100 eligible men are promoted in a year, a ratio of 81 means 8.1 in every 100 eligible women, a gap of 1.9 people. If the men's rate is 2 in 100, the women's is 1.6, a gap of 0.4. The same 81 describes two very different situations. The post on [relative and absolute risk](https://staging.khoi.co.uk/en/blog/rui-ro-tuong-doi-tuyet-doi) covers this trap in more depth.

On certainty: page 8 gives neither the number of companies nor the number of promotions behind each bar, and there are no error margins. The Black women's bars swing widely between years; by my reading of the bar heights, from about 54 to about 96 (McKinsey & Company and LeanIn.Org, 2024, p. 8). The page itself says that, after "notable improvements in 2021 and 2022", their rates "regressed to 2020 levels" this year (McKinsey & Company and LeanIn.Org, 2024, p. 8). The chart alone cannot show whether such swings exceed what a different sample each year could produce. For all women, the page says the gap is "not improving": 79 in 2018, 81 in 2024 (McKinsey & Company and LeanIn.Org, 2024, p. 8). My own reading is that a two-point difference between yearly samples is too small to call a trend from the chart alone.

## Question 5: does it prove what people want to conclude, and does it apply to me?

Page 8 says that, because of the broken rung, men significantly outnumber women at manager level, which makes it nearly impossible for companies to sustain progress at more senior levels (McKinsey & Company and LeanIn.Org, 2024, p. 8). The chart does not test that claim; it shows only the promotion ratio. Cairo advises against reading more into a chart than it shows, because readers tend to project their own beliefs onto it (Cairo, 2019).

Gigerenzer and colleagues' "does it apply to me?" asks whether the information rests on people like you (Gigerenzer et al., 2007, p. 58). The report draws on 281 companies in the US and Canada, which the "About the study" page says employ over 10 million people between them (McKinsey & Company and LeanIn.Org, 2024, p. 2). A company of 60 people in Can Tho needs its own promotion rates, by group, with the number of people attached.

## What do you ask the authors, and how would you redraw it?

Ask for three things: the two rates behind the ratio, the number of people or companies in each bar, and error margins where they exist. If they are missing, redraw the chart as counts per 100 people, one dot per person, or as dots with intervals instead of bars.

## Going deeper: what did I read, and what did I not?

I read the relevant pages of the PDF of Women in the Workplace 2024 (68 pages): pages 2, 5 and 8, and the methodology and endnotes on pages 57 to 62; I did not read all of it closely. Page numbers are printed page numbers. I do not have the underlying data, so I could not check the rates. For Gigerenzer and colleagues (2007) I read the risk-question box on page 58 and the sections on relative risk, not every page.

For Jansen and colleagues (2022) I read only the abstract, because I could not get the full text. For Tufte (2001) I do not have the book; I read the statement of the principle and the definition of the lie factor on a York University resource page, which takes its figures from the 1983 edition. For Cairo (2019) I read an interview he gave about the book (https://brown.columbia.edu/cairo-qa/), not the book itself.

The biggest limit: this post critiques presentation, not the research behind it. Judging the report's conclusions would mean reading the data and independent studies, which I have not done here.

## Frequently asked questions

### Must a bar chart always start at zero?

For bars, almost always, because the eye compares their lengths. Tufte states the principle that the numbers shown on the surface of a graphic should be directly proportional to the quantities they represent (Tufte, 2001); applied to bars, I read that as requiring a zero baseline. Cairo notes that online articles about misleading charts tend to focus on the most obvious example, a truncated y-axis on a bar chart, which he says is worth continuing to discuss, and that he wanted to go well beyond it (Cairo, 2019).

### Does this post say the Women in the Workplace 2024 conclusions are right or wrong?

No. It asks only what the chart on page 8 does and does not tell you (McKinsey & Company and LeanIn.Org, 2024, p. 8). A chart that lacks a baseline rate or an error margin can still describe reality accurately; those are separate questions. I also do not have the underlying data to recheck the figures.

### What if the report's authors cannot say who is in the data?

Treat the chart as a hypothesis, not as evidence for your decision. Its value then lies in the question it prompts about your own figures: who is eligible, how many people are promoted each year, how many are in each group. Gigerenzer and colleagues advise checking whether the information is based on people like you (Gigerenzer et al., 2007, p. 58).

## References

- Cairo, A. (2019) *How charts lie: getting smarter about visual information*. New York: W.W. Norton.
- Gigerenzer, G. et al. (2007) ‘Helping doctors and patients make sense of health statistics’, *Psychological Science in the Public Interest*, 8(2), pp. 53–96. https://doi.org/10.1111/j.1539-6053.2008.00033.x
- Jansen, B.J. et al. (2022) ‘The illusion of data validity: why numbers about people are likely wrong’, *Data and Information Management*, 6(4), article 100020. https://doi.org/10.1016/j.dim.2022.100020
- McKinsey & Company and LeanIn.Org (2024) *Women in the workplace 2024: the 10th-anniversary report*. McKinsey & Company and LeanIn.Org. Available at: https://www.mckinsey.com/featured-insights/diversity-and-inclusion/women-in-the-workplace-2024.
- Tufte, E.R. (2001) *The visual display of quantitative information*. Cheshire, CT: Graphics Press.

---

Canonical HTML version: https://staging.khoi.co.uk/en/blog/nam-cau-hoi-doc-bieu-do