Framework

Chart critique and redesign

The question it answers

When a manager receives a chart in a report ahead of a decision, how can they tell whether it is a reliable basis, and how should they ask for it to be redrawn?

Origin
Tufte, 2001; Cairo, 2019; Gigerenzer et al., 2007
Diagram type
Process by phase
Level of analysis
Individual
Topics
Data & analyticsBusiness consulting
Magnifying lens over a line chart tile, icon for data and analytics

Diagram

Chart critique and redesign

  1. Read
  2. What does the chart claimSeparate the claim from the design; read footnotes and sample sizes.
  3. Test its strength
  4. Does it pool unlike groupsA cross-industry average may not fit your industry.
  5. Are there confidence intervalsSmall groups without error bars are falsely precise.
  6. Perception or outcomePerception surveys differ from objective metrics; put them side by side.
  7. Where is the base rateA big multiplier can hide a small absolute level.
  8. Redraw
  9. Pick a more honest formDot plots with intervals, paired panels, natural-frequency dot grids.
  10. Pick the toolInteractive dashboards with filters; Python for the groundwork.
  11. Recommend
  12. Data to ask forStructural metrics, behavioural data, and data over time.

Pitfalls

Pitfall

Critiquing without proposing a redesign; concluding the authors' intent when only the framing can be shown.

Sources

References

Listed A–Z by author surname, Harvard style. A source with a DOI links to https://doi.org/…; one without links to its page or says plainly that it has no DOI.

  1. Cairo, A. (2019) How charts lie: getting smarter about visual information. New York: W.W. Norton.No DOISource page
  2. Gigerenzer, G. et al. (2007) ‘Helping doctors and patients make sense of health statistics’, Psychological Science in the Public Interest, 8(2), pp. 53–96.doi:10.1111/j.1539-6053.2008.00033.xSource page
  3. Tufte, E.R. (2001) The visual display of quantitative information. Cheshire, CT: Graphics Press.No DOISource page