Framework

Four layers of algorithmic bias

The question it answers

Before an HR manager adopts an algorithm to screen job applicants, in what ways could it discriminate against candidates, and at which point can each bias be fixed?

Origin
Binns, 2018; Zimmermann et al., 2020; Dattner et al., 2019
Diagram type
Process by phase
Level of analysis
Firm
Topics
AI for leadersData & analytics
Glass sphere with a glowing neural-network core, icon for AI for leaders

Diagram

Four layers of algorithmic bias

  1. Layer 1
  2. Direct discrimination and statistical biasSkewed training data; fix with more representative data.
  3. Layer 2
  4. Indirect discrimination through proxiesDropping the sensitive column is not enough because other variables rebuild it (postcode, interests, arrest history).
  5. Layer 3
  6. Structural injusticeThe data is a product of unjust institutions; feedback loops make it worse.
  7. Layer 4
  8. Competing conceptions of fairnessNot every metric can be met at once; choose deliberately and transparently.

Sources

References

Listed A–Z by author surname, Harvard style. A source with a DOI links to https://doi.org/…; one without links to its page or says plainly that it has no DOI.

  1. Binns, R. (2018) ‘Fairness in machine learning: lessons from political philosophy’, in Proceedings of Machine Learning Research. pp. 149–159.No DOIproceedings.mlr.pressSource page
  2. Dattner, B. et al. (2019) The legal and ethical implications of using AI in hiring. Harvard Business Review.No DOIhbr.orgSource page
  3. Zimmermann, A., Di Rosa, E. and Kim, H. (2020) Technology can't fix algorithmic injustice. Boston Review.No DOIbostonreview.netSource page