Every AI system embeds the values of the people who built it, and I mean that literally. The data you train on, the objective you optimise for, and the way you define success all encode assumptions about what matters. Ignoring those assumptions doesn’t remove them; it just hides them.
Ethical AI development means making those assumptions explicit, examining them, and taking responsibility for what they cause.
Where bias enters
Bias gets into AI systems at several points, and it compounds across them.
The first is the training data. If your data over-represents some groups and under-represents others, the model performs better on the groups it saw more of. Facial recognition systems trained mostly on lighter-skinned faces perform worse on darker-skinned faces because of what they learned, without any bug involved. This has been documented in production systems.
Labels are the second. Human annotators bring their own assumptions to the task. Sentiment labels, content moderation labels, and relevance judgements all reflect the annotator’s cultural context, and annotator pools rarely represent the global population.
Third, feedback loops. A recommendation algorithm that optimises for clicks learns to recommend content that gets clicks, which is often sensational or inflammatory. The algorithm produces more of it, that data trains the next version of the model, and the loop amplifies the initial bias.
Fourth, proxy variables. A model that isn’t given a protected attribute like race or gender will often reconstruct it from correlated variables such as zip code, name, or browsing history. Removing the obvious variable doesn’t remove the bias.
Transparency and explainability
“The model said so” is not an acceptable explanation for consequential decisions like credit denial, hiring, medical triage, or bail recommendations. Yet many deployed models are black boxes that produce outputs with no account of how they got there.
Explainability is partly a technical problem and partly a governance problem.
On the technical side, there are a few established approaches:
- SHAP (SHapley Additive exPlanations) assigns each input feature a contribution score for a given prediction. It’s computationally expensive but theoretically grounded.
- LIME (Local Interpretable Model-agnostic Explanations) fits a simple interpretable model, such as a linear regression, around a specific prediction to approximate local behaviour.
- For transformer models, attention weights give some insight into which tokens the model attended to, though how far you can interpret them is debated.
- Decision trees, linear models, and rule-based systems are interpretable by construction. For high-stakes domains like medicine and criminal justice, there’s a strong argument for using them even at some cost in accuracy.
On the governance side, Datasheets for Datasets and Google’s Model Cards are documentation formats that describe what a model was trained on, what it’s for, its known limitations, and how it performs across demographic groups. Treating these as genuine communication and not a compliance checkbox changes the relationship between the people who build models and the people who deploy them.
Fairness metrics
There is no single definition of fairness, and in most real-world scenarios the definitions are mathematically incompatible. Being clear about which one you’re optimising for, and why, is itself an ethical choice. The common ones are:
- Demographic parity: the model’s positive prediction rate is equal across groups.
- Equal opportunity: the true positive rate is equal across groups.
- Predictive parity: precision (positive predictive value) is equal across groups.
- Individual fairness: similar individuals receive similar outcomes.
You can’t satisfy all of these at once unless base rates are equal across groups, which they rarely are. That’s no excuse to do nothing. It means you have to decide consciously which criterion matters most for your application.
Practical steps for developers
Audit your training data. What does the dataset contain? Who is represented and who isn’t? What labels were applied, by whom, and under what instructions? An audit before training is cheaper than an incident after deployment.
Break your evaluation metrics down by group. Aggregate accuracy hides differences in performance. Measure performance on demographic subgroups and put those numbers in your model cards.
Red team the system. Before deploying, actively try to make it fail in harmful ways. Adversarial testing is the only way to find failure modes that weren’t in your test set.
Collect only the data you need. If a feature isn’t necessary for the task, don’t collect it and don’t train on it. Data you don’t have can’t be misused.
Give affected users a way to push back. People harmed by AI systems often have no way to report it. A clear, accessible appeal or correction process is the ethical thing to do, and it’s also how you find problems.
Involve affected communities early. Participatory design, user research with under-represented groups, and consultation with domain experts (especially in healthcare and criminal justice) surface concerns developers can’t anticipate from the inside.
Accountability
Technical fixes are necessary but not sufficient. Bias in AI systems often reflects bias in the institutions deploying them. A fairer hiring algorithm doesn’t fix a discriminatory hiring culture; it changes where the discrimination happens.
Real accountability needs:
- Clear ownership of decisions made by AI systems
- Legal and institutional frameworks that assign liability for harms caused by AI
- Whistleblower protection for engineers who raise concerns internally
- Regulation that sets minimum standards for high-stakes applications
These are political and organisational problems as much as technical ones. Developers who care about ethical AI need to engage with them in their own organisations, in public debate, and in the standards bodies and regulatory processes that will set the rules. The technology isn’t neutral, and neither are the people building it.