Algorithmic Bias: What It Is and How the Machine Reproduces Our Past

You applied for a mortgage and were rejected within seconds by an algorithm that never saw your face. Then you learned that your neighbour, with the same income and credit history, was approved because his postcode lies in another district. Was the decision truly neutral? This is the essence of algorithmic bias: a machine we imagine to be free of passions, yet which learns from our records, weighed down by exclusion and prejudice.

Where Does It Come From?

Machine-learning systems are not programmed with explicit rules; they draw their patterns from historical data. If that data reflects a society that once discriminated against some of its own people, the model picks up this bias as a statistical "truth". The "Gender Shades" study (2018) showed that three commercial systems for classifying gender from face images were nearly perfect with lighter-skinned men (error up to 0.8%), while misclassifying darker-skinned women at rates as high as 34.7%. The cause is likely the weak representation of these women in the commonly used test and training sets, which were dominated by lighter skin (about 80% and 86% in the two sets the researchers examined). The algorithm was not racist by hidden intent; it simply failed to "see" those absent from its data.

More dangerous than the error is the aura of neutrality: the machine's rejection is wrapped in scientific legitimacy that makes objecting to it like objecting to the law of gravity, so discrimination becomes a mathematical output for which no one is answerable.

Examples from Reality

• Criminal justice: the risk-assessment tool COMPAS provoked controversy after a journalistic investigation (ProPublica, 2016) held that it erred at different rates depending on the defendant's race: Black defendants who did not reoffend were classed as high risk at 44.9%, against 23.5% for white defendants. Its developer and other researchers objected to the method, and this very disagreement is the subject of the second point below.
• Hiring: Reuters (2018) reported that one of the largest companies abandoned a CV-screening tool because it had learned to downgrade CVs indicating that the applicant was a woman, since past hiring data was mostly male; the company said its staff never relied on it alone.

Three Truths the Common Talk Overlooks

First: removing the sensitive attribute is not enough. Algorithms are skilled at finding "proxy variables": a postcode may stand in for race, and spending patterns for class (Dwork et al., 2012). Fairness may require that we measure and openly disclose the differences between groups, then address them with safeguards approved by specialists and reviewed by humans, rather than turning a blind eye; this point is disputed among researchers.

Second: there is no single mathematical measure of fairness. Jon Kleinberg and colleagues proved in 2016 that three reasonable fairness conditions for risk scores (calibration within each group, balance of errors for the positive class, and balance for the negative class) cannot all hold together unless prediction is perfect or the base rates are equal between the two groups. So choosing which measure we want, and at what cost, is a value-laden and political decision, not a purely engineering one. Justice in our Sharia is the ultimate reference; the measures are tools to measure against it, not a substitute for it.

Third: the feedback loop. If a predictive policing algorithm concentrates patrols in a particular district (Lum and Isaac, 2016), arrest figures there rise because the watching eye is greater, and the model then sees in the figures a confirmation of its first bias. A slight bias thus becomes structural discrimination over successive cycles.

How Do We Address It?

1. Diversify the data and audit its sources before training.
2. Explainability: the decision should not remain a black box.
3. Governance and oversight: independent audit, a human in the loop and teams of varied backgrounds, covered in the article "Responsible Artificial Intelligence".

The risk doubles with Arabic: the data used to train the major models has been dominated by English and Western culture, so linguistic and cultural bias seeps into our context (Naous et al., 2024), touching our social and religious particularities; this is covered in the article "Arabic Artificial Intelligence".

Algorithmic bias is not a passing flaw that disappears as processors advance, but a civilisational question about the relation between justice and the machine. A biased algorithm may at times be a harsh mirror that exposes social ills we had ignored. Let us make of this mirror a beginning of reform, not a pretext for surrender, and build tomorrow on vigilant ethical oversight; what counts is not the machine's intelligence but the wisdom of those who build it and those who hold it to account.