Fairness and Bias in Artificial Intelligence: 2026 Guide

No AI system can be fair by every definition at the same time. Kleinberg, Mullainathan, and Raghavan(2016) and Chouldechova (2017) provide that calibration, equal false positive rates, and equal false negative rates cannot all hold at once unless base rates are identical across groups or the model is perfect. Every AI system in use makes an implicit choice about which group absorbs its errors. This guide includes an overview of AI bias, the eight different kinds of bias found in production systems, the fairness measures one can compute, the legal position in the EU and the US until 2026, and the seven stages of a bias audit.

What Is Bias in Artificial Intelligence?

Bias in Artificial Intelligence is a systematic error in the AI model that yields worse results for one particular group of people compared to another group. It recurs predictably each time the model runs, unlike random error. 

The difference between bias and algorithmic bias comes from their scope. Algorithmic bias is a general term for any automated systems that yield skewed results. Bias in AI is a systematic error in a machine learning model trained on some data set. The third term, statistical bias, is a discrepancy between an estimator and the true value.

Bias can be measured, but fairness cannot, because fairness has no single technical definition. 

Where AI Bias Comes From: Data, Algorithms, and People

AI bias creeps in from three entry points: training data, algorithms, and humans. All three correspond to specific phases of the machine learning lifecycle.

• Data bias refers to the distortion of the population represented in the training data. Example: the IJB-A facial analysis benchmark was 79.6% light-skinned subjects, while the Audience benchmark was 86.2% lighter-skinned subjects (Buolamwini and Gebru, 2018). 
• Algorithm bias refers to the accentuation of the data pattern during the model development. Example: use of an objective function aimed at minimizing healthcare costs rather than minimizing the disease, use of a loss function that does not care about minority-class errors, feature selection with the inclusion of proxy features.
• Human bias occurs due to the influence of subjective perception on model development. Example: writing labels with assumptions of a single group, using annotation guidelines with the assumption of one cultural context, choosing thresholds without testing them on subgroups.

Any bias entering the process at any of the six phases of the lifecycle surfaces in the output. 

The 8 Types of AI Bias

Eight bias types appear repeatedly in production AI systems. Each type has a distinct mechanism and a distinct fix.

Bias type	Mechanism	Documented example
Sampling bias	Training data does not represent the deployment population	Facial analysis benchmarks composed of 79.6–86.2% lighter-skinned subjects
Measurement bias	The measured variable differs from the target variable	Healthcare cost measured as a stand-in for illness severity
Algorithmic bias	Model design or objective function amplifies a data pattern	Recruitment models weighting features correlated with historical hires
Confirmation bias	The system reinforces existing patterns and misses new ones	Recommendation loops that narrow to prior behavior
Reporting bias	Event frequency in training data differs from real-world frequency	Sentiment models trained on review data skewed toward positive ratings
Automation bias	Human reviewers accept machine output despite higher error rates	Operators approving flagged matches without independent checks
Group attribution bias	The model assigns group characteristics to individuals	Resume screeners favoring candidates sharing attributes with training examples
Generative bias	Output content over-represents one demographic	Text-to-image models producing men for “CEO” prompts

What Is Fairness in Artificial Intelligence?

AI fairness is the lack of systematic favor or disfavor for people or categories of people defined by a protected characteristic, which can include race, gender, age, disability, or national origin. Fairness is an explicit design objective. Bias is an unintentional consequence.

Fairness has no one-size-fits-all technical definition. More than two dozen have been defined by researchers. This is not a void in the literature. Each definition represents a unique set of values, and the selection among them is a matter of policy.

Group, Individual, Counterfactual, Procedural, and Causal Fairness

Five types of fairness cover the definitions used today. 

1. Group fairness is the type of fairness that entails the even or proportional allocation of outcomes among demographic groups. The measures here include demographic parity, equal opportunity, and equalized odds.
2. Individual fairness is the type of fairness where similar individuals get similar outcomes as measured through distance-based similarity functions (Dwork et al., 2012). This one is agnostic to any grouping.
3. Counterfactual fairness is the type of fairness where a decision remains the same even if there was a change in an individual’s protected attributes while keeping everything else constant.
4. Procedural fairness involves documentation and auditing of the decision-making process as well as the appeal of the process irrespective of the outcome distribution.
5. Causal fairness involves making decisions based on causal relationships and not historical correlations. Here the aim is to differentiate predictors from inherited inequalities.

These definitions of fairness often conflict with one another. 

Fairness Metrics You Can Actually Calculate

Four group fairness metrics can be computed from a confusion matrix. Each answers a different question.

Metric	Requirement	Question it answers
Demographic parity	Equal selection rate across groups	Do groups receive positive outcomes at the same rate?
Equal opportunity	Equal true positive rate across groups	Among qualified people, are groups approved at the same rate?
Equalized odds	Equal true positive rate and equal false positive rate	Are both error types balanced across groups?
Predictive parity (calibration)	Equal precision across groups	Does the same score mean the same thing for every group?

Worked Example. A lending model reviews 1,000 applicants from Group A and 1,000 from Group B. Demographic parity fails, since selection rates are 40% and 25%. Suppose that 500 of applicants from Group A and 313 of applicants from Group B were going to pay back their loans and model selected 320 and 200 correspondingly.Both groups have a 64% true positive rate. Equal opportunity passes the test.

Why You Can’t Satisfy Every Fairness Definition at Once

Calibration, equal false positive rates, and equal false negative rates cannot hold simultaneously unless base rates are equal across groups or the classifier is perfect. This was proved independently by Kleinberg, Mullainathan, and Raghavan in 2016. Chouldechova gave another proof for the recidivism case in 2017.

The controversy around COMPAS provides evidence. ProPublica studied COMPAS risk scores for more than 7,000 defendants in Broward County, Florida, in 2016. Of the defendants who had not reoffended in two years, 45% of the Black defendants were scored high risk and 23% of the white defendants. Of those who had reoffended, 48% of white defendants and 28% of Black defendants were scored low risk.

The company, Northpointe, replied that the scores of COMPAS were calibrated, meaning that the scores indicated the same reoffending frequency across race. For those with scores 7, 60% of white defendants and 61% of Black defendants reoffended.

Both analyses were mathematically sound. ProPublica looked at the error rate balance, while Northpointe was concerned with calibration. Since the reoffending base rates were different between the two classes of the data, the impossibility theorem proves that no scoring scheme could satisfy both.

The operational implication is clear. The choice of a fairness metric decides which group bears the errors of the model. In systems without documented metric selection, that decision was made but not recorded.

Bias vs Fairness in AI: What’s the Difference?

Bias is a measurable property of a model’s output. Fairness is a target the model is designed to meet. Nine dimensions separate them.

Dimension	Bias	Fairness
Definition	Systematic deviation of output from the true value	Absence of discrimination based on protected characteristics
Intent	Usually unintentional	Always deliberate
Nature	Technical property	Ethical and legal target
Measurement	Directly measurable from outputs	Measurable only after selecting a definition
Objective	Reduce systematic error	Distribute outcomes according to a chosen standard
Detection	Statistical testing on model outputs	Audit against a stated fairness criterion
Methods	Data pre-processing, algorithmic adjustment, threshold tuning	Fairness-aware training, constrained optimization, post-processing
Directionality	Can favor or disadvantage a group	Concerned only with disadvantage
Accountability	Attaches to the model developer	Attaches to the developer and the deployer separately under 2026 law

Real Examples of AI Bias (and What Actually Happened Next)

Five documented cases show measured bias and the corrective action that followed.

Gender Shades (2018). Buolamwini and Gebru analyzed three commercial gender classifiers of IBM, Microsoft, and Face++. The error rate for dark-skinned women was 34.7%. The maximal error rate for light-skinned men was 0.8% — a gap of more than 40-fold. In 2020, IBM removed its general facial recognition technology from the market.

Healthcare risk scoring (2019). Obermeyer, Powers, Vogeli, and Mullainathan analyzed in Science a commercial algorithm applied to roughly 200 million people annually in the US healthcare system. With the same risk score, Black patients were objectively sicker than white patients. Black patients constituted 17.7% of patients who should be referred to additional care services; with correction of bias, their proportion increased to 46.5%. The reason for this was the objective function of the algorithm, which predicted healthcare expenses — and at the same level of illness, less money is spent on Black patients. Developers confirmed the findings and redesigned the algorithm.

COMPAS (2016). Discussed above. Broward County study involving more than 7,000 defendants; 45% and 23% false positive rates.

Amazon recruitment tool (2017). According to Reuters, in October 2018, Amazon decided to discontinue its internal software that analyzed resumes because it was biased against resumes that mentioned the term “women’s” and negatively scored candidates who graduated from two women’s colleges. The software was developed using 10 years of application data.

NIST FRVT Part 3 (2019). NIST tested 189 algorithms from 99 developers across 18 million images. False positive differentials between demographic groups ranged from a factor of 10 to 100 depending on the algorithm. False negative differentials varied by a factor of about 2. Algorithms with the highest overall accuracy showed the smallest demographic gaps.

Why Deleting the Race Column Doesn’t Fix Bias

The exclusion of protected attributes from training data does not reduce bias, since other variables still carry the same information. This method is known as fairness through unawareness. It will fail for models with enough capacity.

Some examples of proxy variables are:

• Postal code – related to race and income in socially segregated geographic areas
• Surname – related to ethnicity and national origin
• Educational institution – related to race, income, and geography
• Browsing and purchasing history – related to gender, age, and income
• Type of device and operating system – related to income
• Labor gaps – related to gender through caregiving

The Obermeyer case serves as an example. Race was not among the input parameters. But there was another parameter: healthcare costs. Since access to healthcare is unequal, fewer costs are incurred for Black patients at the same level of illness.

Removing the proxy variable outright is not viable either, since doing so eliminates the signal the model needs to predict the outcome. The only way to proceed is to measure them by keeping the attributes for audits, but excluding them from training and testing the output.

How to Audit an AI System for Bias

The bias audit process involves seven stages. Each stage results in an artifact being created.

1. Specify protected attributes. State the attributes that fall under the purview of the laws and the application environment. State intersections rather than individual attributes.
2. Specify the fairness metrics to be used and provide the rationale for selection. Possible fairness metrics are demographic parity, equal opportunity, equalized odds, or predictive parity. Provide the rationale for selecting the specified metric given the harm profile of the application context.
3. Specify a disparity threshold. Popular disparities are the four-fifths rule, with a selection rate ratio above 0.80, and rate differences with a specified tolerance.
4. Slice the test data set. Calculate the metrics separately for each group and for each intersection. In Gender Shades, the maximum disparity was at the intersection of dark skin color and gender (female).
5. Record measurements. Generate per-group confusion matrices, selection rates, and metric scores together with the number of samples.
6. Mitigate and re-measure. Deploy a mitigation technique and repeat stages 4 and 5. Record the difference in the accuracy score.
7. Monitor in production. The data distribution changes after deployment. Schedule repeated measurement and define thresholds for alerts.

Bias Mitigation: Pre-Processing, In-Processing, and Post-Processing

Mitigation methods apply at three points in the pipeline, and each carries a different cost.

Stage	Methods	Cost
Pre-processing	Reweighting, resampling, synthetic data generation, relabeling	Requires access to training data; may reduce representativeness
In-processing	Fairness constraints in the objective function, adversarial debiasing, regularization	Requires retraining; typically reduces overall accuracy
Post-processing	Group-specific decision thresholds, calibrated score adjustment, equalized odds correction	Applies different thresholds by group, which some jurisdictions restrict

Buolamwini and Gebru demonstrated pre-processing gains directly: oversampling darker-skinned subjects improved facial classification accuracy for that group.

What These Fixes Can’t Do

Every mitigation method carries a documented limitation.

• Trade-off between accuracy and fairness. Constraining a model to satisfy a fairness metric typically reduces its accuracy relative to the unconstrained version. The extent of this depends on the distance between the unconstrained model and the constraint.
• Intersectional blindness. Metrics for group fairness when calculated using individual attributes may pass, but fail for an intersectional sub-group. Gender Shades provides a classic example. The single-attribute metrics, based on either gender or skin color separately, underestimated the 34.7% error rate for dark-skinned women. 
• Adverse consequences. Ferrara (2024) reports instances where efforts to mitigate biases in predictive policing systems led to greater racial disparities in the outcomes of arrests.
• Instability of definitions. Fairness criteria vary with geography, field, and time. A fair model by one definition does not imply fairness by another.

The limitations determine when an AI system should not be deployed. That decision is a documented output of the audit, not a failure of it.

Tools for Testing AI Fairness

Four open-source toolkits cover the standard fairness testing requirements.

Tool	Maintainer	Use case
AI Fairness 360 (AIF360)	LF AI (moved from IBM, July 2020)	Broadest metric library; Python and R; 70+ fairness metrics
Fairlearn	Microsoft	Constrained optimization and post-processing; integrates with Azure ML
Aequitas	Center for Data Science and Public Policy, University of Chicago	Audit reporting for policy and public-sector contexts
What-If Tool	Google PAIR	Interactive counterfactual inspection inside Jupyter and Colab

AI Fairness Laws and Regulations in 2026

Fairness obligations tightened between 2021 and 2025, then were partially deferred in 2026. The current position across four jurisdictions:

Jurisdiction	Instrument	Status as of August 2026
European Union	EU AI Act (Reg. 2024/1689), amended by the Digital Omnibus on AI (Reg. 2026/1744)	Omnibus published in the Official Journal 24 July 2026, in force 27 July 2026. Annex III high-risk obligations deferred from 2 August 2026 to 2 December 2027. Annex I embedded systems deferred to 2 August 2028. Article 5 prohibitions (in force February 2025), Article 50 transparency duties, and GPAI obligations (in force August 2025) are unchanged.
Colorado	Colorado AI Act (SB 24-205), repealed and replaced by SB 26-189	SB 26-189 signed 14 May 2026, effective 1 January 2027. The duty of care against algorithmic discrimination, deployer risk-management programs, and impact assessments were removed and replaced with disclosure requirements for automated decision-making technology.
New York City	Local Law 144 of 2021	In force. Enforced by DCWP since 5 July 2023. Requires annual independent bias audits of automated employment decision tools, public posting of results, and 10 business days’ candidate notice. Penalties: $500 for a first violation, $500– $1,500 for each subsequent violation, with each day treated as a separate violation.
California / Illinois	Civil Rights Council ADS regulations; HB 3773	California ADS regulations effective 1 October 2025. Illinois HB 3773 amends the Human Rights Act for AI in employment decisions.

Two voluntary frameworks apply regardless of jurisdiction. NIST AI RMF 1.0, released on 26 January 2023, contains the Govern-Map-Measure-Manage framework adopted by the majority of American enterprise programs. ISO/IEC 42001, issued in December 2023, serves as an AI management system standard available for certification.

Penalties under EU AI Act have not been amended: €35 million or 7% of global turnover for prohibited activities; up to €15 million or 3% of the global turnover for high-risk failure to comply; and up to €7.5 million or 1% of the global turnover for providing incorrect information.

Who’s Responsible: Developer or Deployer?

The EU AI Act assigns separate obligations to providers and deployers, and the split determines liability.

• Providers (developers that put a system out there on the market) have duties specified in Articles 9–17: risk management, data governance, technical documentation, logging, transparency, design of human oversight, and accuracy and robustness testing.
• Deployers (companies using the system) have duties of Article 26: running the system in accordance with instructions, designating competent human oversight, monitoring the operation, and logging.

If a deployer significantly modifies a system or markets it in their name, they become providers and assume provider duties. In NYC Local Law 144, the obligation is imposed on an employer rather than on a vendor providing the tool.

Fairness in Generative AI and Large Language Models

Generative models require different fairness tests than classifiers, because their outputs are unbounded text and images rather than labeled decisions.

Known results in relation to text-to-image models include stereotype generation; for example, when prompted with “CEO,” the model produced mostly male individuals, whereas when asked about “criminal” or “terrorist,” it generated more people of color through Stable Diffusion, DALL·E, and Midjourney (Ferrara, 2024).

Four test types can be used for large language models:

1. Name-swap testing. Send identical prompts but changing only the name corresponding to some demographic group. Assess the difference in sentiment, length, and recommendation provided.
2. Refusal-rate measurement. Assess whether the refusal rates depend on the demographic content in the prompt.
3. Retrieval bias testing. If the system uses some retrieval technique, assess whether the retrieved documents depend on the demographic frame of the query.
4. Annotator-pool review. Reinforcement learning from human feedback learns the preferences of the annotators pool. Document the demographic characteristics of this pool.

Frequently Asked Questions

Conclusion

Each model already has a definition of fairness, whether or not that choice was documented. The impossibility theorem makes it impossible to be compliant with all definitions, leaving only one approach available: document what metric is controlling the model, the threshold, the errors that metric allows, and the justification for those choices.

The deferral in 2026 from the EU and Colorado has changed the deadline for compliance, but it hasn’t changed either the math or the discrimination prohibition laws as applied to outcomes irrespective of how those outcomes were generated. Organizations that have made that choice today will be able to claim that documentation when the deferral deadline is reached in December 2027.