Bias Assessment Framework

When to Use This Skill

Use this skill when:

• Bias Assessment tasks - Working on evaluate ai systems for fairness using demographic parity, equalized odds, and bias detection techniques with mitigation strategies
• Planning or design - Need guidance on Bias Assessment approaches
• Best practices - Want to follow established patterns and standards

Overview

Bias assessment systematically evaluates AI systems for unfair treatment across demographic groups. Effective assessment requires defining fairness criteria, measuring disparities, and implementing mitigations while documenting trade-offs.

Fairness Definitions

Fairness Metric Taxonomy

┌─────────────────────────────────────────────────────────────────┐
│                    FAIRNESS METRICS                              │
├─────────────────────────────────────────────────────────────────┤
│                                                                  │
│  GROUP FAIRNESS (Statistical Parity)                            │
│  ├── Demographic Parity: P(Ŷ=1|A=0) = P(Ŷ=1|A=1)               │
│  ├── Equalized Odds: P(Ŷ=1|Y=y,A=0) = P(Ŷ=1|Y=y,A=1)           │
│  ├── Equal Opportunity: P(Ŷ=1|Y=1,A=0) = P(Ŷ=1|Y=1,A=1)         │
│  └── Predictive Parity: P(Y=1|Ŷ=1,A=0) = P(Y=1|Ŷ=1,A=1)         │
│                                                                  │
│  INDIVIDUAL FAIRNESS                                             │
│  ├── Similar individuals → Similar predictions                  │
│  └── Counterfactual fairness                                    │
│                                                                  │
│  CAUSAL FAIRNESS                                                 │
│  ├── No direct discrimination                                   │
│  └── No indirect discrimination via proxies                     │
│                                                                  │
└─────────────────────────────────────────────────────────────────┘

Metric Selection Guide

Metric	Use When	Limitation
Demographic Parity	Equal outcomes needed	Ignores base rates
Equalized Odds	Equal error rates needed	Requires labels
Equal Opportunity	Focus on true positives	Ignores false positives
Predictive Parity	Equal precision needed	May conflict with others
Calibration	Probabilistic predictions	Hard to achieve with others

Impossibility Theorem

Note: It is mathematically impossible to satisfy all fairness metrics simultaneously when base rates differ across groups. Document trade-off decisions explicitly.

Protected Attributes

Common Protected Categories

Category	Examples	Legal Framework
Race/Ethnicity	Self-identified, inferred	Title VII, Civil Rights Act
Gender	Binary, non-binary, gender identity	Title VII, Equal Pay Act
Age	Date of birth, age ranges	ADEA
Disability	Physical, mental, cognitive	ADA
Religion	Affiliation, practices	Title VII
National Origin	Country, citizenship, ancestry	Title VII
Sexual Orientation	Preference, identity	Varies by jurisdiction
Socioeconomic Status	Income, education, zip code	Often proxy for race

Proxy Variable Detection

public class ProxyDetector
{
    public async Task<ProxyAnalysis> DetectProxies(
        Dataset data,
        string protectedAttribute,
        CancellationToken ct)
    {
        var proxies = new List<ProxyVariable>();
        var features = data.GetNonProtectedFeatures();

        foreach (var feature in features)
        {
            // Calculate mutual information with protected attribute
            var mutualInfo = CalculateMutualInformation(
                data.GetColumn(feature),
                data.GetColumn(protectedAttribute));

            // Calculate correlation
            var correlation = CalculateCorrelation(
                data.GetColumn(feature),
                data.GetColumn(protectedAttribute));

            if (mutualInfo > 0.3 || Math.Abs(correlation) > 0.5)
            {
                proxies.Add(new ProxyVariable
                {
                    FeatureName = feature,
                    MutualInformation = mutualInfo,
                    Correlation = correlation,
                    RiskLevel = ClassifyRisk(mutualInfo, correlation)
                });
            }
        }

        return new ProxyAnalysis
        {
            ProtectedAttribute = protectedAttribute,
            DetectedProxies = proxies,
            Recommendations = GenerateRecommendations(proxies)
        };
    }

    private RiskLevel ClassifyRisk(double mi, double corr)
    {
        var maxSignal = Math.Max(mi, Math.Abs(corr));
        return maxSignal switch
        {
            > 0.7 => RiskLevel.High,
            > 0.5 => RiskLevel.Medium,
            _ => RiskLevel.Low
        };
    }
}

Bias Measurement

Disparate Impact Analysis

public class FairnessEvaluator
{
    public FairnessReport Evaluate(
        IEnumerable<Prediction> predictions,
        string protectedAttribute)
    {
        var groups = predictions.GroupBy(p => p.GetAttribute(protectedAttribute));

        var metrics = new Dictionary<string, GroupMetrics>();

        foreach (var group in groups)
        {
            var groupPredictions = group.ToList();

            metrics[group.Key] = new GroupMetrics
            {
                Group = group.Key,
                Count = groupPredictions.Count,
                PositiveRate = groupPredictions.Count(p => p.Predicted == 1)
                    / (double)groupPredictions.Count,
                TruePositiveRate = CalculateTPR(groupPredictions),
                FalsePositiveRate = CalculateFPR(groupPredictions),
                Precision = CalculatePrecision(groupPredictions)
            };
        }

        return new FairnessReport
        {
            GroupMetrics = metrics,
            DemographicParity = CalculateDemographicParity(metrics),
            EqualizedOdds = CalculateEqualizedOdds(metrics),
            DisparateImpact = CalculateDisparateImpact(metrics),
            Recommendations = GenerateRecommendations(metrics)
        };
    }

    private double CalculateDisparateImpact(Dictionary<string, GroupMetrics> metrics)
    {
        var rates = metrics.Values.Select(m => m.PositiveRate).ToList();
        var minRate = rates.Min();
        var maxRate = rates.Max();

        // Disparate impact ratio (4/5ths rule: should be > 0.8)
        return minRate / maxRate;
    }

    private EqualizedOddsResult CalculateEqualizedOdds(
        Dictionary<string, GroupMetrics> metrics)
    {
        var tprValues = metrics.Values.Select(m => m.TruePositiveRate).ToList();
        var fprValues = metrics.Values.Select(m => m.FalsePositiveRate).ToList();

        return new EqualizedOddsResult
        {
            TprDisparity = tprValues.Max() - tprValues.Min(),
            FprDisparity = fprValues.Max() - fprValues.Min(),
            Satisfied = tprValues.Max() - tprValues.Min() < 0.1
                && fprValues.Max() - fprValues.Min() < 0.1
        };
    }
}

LLM-Specific Bias Testing

public class LlmBiasTester
{
    public async Task<LlmBiasReport> TestBias(
        ILlmClient llm,
        BiasTestSuite testSuite,
        CancellationToken ct)
    {
        var results = new List<BiasTestResult>();

        foreach (var testCase in testSuite.TestCases)
        {
            // Generate variations with different demographic references
            var variations = GenerateDemographicVariations(testCase);
            var responses = new Dictionary<string, string>();

            foreach (var (demographic, prompt) in variations)
            {
                var response = await llm.Complete(prompt, ct);
                responses[demographic] = response;
            }

            // Analyze response consistency
            var analysis = AnalyzeResponseVariation(responses, testCase.ExpectedConsistency);

            results.Add(new BiasTestResult
            {
                TestCase = testCase,
                Responses = responses,
                ConsistencyScore = analysis.ConsistencyScore,
                BiasIndicators = analysis.BiasIndicators,
                Passed = analysis.ConsistencyScore >= testCase.Threshold
            });
        }

        return new LlmBiasReport
        {
            TestResults = results,
            OverallScore = results.Average(r => r.ConsistencyScore),
            FailedTests = results.Where(r => !r.Passed).ToList(),
            Recommendations = GenerateRecommendations(results)
        };
    }

    private Dictionary<string, string> GenerateDemographicVariations(BiasTestCase testCase)
    {
        var variations = new Dictionary<string, string>();

        // Gender variations
        variations["male"] = testCase.Template.Replace("{person}", "John");
        variations["female"] = testCase.Template.Replace("{person}", "Sarah");

        // Race/ethnicity variations (when appropriate for test)
        if (testCase.TestDemographics.Contains("race"))
        {
            variations["name_a"] = testCase.Template.Replace("{person}", "James");
            variations["name_b"] = testCase.Template.Replace("{person}", "Jamal");
            variations["name_c"] = testCase.Template.Replace("{person}", "Jose");
        }

        return variations;
    }
}

Bias Mitigation Strategies

Mitigation Approaches

Stage	Technique	Description
Pre-processing	Reweighting	Adjust sample weights
	Resampling	Over/undersample groups
	Data augmentation	Add synthetic examples
	Feature transformation	Remove proxy signals
In-processing	Constrained optimization	Add fairness constraints
	Adversarial debiasing	Train discriminator
	Fair representations	Learn fair embeddings
Post-processing	Threshold adjustment	Group-specific thresholds
	Calibration	Equalize calibration
	Reject option	Abstain on uncertain cases

Implementation Example

public class BiasAwarePipeline
{
    public async Task<DebiasedModel> TrainWithFairnessConstraints(
        Dataset data,
        string protectedAttribute,
        FairnessConstraint constraint,
        CancellationToken ct)
    {
        // Pre-processing: Reweight samples
        var weights = CalculateReweightingFactors(data, protectedAttribute);

        // Split data
        var (train, validation) = data.Split(0.8);

        // Train with fairness-aware loss
        var model = new FairnessAwareClassifier(constraint);

        var options = new TrainingOptions
        {
            SampleWeights = weights,
            FairnessLambda = 0.5,  // Trade-off parameter
            EarlyStoppingMetric = "fairness_adjusted_auc"
        };

        await model.Train(train, options, ct);

        // Post-processing: Adjust thresholds
        var thresholds = OptimizeGroupThresholds(
            model,
            validation,
            protectedAttribute,
            constraint);

        return new DebiasedModel
        {
            Model = model,
            GroupThresholds = thresholds,
            FairnessMetrics = EvaluateFairness(model, validation, protectedAttribute)
        };
    }

    private Dictionary<string, double> OptimizeGroupThresholds(
        IClassifier model,
        Dataset validation,
        string protectedAttribute,
        FairnessConstraint constraint)
    {
        var groups = validation.GetUniqueValues(protectedAttribute);
        var thresholds = new Dictionary<string, double>();

        // Grid search for thresholds that satisfy constraint
        foreach (var group in groups)
        {
            var groupData = validation.Filter(protectedAttribute, group);
            var bestThreshold = 0.5;
            var bestFairness = double.MaxValue;

            for (var t = 0.1; t <= 0.9; t += 0.05)
            {
                var fairnessGap = EvaluateFairnessGap(
                    model, validation, protectedAttribute, group, t, constraint);

                if (fairnessGap < bestFairness)
                {
                    bestFairness = fairnessGap;
                    bestThreshold = t;
                }
            }

            thresholds[group] = bestThreshold;
        }

        return thresholds;
    }
}

Bias Assessment Template

# Bias Assessment: [System Name]

## 1. Scope
- **Protected Attributes**: [List attributes assessed]
- **Fairness Metrics**: [Metrics used]
- **Threshold**: [Acceptable disparity level]

## 2. Data Analysis
### Dataset Demographics
| Group | Count | Percentage | Base Rate |
|-------|-------|------------|-----------|
| [Group A] | [N] | [%] | [Rate] |
| [Group B] | [N] | [%] | [Rate] |

### Proxy Analysis
| Feature | Correlation | Risk Level | Action |
|---------|-------------|------------|--------|
| [Feature] | [Corr] | [H/M/L] | [Action] |

## 3. Fairness Metrics

### Demographic Parity
| Group | Positive Rate | Gap from Reference |
|-------|---------------|-------------------|
| [Group A] | [Rate] | - (reference) |
| [Group B] | [Rate] | [Gap] |

**Disparate Impact Ratio**: [X.XX] (Target: > 0.8)

### Equalized Odds
| Group | TPR | FPR |
|-------|-----|-----|
| [Group A] | [Rate] | [Rate] |
| [Group B] | [Rate] | [Rate] |

**TPR Disparity**: [X.XX] | **FPR Disparity**: [X.XX]

## 4. Findings

### Identified Biases
| Bias | Severity | Evidence | Affected Group |
|------|----------|----------|----------------|
| [Bias 1] | [H/M/L] | [Evidence] | [Group] |

### Root Cause Analysis
[Analysis of why bias exists]

## 5. Mitigation Plan

| Bias | Mitigation Strategy | Expected Impact | Trade-off |
|------|---------------------|-----------------|-----------|
| [Bias 1] | [Strategy] | [Impact] | [Trade-off] |

## 6. Monitoring Plan
- [ ] Automated fairness monitoring
- [ ] Periodic reassessment schedule
- [ ] Drift detection for fairness metrics

## 7. Sign-off
- [ ] Data science review
- [ ] Legal/compliance review
- [ ] Ethics board review (if applicable)

Validation Checklist

• Protected attributes identified
• Proxy variables analyzed
• Fairness metrics selected
• Baseline measurements taken
• Disparate impact calculated
• Equalized odds evaluated
• Bias root causes analyzed
• Mitigation strategies defined
• Trade-offs documented
• Monitoring plan established

Integration Points

Inputs from:

• Data sources → Training data demographics
• Legal/compliance → Protected attribute requirements

• ml-project-lifecycle

  skill → Project constraints

Outputs to:

• ai-safety-planning

  skill → Fairness requirements

• explainability-planning

  skill → Bias explanations
• Documentation → Compliance evidence