Bias Mitigation Techniques in Large Language Models: An Empirical Evaluation of Post-Training and In-Training Approaches

Abstract

The proliferation of large language models (LLMs) in critical applications has intensified concerns about embedded social biases that can perpetuate discrimination and inequality. While numerous bias mitigation techniques have been proposed, systematic comparison of intervention timing pacifically in-training versus post-training approaches remains limited. This paper presents a comprehensive empirical evaluation of bias mitigation strategies across six LLM architectures ranging from 340 million to 1.7 trillion parameters. We implement and compare four in-training methods (data preprocessing, adversarial training, fairness regularization, multi-task learning) against four post-training approaches (supervised fine-tuning, reinforcement learning from human feedback, constitutional AI, inference filtering) across nine bias categories including gender, race, religion, age, and socioeconomic status. Our evaluation employs established benchmarks (StereoSet, CrowS-Pairs) supplemented by custom synthetic datasets, with rigorous statistical analysis including bootstrap confidence intervals and effect size calculations. Results demonstrate that in-training methods achieve superior bias reduction effectiveness across all categories, with large effect sizes (Cohen's d > 1.2) and statistical significance (p < 0.001). Notably, in-training approaches maintain scale-invariant performance across model sizes while post-training methods show degradation for models exceeding 10 billion parameters. Counterintuitively, in-training methods preserve task performance better than post-training corrections (0.00-0.02 vs 0.03-0.05 GLUE score degradation). The 2.4× computational overhead of in-training methods is offset by 15-20% improvements in bias reduction effectiveness and superior robustness to intersectional biases. These findings provide definitive guidance for practitioners deploying bias mitigation in production LLM systems and establish the empirical foundation for prioritizing in-training approaches in fairness-critical applications.