TruthfulQA capability update — measuring models' propensity to mimic human falsehoods
AI Impact Summary
This change updates the evaluation framework used to measure models' truthfulness by simulating human falsehoods. Teams relying on TruthfulQA for model selection or safety tuning should expect shifts in truthfulness scores and relative rankings across model versions. Prepare to re-run benchmark assessments, adjust acceptance criteria, and align product release gates with the revised metrics.
Affected Systems
Business Impact
Teams using TruthfulQA scores for model selection or safety tuning should expect changed rankings and updated acceptance criteria, affecting release readiness.
- Date
- Date not specified
- Change type
- capability
- Severity
- medium