Two Stanford-led studies for COLM 2026 test AI benchmarks the way psychologists test exams. Across 56 benchmarks and 53 models, safety tests with the same label often disagreed, capability tests blurred together, and the scoring format mattered more than the topic. We explain the findings and what they mean for choosing a model.
Johns Hopkins researchers took 427 real workplace writing prompts, rewrote each into a women-associated and a men-associated version, and ran both through GPT-4, Llama, Mistral and Gemma. Every model returned shorter, plainer, less formal documents for the women-coded phrasing. Adding a male or female sign-off name changed nothing. Linear probes decode the linguistic register at 0.988 accuracy by layer 5 while name gender reaches only 0.717, and activation patching puts the causal weight in layers 0 to 7. This article covers the method, the per-model results, the two controls that rule out mirroring, and why the authors say users cannot fix it themselves.
Progressive Robot: Your Gateway to Comprehensive IT Solutions — Specializing in Web Development, Mobile App Development, and Expert IT Services.
© All Copyright by Progressiverobot.com
VAT Number ( 506152326 )
