Separating signal from noise in coding evaluations Separating signal from noise in coding evaluations openai.com/ai 2026-07-08 1 min read original #research A new analysis from OpenAI reveals issues in SWE-Bench Pro, a popular coding benchmark, raising concerns about reliability and accuracy in evaluating AI models.