llms.blog
Benchmarks7 articles

Benchmarks

Articles

  • Stealth Model 'Ox Alpha' Surfaces with 1M Multimodal Context and 100 Trillion Daily Token Allocation

    An anonymous frontier model named "Ox Alpha" has launched across OpenRouter and the OpenCode developer environment, offering free access backed by a reported 100 trillion tokens per day in serving capacity. The model features a 1,048,576-token context window with native multimodal support spanning text, image, and video inputs. The unannounced release follows a pattern of stealth evaluations across AI model aggregators, where frontier labs deploy pre-release checkpoints to collect blind compara

    1 min

  • Study Exposes Benchmark Overfitting and Acoustic Leakage in Speech Recognition Models

    A joint empirical study by researchers from Hugging Face and Hume AI has uncovered evidence that top-ranking automatic speech recognition (ASR) models frequently exhibit benchmark optimization, commonly referred to as "benchmaxxing." The findings demonstrate that models often achieve low word error rates on public leaderboards by memorizing test-set reference anomalies and responding to subtle acoustic channel signatures rather than generalizing audio transcription. Evaluating 11 open-source sp

    1 min

  • Vals AI Raises $40M Series A at $400M Valuation Led by a16z to Build Real-World AI Benchmarks

    San Francisco evaluation startup Vals AI announced a $40 million Series A funding round at a $400 million post-money valuation, led by Andreessen Horowitz. The round included participation from existing seed backers 8VC, Pear VC, and Bloomberg Beta, alongside new institutional investors HRT Ventures and Next Ladder Ventures. The financing brings total capital raised by the company to $45 million, following a $5 million seed round. Founded by Stanford computer science graduates Rayan Krishnan a

    1 min

  • GLM-5.3 Scores 60 on Artificial Analysis Intelligence Index, Matching Kimi K3

    Independent AI evaluation platform Artificial Analysis has published its benchmark results for Z.ai's GLM-5.3, awarding the reasoning model a score of 60 on its Intelligence Index v4.1.1. The result places GLM-5.3 level with Moonshot AI's Kimi K3 and three points behind frontier leader Claude Opus 5 (63). The evaluation tested GLM-5.3 at its maximum reasoning effort configuration across a nine-part battery that measures agentic tool execution, terminal coding, graduate-level scientific problem-

    1 min

  • Artificial Analysis Launches Search Index Benchmark for AI Agent Search APIs

    Artificial Analysis has released the Search Index, a benchmark suite designed to evaluate web search APIs for autonomous AI agents across retrieval quality, query latency, and end-to-end task economics. The initial evaluation tests seven dedicated search providers: Parallel, Exa, Firecrawl, You.com, Tavily, Keenable, and Brave. Benchmark Setup and Evaluation Methodology To isolate search API performance from model variance, the evaluation executes all tests with GPT-5.6 Luna inside Stirrup,

    1 min

  • Zhipu's GLM-5.2 Narrows the Gap With Anthropic's Fable 5 on Coding Benchmarks

    Zhipu AI's open-weight GLM-5.2 model has placed second globally on the Code Arena coding benchmark, trailing only Anthropic's Claude Fable 5 and sitting within one point of Claude Opus 4.8 on the hardest agentic tasks. The result, recorded after the model's June 13 release under an MIT license, signals a fast-closing capability gap between Chinese open-weight systems and the leading U.S. frontier models. Benchmark Performance On Code Arena's front-end coding leaderboard, GLM-5.2 places second

    1 min