llms.blog
AI

NVIDIA AVO Achieves 100% on ARC-AGI-3 with Claude Opus 5

NVIDIA Research has published details on Agentic Variation Operators (AVO), a general-purpose autonomous agent architecture designed to sustain long-horizon engineering and reasoning workflows. In benchmark evaluations on ARC-AGI-3, an interactive reasoning suite that tests fluid intelligence without instructions or stated goals, AVO scored 100.00 Relative Human Action Efficiency (RHAE) across all 25 public-set environments and 183 levels using Anthropic's Claude Opus 5. The result highlights a

axc
axc
3 min
NVIDIA AVO Achieves 100% on ARC-AGI-3 with Claude Opus 5

NVIDIA Research has published details on Agentic Variation Operators (AVO), a general-purpose autonomous agent architecture designed to sustain long-horizon engineering and reasoning workflows. In benchmark evaluations on ARC-AGI-3, an interactive reasoning suite that tests fluid intelligence without instructions or stated goals, AVO scored 100.00 Relative Human Action Efficiency (RHAE) across all 25 public-set environments and 183 levels using Anthropic's Claude Opus 5.

The result highlights a growing trend in AI systems research: benchmark performance over extended tasks is determined as much by the surrounding agent harness as by the foundational language model itself.

Architecture: Persistent Memory and Trajectory Supervision

AVO structures agent execution around an iterative inspect-plan-implement-evaluate loop backed by two core mechanisms: persistent memory and hierarchical supervision.

Unlike standard conversational loops that discard context or compress histories destructively across turns, AVO maintains structured state across invocations. In engineering settings, this includes prior implementation attempts, compiler diagnostics, profiling metrics, and hardware traces. In interactive reasoning settings, memory tracks inferred environmental dynamics, previous action consequences, and spatial configurations.

AVO Architecture Diagram

To prevent agents from getting trapped in local minima or repeating unproductive exploratory actions, AVO introduces an external supervisor process. The supervisor monitors the overall search trajectory. When rate-of-progress metrics indicate stagnation or cyclical failures, the supervisor intervenes to redirect the main agent toward alternative search branches or hypothesis spaces.

For ARC-AGI-3, AVO interacted through a text-only representation, parsing 64x64 character grids rather than processing raw vision tokens or multi-modal embeddings.

Benchmarking: ARC-AGI-3 and Kernel Optimization

ARC-AGI-3 measures an agent's ability to discover rules and reach goals through direct interaction in unfamiliar game-like grid worlds. Performance is scored via Relative Human Action Efficiency (RHAE), which balances level completion against action counts relative to human baselines.

| Benchmark / Workload | Baseline / Prior Art | AVO Result | Key Metric / Delta | | :--- | :--- | :--- | :--- | | ARC-AGI-3 Public Set (Opus 5) | VISTA Harness (7,542 actions) | AVO (6,624 actions) | 100.00 RHAE (12% fewer actions) | | ARC-AGI-3 Standalone Model | Claude Opus 5 (High effort) | AVO + Opus 5 | 30% baseline to 100% completion | | B200 Attention Kernel Optimization | FlashAttention-4 / cuDNN | AVO Autonomous Search | +10.5% vs FlashAttention-4, +3.5% vs cuDNN | | Kernel Exploration Horizon | Manual Heuristic Tuning | 7 Days Continuous Run | 500+ paths explored, 40 versions committed |

While standalone Claude Opus 5 scores approximately 30% on ARC-AGI-3 under standard single-context evaluation, pairing the model with AVO's state management and supervisor loop enabled 100% task completion across all 183 public levels. In cross-harness comparisons, AVO completed the suite in 6,624 actions compared to 7,542 actions for the VISTA visual harness.

NVIDIA also evaluated AVO with OpenAI's GPT-5.6 Sol on a subset of levels, noting that Sol demonstrated faster wall-clock execution while Opus 5 required fewer total environment actions.

Prior to the ARC-AGI-3 evaluation, NVIDIA tested AVO on low-level GPU kernel development. Over a seven-day autonomous run on NVIDIA DGX B200 systems, AVO explored more than 500 optimization strategies, generating 40 verified kernel commits. The resulting attention kernels improved throughput by up to 10.5% over FlashAttention-4 and up to 3.5% over cuDNN on evaluated Blackwell configurations. The agent subsequently adapted the kernel for Grouped-Query Attention (GQA) in roughly 30 minutes of additional compute.

System Design Over Model-Only Scaling

The cross-domain transfer of AVO—from hardware-bound CUDA optimization to abstract grid-world reasoning—demonstrates that long-horizon autonomous task execution relies on generalizable system scaffolding. By isolating state tracking, feedback parsing, and search recovery into dedicated architectural layers, agent frameworks can extract substantially higher practical capability from existing frontier models.

Sources

AILLMAgentsBenchmarkingNvidia
ShareX (Twitter)LinkedIn

Written by

axc

More to read

  • Listwise Reranking in Production: Comparing Pointwise, Pairwise, and Listwise LLM Architectures, Sliding-Window Permutations, and Serving Economics

    Information retrieval systems in production Retrieval-Augmented Generation (RAG) and enterprise search have transitioned through multiple reranking paradigms. While first-stage retrieval (dense vector embeddings and sparse lexical BM25/SPLADE) retrieves candidate sets of 50 to 200 documents in under 20 milliseconds, the precision of downstream generation depends heavily on the reranking stage. Traditional neural rerankers evaluate candidates through pointwise scoring or pairwise classification.

    1 min

  • The Lottery Ticket Hypothesis in Large Language Models: How Sparse Subnetworks and Iterative Magnitude Pruning Retain Transformer Capacity

    The Lottery Ticket Hypothesis in Large Language Models: How Sparse Subnetworks and Iterative Magnitude Pruning Retain Transformer Capacity Modern large language models operate under extreme overparameterization. Frontier architectures allocate tens or hundreds of billions of parameters to achieve low perplexity and robust generalization across reasoning, code generation, and factual retrieval. Yet empirical pruning consistently demonstrates that post-training models can lose 30% to 50% of their

    1 min

  • Enterprises Curb AI Agent Autonomy Amid 40% Project Cancellation Projections

    Enterprises deploying agentic artificial intelligence are shifting architectures away from open-ended autonomy toward bounded, verifiable execution as projects encounter governance, security, and financial bottlenecks in production environments. Data from industry research firms highlights a widening divergence between model capability and operational control. According to projections from Gartner, more than 40% of current agentic AI initiatives are projected to be canceled by the end of 2027.

    1 min