UAE-based AIREV has launched Harness Arena, an open-source platform for evaluating autonomous agent harnesses, the scaffolding, tools, system prompts and execution logic built around large language models.
The platform runs identical real-world assignments across competing agent frameworks using the same model configuration and isolated workspaces. Tasks are designed to produce practical deliverables such as reports, dashboards and codebases.
Harness identities are hidden during evaluation. Human reviewers score anonymized outputs from 1 to 10, with labels revealed only after all scores are submitted. Rankings are updated using a pairwise Elo formula capped at K=32 per task.
According to the announcement, the system is intended to move beyond static language-model benchmarks by measuring how different orchestration layers handle complex, multi-step enterprise workflows. Tasks are supplied through Excel datasets containing prompts, rubrics, expected deliverables and reference material.
Sponsored by AIREV’s enterprise automation platform OnDemand, Harness Arena compares open-source and proprietary frameworks, including Claude Code, Codex, Hermes and OnDemand.
Source: Middle East AI News
