BenchMatch bench.match@1
Generated at from
openvibe-contracts v0.443.0 and
openvibe-sdk v0.177.0.
- Version
- 1.0.0
- Owner
- bench
- Visibility
- public
- Status
- active
- Compatibility
- backward
- Schema
https://openvibe.network/contracts/bench/match.v1.json
bench.match@1 (compare-arena-ecosystem.md, 2026-11-13): one task given to two or more entrants under the same budget and machine class, the receipts of each side, and every judgement that came in. Every comparison on the network is one: an Arena vote, a Chat draft picked, a Codes race whose patch was merged, a check job's tests, a rubric model, a game, a Live chat's poll. SOURCE says where it came from; SLOTS are the sides (a, b, …) with the entrant, the run that answered and its receipts; SHOWN is the order the sides were shown in (position bias is modelled, never ignored); BLIND says whether the names were hidden until the first judgement. OpenVibe.Bench keeps matches and rates entrants from them (bench.rating@1).
Fields
| Field | Type | Required | Description | Constraints |
|---|---|---|---|---|
id | string | yes |
| |
source | enum | yes |
| |
task | object | yes |
| |
task.kind | enum | yes |
| |
task.category | string | null | What the task is about, from the classifier (code.debug, writing.email, math …), so boards are per category. |
| |
task.prompt_hash | string | null | sha256 of the prompt: dedupe and the private pool, never the prompt itself. |
| |
task.prompt | string | null | Only when the asker opted in to share it (PUBLIC true). |
| |
task.public | boolean |
| ||
task.repo | string | null |
| ||
task.sha | string | null |
| ||
task.suite | string | null | A Bench suite and task id, when the task came from one. |
| |
slots | array of object | yes |
| |
slots[].slot | string | yes |
| |
slots[].entrant | string | yes | A bench.entrant@1 id. |
|
slots[].run_id | string | null | The agents.run@1 (or AI run) that answered, so its steps and receipts can be read. |
| |
slots[].receipts | object |
| ||
slots[].receipts.outcome | enum |
| ||
slots[].receipts.cost_usd | number | null |
| ||
slots[].receipts.tokens_in | integer | null |
| ||
slots[].receipts.tokens_out | integer | null |
| ||
slots[].receipts.ttft_ms | number | null |
| ||
slots[].receipts.tps | number | null |
| ||
slots[].receipts.wall_ms | number | null |
| ||
slots[].receipts.tool_calls | integer | null |
| ||
slots[].receipts.length_chars | integer | null | The answer's length, a style covariate. |
| |
slots[].receipts.markdown_items | integer | null | Headings, lists and bold spans in the answer, a style covariate. |
| |
shown | array of string | The order the sides were shown in, left to right or top to bottom. |
| |
blind | boolean |
| ||
budget | object | null | The same for every side: a match between unequal budgets is not a match. | ||
judgements | array of object | yes |
| |
judgements[].judge | enum | yes |
| |
judgements[].winner | string | null |
| ||
judgements[].scores | object | null | Per side: tests passed (0..1), a rubric score, a game's points. | ||
judgements[].signal | enum | An implicit judge's signal: what the person did with a side. |
| |
judgements[].votes | object | null | An audience poll's counts. | ||
judgements[].by | object | null | |||
judgements[].weight | number | null | The voter's quality weight when the judgement was counted (gold pairs, agreement). |
| |
judgements[].style_note | string | null |
| ||
judgements[].at | string | yes |
| |
play | object | null | The race play and its parent run, when the match was run on the agent engine. | ||
at | string | yes |
|
Examples
From the contract's own test fixtures: valid ones validate, rejected ones must fail.
Valid: a-codes-race-merged
{
"id": "mat_01KA0000000000000000000003",
"source": "codes",
"task": {
"kind": "code",
"category": "code.bugfix",
"repo": "https://github.com/OpenVibers/OpenVibe.Compare",
"sha": "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb"
},
"slots": [
{
"slot": "a",
"entrant": "agent:codex+gpt-5-mini#3f9a1c22",
"receipts": {
"outcome": "succeeded",
"cost_usd": 0.21,
"wall_ms": 312000,
"tool_calls": 41
}
},
{
"slot": "b",
"entrant": "agent:opencode+glm-5.3#9b7e2d10",
"receipts": {
"outcome": "succeeded",
"cost_usd": 0.09,
"wall_ms": 401000,
"tool_calls": 57
}
}
],
"blind": false,
"judgements": [
{
"judge": "test_suite",
"scores": {
"a": 1,
"b": 0.8
},
"by": {
"type": "suite"
},
"at": "2026-11-13T02:10:00Z"
},
{
"judge": "implicit",
"winner": "a",
"signal": "merged",
"by": {
"type": "person",
"id_hash": "fedcba9876543210"
},
"at": "2026-11-13T02:30:00Z"
}
],
"at": "2026-11-13T02:00:00Z"
}Valid: an-arena-vote
{
"id": "mat_01KA0000000000000000000000",
"source": "arena",
"task": {
"kind": "chat",
"category": "writing.email",
"prompt_hash": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
"public": false
},
"slots": [
{
"slot": "a",
"entrant": "model:deepseek-flash",
"run_id": "run_01KA0000000000000000000001",
"receipts": {
"outcome": "succeeded",
"cost_usd": 0.0004,
"tokens_in": 120,
"tokens_out": 300,
"ttft_ms": 540,
"tps": 86,
"wall_ms": 4100,
"length_chars": 1210,
"markdown_items": 3
}
},
{
"slot": "b",
"entrant": "model:gpt-5-mini",
"run_id": "run_01KA0000000000000000000002",
"receipts": {
"outcome": "succeeded",
"cost_usd": 0.0011,
"tokens_in": 120,
"tokens_out": 280,
"ttft_ms": 820,
"tps": 61,
"wall_ms": 5300,
"length_chars": 980,
"markdown_items": 0
}
}
],
"shown": [
"b",
"a"
],
"blind": true,
"budget": {
"usd": 0.01,
"ms": 60000,
"machine_class": null
},
"judgements": [
{
"judge": "human_pairwise",
"winner": "a",
"by": {
"type": "person",
"id_hash": "0123456789abcdef"
},
"weight": 0.9,
"at": "2026-11-13T02:00:00Z"
}
],
"play": {
"id": "race",
"run_id": "run_01KA0000000000000000000000"
},
"at": "2026-11-13T01:59:30Z"
}Rejected: a-judge-nobody-has
{
"id": "mat_01KA0000000000000000000000",
"source": "arena",
"task": {
"kind": "chat",
"category": "writing.email",
"prompt_hash": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
"public": false
},
"slots": [
{
"slot": "a",
"entrant": "model:deepseek-flash",
"run_id": "run_01KA0000000000000000000001",
"receipts": {
"outcome": "succeeded",
"cost_usd": 0.0004,
"tokens_in": 120,
"tokens_out": 300,
"ttft_ms": 540,
"tps": 86,
"wall_ms": 4100,
"length_chars": 1210,
"markdown_items": 3
}
},
{
"slot": "b",
"entrant": "model:gpt-5-mini",
"run_id": "run_01KA0000000000000000000002",
"receipts": {
"outcome": "succeeded",
"cost_usd": 0.0011,
"tokens_in": 120,
"tokens_out": 280,
"ttft_ms": 820,
"tps": 61,
"wall_ms": 5300,
"length_chars": 980,
"markdown_items": 0
}
}
],
"shown": [
"b",
"a"
],
"blind": true,
"budget": {
"usd": 0.01,
"ms": 60000,
"machine_class": null
},
"judgements": [
{
"judge": "vibes",
"winner": "a",
"by": {
"type": "person",
"id_hash": "0123456789abcdef"
},
"weight": 0.9,
"at": "2026-11-13T02:00:00Z"
}
],
"play": {
"id": "race",
"run_id": "run_01KA0000000000000000000000"
},
"at": "2026-11-13T01:59:30Z"
}Rejected: one-side-is-not-a-match
{
"id": "mat_01KA0000000000000000000000",
"source": "arena",
"task": {
"kind": "chat",
"category": "writing.email",
"prompt_hash": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
"public": false
},
"slots": [
{
"slot": "a",
"entrant": "model:deepseek-flash",
"run_id": "run_01KA0000000000000000000001",
"receipts": {
"outcome": "succeeded",
"cost_usd": 0.0004,
"tokens_in": 120,
"tokens_out": 300,
"ttft_ms": 540,
"tps": 86,
"wall_ms": 4100,
"length_chars": 1210,
"markdown_items": 3
}
}
],
"shown": [
"b",
"a"
],
"blind": true,
"budget": {
"usd": 0.01,
"ms": 60000,
"machine_class": null
},
"judgements": [
{
"judge": "human_pairwise",
"winner": "a",
"by": {
"type": "person",
"id_hash": "0123456789abcdef"
},
"weight": 0.9,
"at": "2026-11-13T02:00:00Z"
}
],
"play": {
"id": "race",
"run_id": "run_01KA0000000000000000000000"
},
"at": "2026-11-13T01:59:30Z"
}Validate
const contracts = require('openvibe-contracts');
contracts.validate('bench.match@1', value); // { valid, errors: [{ path, message }] }