OpenVibe.Bench API
Generated at from
openvibe-contracts v0.419.0 and
openvibe-sdk v0.161.0.
Server https://bench.openvibe.services. 27 routes performing 13 capabilities. OpenAPI 3.1 document.
POST /api/v1/benchmarks
Create, edit, freeze and fork a benchmark on OpenVibe.Bench: POST /api/v1/benchmarks, PUT /api/v1/benchmarks/:id, POST /api/v1/benchmarks/:id/versions (freeze) and POST /api/v1/benchmarks/:id/fork.
- Capabilities
- bench.benchmark.write
- Visibility
- first-party
- Request body
application/jsonbench.benchmark@1- Response
application/jsonbench.benchmark@1- Errors
- errors.problem@1 (application/problem+json)
Description
<p><strong>bench.benchmark.write</strong> (first-party): Create, edit, freeze and fork a benchmark on OpenVibe.Bench: POST /api/v1/benchmarks, PUT /api/v1/benchmarks/:id, POST /api/v1/benchmarks/:id/versions (freeze) and POST /api/v1/benchmarks/:id/fork. A version is immutable once a run names it; a fork records its parent.</p>PUT /api/v1/benchmarks/{id}
Create, edit, freeze and fork a benchmark on OpenVibe.Bench: POST /api/v1/benchmarks, PUT /api/v1/benchmarks/:id, POST /api/v1/benchmarks/:id/versions (freeze) and POST /api/v1/benchmarks/:id/fork.
- Capabilities
- bench.benchmark.write
- Visibility
- first-party
- Parameters
id(path, required)
- Request body
application/jsonbench.benchmark@1- Response
application/jsonbench.benchmark@1- Errors
- errors.problem@1 (application/problem+json)
Description
<p><strong>bench.benchmark.write</strong> (first-party): Create, edit, freeze and fork a benchmark on OpenVibe.Bench: POST /api/v1/benchmarks, PUT /api/v1/benchmarks/:id, POST /api/v1/benchmarks/:id/versions (freeze) and POST /api/v1/benchmarks/:id/fork. A version is immutable once a run names it; a fork records its parent.</p>POST /api/v1/benchmarks/{id}/fork
Create, edit, freeze and fork a benchmark on OpenVibe.Bench: POST /api/v1/benchmarks, PUT /api/v1/benchmarks/:id, POST /api/v1/benchmarks/:id/versions (freeze) and POST /api/v1/benchmarks/:id/fork.
- Capabilities
- bench.benchmark.write
- Visibility
- first-party
- Parameters
id(path, required)
- Request body
application/jsonbench.benchmark@1- Response
application/jsonbench.benchmark@1- Errors
- errors.problem@1 (application/problem+json)
Description
<p><strong>bench.benchmark.write</strong> (first-party): Create, edit, freeze and fork a benchmark on OpenVibe.Bench: POST /api/v1/benchmarks, PUT /api/v1/benchmarks/:id, POST /api/v1/benchmarks/:id/versions (freeze) and POST /api/v1/benchmarks/:id/fork. A version is immutable once a run names it; a fork records its parent.</p>POST /api/v1/benchmarks/{id}/versions
Create, edit, freeze and fork a benchmark on OpenVibe.Bench: POST /api/v1/benchmarks, PUT /api/v1/benchmarks/:id, POST /api/v1/benchmarks/:id/versions (freeze) and POST /api/v1/benchmarks/:id/fork.
- Capabilities
- bench.benchmark.write
- Visibility
- first-party
- Parameters
id(path, required)
- Request body
application/jsonbench.benchmark@1- Response
application/jsonbench.benchmark@1- Errors
- errors.problem@1 (application/problem+json)
Description
<p><strong>bench.benchmark.write</strong> (first-party): Create, edit, freeze and fork a benchmark on OpenVibe.Bench: POST /api/v1/benchmarks, PUT /api/v1/benchmarks/:id, POST /api/v1/benchmarks/:id/versions (freeze) and POST /api/v1/benchmarks/:id/fork. A version is immutable once a run names it; a fork records its parent.</p>GET /api/v1/boards/{kindId}
Read a benchmark board on OpenVibe.Bench: GET /api/v1/boards/:kindId?type=elo|pareto|cost returns the rendered board for a kind.
- Capabilities
- bench.board.read
- Visibility
- first-party
- Parameters
kindId(path, required)
- Response
application/jsonbench.rating@1- Errors
- errors.problem@1 (application/problem+json)
Description
<p><strong>bench.board.read</strong> (first-party): Read a benchmark board on OpenVibe.Bench: GET /api/v1/boards/:kindId?type=elo|pareto|cost returns the rendered board for a kind.</p>GET /api/v1/compare
Read the runs of a benchmark on OpenVibe.Bench: GET /api/v1/runs/:id returns the result, GET /api/v1/compare?runs=a,b,c puts runs side by side, GET /api/v1/rankings gives the per-benchmark rankings an
- Capabilities
- bench.run.read
- Visibility
- first-party
- Response
application/jsonbench.run@1- Errors
- errors.problem@1 (application/problem+json)
Description
<p><strong>bench.run.read</strong> (first-party): Read the runs of a benchmark on OpenVibe.Bench: GET /api/v1/runs/:id returns the result, GET /api/v1/compare?runs=a,b,c puts runs side by side, GET /api/v1/rankings gives the per-benchmark rankings and GET /api/v1/percentile says where a score sits among a model's runs.</p>GET /api/v1/datasets/{kindId}.jsonl
Read the open dataset for a benchmark kind on OpenVibe.Bench: GET /api/v1/datasets/:kindId.jsonl returns prompts, entries, judgements without voter ids and ratings as JSON lines.
- Capabilities
- bench.dataset.read
- Visibility
- first-party
- Parameters
kindId(path, required)
- Response
application/octet-streambytes- Errors
- errors.problem@1 (application/problem+json)
Description
<p><strong>bench.dataset.read</strong> (first-party): Read the open dataset for a benchmark kind on OpenVibe.Bench: GET /api/v1/datasets/:kindId.jsonl returns prompts, entries, judgements without voter ids and ratings as JSON lines.</p>POST /api/v1/evals
Manage private evals on OpenVibe.Bench: POST /api/v1/evals creates a private task suite, POST /api/v1/evals/:id/run runs it against chosen offers, and GET /api/v1/evals/:id/results reads a private eva
- Capabilities
- bench.eval.manage
- Visibility
- first-party
- Response
application/jsonbench.task@1- Errors
- errors.problem@1 (application/problem+json)
Description
<p><strong>bench.eval.manage</strong> (first-party): Manage private evals on OpenVibe.Bench: POST /api/v1/evals creates a private task suite, POST /api/v1/evals/:id/run runs it against chosen offers, and GET /api/v1/evals/:id/results reads a private eval and its runs. Never in the public dataset.</p>GET /api/v1/evals/{id}/results
Manage private evals on OpenVibe.Bench: POST /api/v1/evals creates a private task suite, POST /api/v1/evals/:id/run runs it against chosen offers, and GET /api/v1/evals/:id/results reads a private eva
- Capabilities
- bench.eval.manage
- Visibility
- first-party
- Parameters
id(path, required)
- Response
application/jsonbench.task@1- Errors
- errors.problem@1 (application/problem+json)
Description
<p><strong>bench.eval.manage</strong> (first-party): Manage private evals on OpenVibe.Bench: POST /api/v1/evals creates a private task suite, POST /api/v1/evals/:id/run runs it against chosen offers, and GET /api/v1/evals/:id/results reads a private eval and its runs. Never in the public dataset.</p>POST /api/v1/evals/{id}/run
Manage private evals on OpenVibe.Bench: POST /api/v1/evals creates a private task suite, POST /api/v1/evals/:id/run runs it against chosen offers, and GET /api/v1/evals/:id/results reads a private eva
- Capabilities
- bench.eval.manage
- Visibility
- first-party
- Parameters
id(path, required)
- Response
application/jsonbench.task@1- Errors
- errors.problem@1 (application/problem+json)
Description
<p><strong>bench.eval.manage</strong> (first-party): Manage private evals on OpenVibe.Bench: POST /api/v1/evals creates a private task suite, POST /api/v1/evals/:id/run runs it against chosen offers, and GET /api/v1/evals/:id/results reads a private eval and its runs. Never in the public dataset.</p>GET /api/v1/kinds
Read a benchmark kind on OpenVibe.Bench: GET /api/v1/kinds lists the kinds, GET /api/v1/kinds/:kindId reads one kind and its entry schema and metrics.
- Capabilities
- bench.kind.read
- Visibility
- first-party
- Response
application/jsonbench.kind@1- Errors
- errors.problem@1 (application/problem+json)
Description
<p><strong>bench.kind.read</strong> (first-party): Read a benchmark kind on OpenVibe.Bench: GET /api/v1/kinds lists the kinds, GET /api/v1/kinds/:kindId reads one kind and its entry schema and metrics. Granted to rooms and developer services that render a kind or its board.</p>GET /api/v1/kinds/{kindId}
Read a benchmark kind on OpenVibe.Bench: GET /api/v1/kinds lists the kinds, GET /api/v1/kinds/:kindId reads one kind and its entry schema and metrics.
- Capabilities
- bench.kind.read
- Visibility
- first-party
- Parameters
kindId(path, required)
- Response
application/jsonbench.kind@1- Errors
- errors.problem@1 (application/problem+json)
Description
<p><strong>bench.kind.read</strong> (first-party): Read a benchmark kind on OpenVibe.Bench: GET /api/v1/kinds lists the kinds, GET /api/v1/kinds/:kindId reads one kind and its entry schema and metrics. Granted to rooms and developer services that render a kind or its board.</p>POST /api/v1/kinds/{kindId}/tasks
Create a benchmark task in a kind on OpenVibe.Bench: POST /api/v1/kinds/:kindId/tasks with { prompt, task_class?, public?
- Capabilities
- bench.task.create
- Visibility
- first-party
- Parameters
kindId(path, required)
- Response
application/jsonbench.task@1- Errors
- errors.problem@1 (application/problem+json)
Description
<p><strong>bench.task.create</strong> (first-party): Create a benchmark task in a kind on OpenVibe.Bench: POST /api/v1/kinds/:kindId/tasks with { prompt, task_class?, public? }. A room or a person's own token may create a task.</p>GET /api/v1/percentile
Read the runs of a benchmark on OpenVibe.Bench: GET /api/v1/runs/:id returns the result, GET /api/v1/compare?runs=a,b,c puts runs side by side, GET /api/v1/rankings gives the per-benchmark rankings an
- Capabilities
- bench.run.read
- Visibility
- first-party
- Response
application/jsonbench.run@1- Errors
- errors.problem@1 (application/problem+json)
Description
<p><strong>bench.run.read</strong> (first-party): Read the runs of a benchmark on OpenVibe.Bench: GET /api/v1/runs/:id returns the result, GET /api/v1/compare?runs=a,b,c puts runs side by side, GET /api/v1/rankings gives the per-benchmark rankings and GET /api/v1/percentile says where a score sits among a model's runs.</p>GET /api/v1/rankings
Read the runs of a benchmark on OpenVibe.Bench: GET /api/v1/runs/:id returns the result, GET /api/v1/compare?runs=a,b,c puts runs side by side, GET /api/v1/rankings gives the per-benchmark rankings an
- Capabilities
- bench.run.read
- Visibility
- first-party
- Response
application/jsonbench.run@1- Errors
- errors.problem@1 (application/problem+json)
Description
<p><strong>bench.run.read</strong> (first-party): Read the runs of a benchmark on OpenVibe.Bench: GET /api/v1/runs/:id returns the result, GET /api/v1/compare?runs=a,b,c puts runs side by side, GET /api/v1/rankings gives the per-benchmark rankings and GET /api/v1/percentile says where a score sits among a model's runs.</p>GET /api/v1/rounds/{id}
Read a benchmark round on OpenVibe.Bench: GET /api/v1/rounds/:id returns the round with entries; entries stay blind until the caller has judged or started the round.
- Capabilities
- bench.round.read
- Visibility
- first-party
- Parameters
id(path, required)
- Response
application/jsonbench.round@1- Errors
- errors.problem@1 (application/problem+json)
Description
<p><strong>bench.round.read</strong> (first-party): Read a benchmark round on OpenVibe.Bench: GET /api/v1/rounds/:id returns the round with entries; entries stay blind until the caller has judged or started the round.</p>POST /api/v1/rounds/{id}/entries
Submit an outside entry to a benchmark round on OpenVibe.Bench: POST /api/v1/rounds/:id/entries with a service token.
- Capabilities
- bench.entry.create
- Visibility
- first-party
- Parameters
id(path, required)
- Response
application/jsonbench.entry@1- Errors
- errors.problem@1 (application/problem+json)
Description
<p><strong>bench.entry.create</strong> (first-party): Submit an outside entry to a benchmark round on OpenVibe.Bench: POST /api/v1/rounds/:id/entries with a service token. A room submits a game match, a test-suite run or another room's result.</p>POST /api/v1/rounds/{id}/judgements
Cast a blind judgement on a benchmark round on OpenVibe.Bench: POST /api/v1/rounds/:id/judgements with { judge?, pair?, outcome?, score?
- Capabilities
- bench.judgement.cast
- Visibility
- first-party
- Parameters
id(path, required)
- Response
application/jsonbench.judgement@1- Errors
- errors.problem@1 (application/problem+json)
Description
<p><strong>bench.judgement.cast</strong> (first-party): Cast a blind judgement on a benchmark round on OpenVibe.Bench: POST /api/v1/rounds/:id/judgements with { judge?, pair?, outcome?, score? }. Human judges are people; automatic, test-suite and game-outcome judges are services.</p>POST /api/v1/runs
Publish a run of a benchmark on OpenVibe.Bench: POST /api/v1/runs with the per-task scores, cost, time and tokens.
- Capabilities
- bench.run.publish
- Visibility
- first-party
- Request body
application/jsonbench.run@1- Response
application/jsonbench.run@1- Errors
- errors.problem@1 (application/problem+json)
Description
<p><strong>bench.run.publish</strong> (first-party): Publish a run of a benchmark on OpenVibe.Bench: POST /api/v1/runs with the per-task scores, cost, time and tokens. A service token (openvibe.codes, Run) makes a verified run; a person's token makes a submitted run, which is badged and kept out of the default rankings. A caller with this capability sees a benchmark's holdout tasks.</p>GET /api/v1/runs/{id}
Read the runs of a benchmark on OpenVibe.Bench: GET /api/v1/runs/:id returns the result, GET /api/v1/compare?runs=a,b,c puts runs side by side, GET /api/v1/rankings gives the per-benchmark rankings an
- Capabilities
- bench.run.read
- Visibility
- first-party
- Parameters
id(path, required)
- Response
application/jsonbench.run@1- Errors
- errors.problem@1 (application/problem+json)
Description
<p><strong>bench.run.read</strong> (first-party): Read the runs of a benchmark on OpenVibe.Bench: GET /api/v1/runs/:id returns the result, GET /api/v1/compare?runs=a,b,c puts runs side by side, GET /api/v1/rankings gives the per-benchmark rankings and GET /api/v1/percentile says where a score sits among a model's runs.</p>POST /api/v1/tasks/{id}/rounds
Pin a reproducible benchmark round on OpenVibe.Bench: POST /api/v1/tasks/:id/rounds with { offers?, seed?
- Capabilities
- bench.round.create
- Visibility
- first-party
- Parameters
id(path, required)
- Response
application/jsonbench.round@1- Errors
- errors.problem@1 (application/problem+json)
Description
<p><strong>bench.round.create</strong> (first-party): Pin a reproducible benchmark round on OpenVibe.Bench: POST /api/v1/tasks/:id/rounds with { offers?, seed? }. The round's job generates entries from the exact offers and seed.</p>GET /internal/config
Manage the configuration namespaces and secrets of OpenVibe.Bench through openvibe-shared/config (D11): GET /internal/config lists the namespace summaries, GET /internal/config/:namespace returns one
- Capabilities
- bench.config.manage
- Visibility
- internal
- Parameters
values(query, required) — The new values, one member per key. The store validates them against the namespace schema; a secret-class value here is refused.reason(query, required) — Why the change is made; it is the revision's reason in the history.merge(query) — Merge into the active values instead of replacing them.unset(query) — Keys to remove from the new revision.
- Input
- common.config-change@1
- Response
application/jsoncommon.config-namespace@1- Errors
- errors.problem@1 (application/problem+json)
Description
<p><strong>bench.config.manage</strong> (internal): Manage the configuration namespaces and secrets of OpenVibe.Bench through openvibe-shared/config (D11): GET /internal/config lists the namespace summaries, GET /internal/config/:namespace returns one as common.config-namespace@1, POST applies common.config-change@1, and PUT /internal/config/:namespace/secrets/:key sets a secret with common.config-secret-put@1 (write-only; the answer is the marker) while DELETE removes it. POST /internal/config/:namespace/test/:key asks the service's tester to try a stored value with the provider on the server, answering { ok, message } in one line that never carries a secret (the owner's Setup tab). Only Network's admin proxy, acting for the owner, holds this capability; it is never grantable to apps. Each request carries the owner as actor, and the service audits every write with that actor and key, never a secret value.</p>GET /internal/config/{namespace}
Manage the configuration namespaces and secrets of OpenVibe.Bench through openvibe-shared/config (D11): GET /internal/config lists the namespace summaries, GET /internal/config/:namespace returns one
- Capabilities
- bench.config.manage
- Visibility
- internal
- Parameters
namespace(path, required)values(query, required) — The new values, one member per key. The store validates them against the namespace schema; a secret-class value here is refused.reason(query, required) — Why the change is made; it is the revision's reason in the history.merge(query) — Merge into the active values instead of replacing them.unset(query) — Keys to remove from the new revision.
- Input
- common.config-change@1
- Response
application/jsoncommon.config-namespace@1- Errors
- errors.problem@1 (application/problem+json)
Description
<p><strong>bench.config.manage</strong> (internal): Manage the configuration namespaces and secrets of OpenVibe.Bench through openvibe-shared/config (D11): GET /internal/config lists the namespace summaries, GET /internal/config/:namespace returns one as common.config-namespace@1, POST applies common.config-change@1, and PUT /internal/config/:namespace/secrets/:key sets a secret with common.config-secret-put@1 (write-only; the answer is the marker) while DELETE removes it. POST /internal/config/:namespace/test/:key asks the service's tester to try a stored value with the provider on the server, answering { ok, message } in one line that never carries a secret (the owner's Setup tab). Only Network's admin proxy, acting for the owner, holds this capability; it is never grantable to apps. Each request carries the owner as actor, and the service audits every write with that actor and key, never a secret value.</p>POST /internal/config/{namespace}
Manage the configuration namespaces and secrets of OpenVibe.Bench through openvibe-shared/config (D11): GET /internal/config lists the namespace summaries, GET /internal/config/:namespace returns one
- Capabilities
- bench.config.manage
- Visibility
- internal
- Parameters
namespace(path, required)
- Request body
application/jsoncommon.config-change@1- Response
application/jsoncommon.config-namespace@1- Errors
- errors.problem@1 (application/problem+json)
Description
<p><strong>bench.config.manage</strong> (internal): Manage the configuration namespaces and secrets of OpenVibe.Bench through openvibe-shared/config (D11): GET /internal/config lists the namespace summaries, GET /internal/config/:namespace returns one as common.config-namespace@1, POST applies common.config-change@1, and PUT /internal/config/:namespace/secrets/:key sets a secret with common.config-secret-put@1 (write-only; the answer is the marker) while DELETE removes it. POST /internal/config/:namespace/test/:key asks the service's tester to try a stored value with the provider on the server, answering { ok, message } in one line that never carries a secret (the owner's Setup tab). Only Network's admin proxy, acting for the owner, holds this capability; it is never grantable to apps. Each request carries the owner as actor, and the service audits every write with that actor and key, never a secret value.</p>PUT /internal/config/{namespace}/secrets/{key}
Manage the configuration namespaces and secrets of OpenVibe.Bench through openvibe-shared/config (D11): GET /internal/config lists the namespace summaries, GET /internal/config/:namespace returns one
- Capabilities
- bench.config.manage
- Visibility
- internal
- Parameters
namespace(path, required)key(path, required)
- Request body
application/jsoncommon.config-change@1- Response
application/jsoncommon.config-namespace@1- Errors
- errors.problem@1 (application/problem+json)
Description
<p><strong>bench.config.manage</strong> (internal): Manage the configuration namespaces and secrets of OpenVibe.Bench through openvibe-shared/config (D11): GET /internal/config lists the namespace summaries, GET /internal/config/:namespace returns one as common.config-namespace@1, POST applies common.config-change@1, and PUT /internal/config/:namespace/secrets/:key sets a secret with common.config-secret-put@1 (write-only; the answer is the marker) while DELETE removes it. POST /internal/config/:namespace/test/:key asks the service's tester to try a stored value with the provider on the server, answering { ok, message } in one line that never carries a secret (the owner's Setup tab). Only Network's admin proxy, acting for the owner, holds this capability; it is never grantable to apps. Each request carries the owner as actor, and the service audits every write with that actor and key, never a secret value.</p>DELETE /internal/config/{namespace}/secrets/{key}
Manage the configuration namespaces and secrets of OpenVibe.Bench through openvibe-shared/config (D11): GET /internal/config lists the namespace summaries, GET /internal/config/:namespace returns one
- Capabilities
- bench.config.manage
- Visibility
- internal
- Parameters
namespace(path, required)key(path, required)values(query, required) — The new values, one member per key. The store validates them against the namespace schema; a secret-class value here is refused.reason(query, required) — Why the change is made; it is the revision's reason in the history.merge(query) — Merge into the active values instead of replacing them.unset(query) — Keys to remove from the new revision.
- Input
- common.config-change@1
- Response
application/jsoncommon.config-namespace@1- Errors
- errors.problem@1 (application/problem+json)
Description
<p><strong>bench.config.manage</strong> (internal): Manage the configuration namespaces and secrets of OpenVibe.Bench through openvibe-shared/config (D11): GET /internal/config lists the namespace summaries, GET /internal/config/:namespace returns one as common.config-namespace@1, POST applies common.config-change@1, and PUT /internal/config/:namespace/secrets/:key sets a secret with common.config-secret-put@1 (write-only; the answer is the marker) while DELETE removes it. POST /internal/config/:namespace/test/:key asks the service's tester to try a stored value with the provider on the server, answering { ok, message } in one line that never carries a secret (the owner's Setup tab). Only Network's admin proxy, acting for the owner, holds this capability; it is never grantable to apps. Each request carries the owner as actor, and the service audits every write with that actor and key, never a secret value.</p>POST /internal/config/{namespace}/test/{key}
Manage the configuration namespaces and secrets of OpenVibe.Bench through openvibe-shared/config (D11): GET /internal/config lists the namespace summaries, GET /internal/config/:namespace returns one
- Capabilities
- bench.config.manage
- Visibility
- internal
- Parameters
namespace(path, required)key(path, required)
- Request body
application/jsoncommon.config-change@1- Response
application/jsoncommon.config-namespace@1- Errors
- errors.problem@1 (application/problem+json)