VisionBenchmarks vision.benchmarks@1

Generated at from openvibe-contracts v0.419.0 and openvibe-sdk v0.161.0.

Version
1.0.0
Owner
vision
Visibility
public
Status
active
Compatibility
backward
Schema
https://openvibe.network/contracts/vision/benchmarks.v1.json

vision.benchmarks@1: GET /api/v1/benchmarks aggregates compare samples per task, engine, provider and model with sample counts and mean latency and cost. No subject, image or answer text is included.

Fields

FieldTypeRequiredDescriptionConstraints
countintegeryes
  • minimum 0
benchmarksarray of objectyes
  • items: no other fields
benchmarks[].taskstringyes
benchmarks[].engineenumyes
  • one of "local", "ai"
benchmarks[].providerstring | nullyes
benchmarks[].modelstring | nullyes
benchmarks[].samplesintegeryes
  • minimum 0
benchmarks[].avg_latency_msintegeryes
  • minimum 0
benchmarks[].avg_cost_usdnumberyes
  • minimum 0
benchmarks[].min_cost_usdnumberyes
  • minimum 0
benchmarks[].max_latency_msintegeryes
  • minimum 0

Examples

From the contract's own test fixtures: valid ones validate, rejected ones must fail.

Valid: one
{
  "count": 1,
  "benchmarks": [
    {
      "task": "ask",
      "engine": "ai",
      "provider": "openai",
      "model": "gpt-4o-mini",
      "samples": 3,
      "avg_latency_ms": 900,
      "avg_cost_usd": 0.0002,
      "min_cost_usd": 0.0001,
      "max_latency_ms": 1200
    }
  ]
}
Rejected: one
{
  "count": 1,
  "benchmarks": [
    {
      "task": "ask",
      "engine": "cloud",
      "provider": null,
      "model": null,
      "samples": 3,
      "avg_latency_ms": 900,
      "avg_cost_usd": 0.0002,
      "min_cost_usd": 0.0001,
      "max_latency_ms": 1200
    }
  ]
}

Validate

const contracts = require('openvibe-contracts');
contracts.validate('vision.benchmarks@1', value);   // { valid, errors: [{ path, message }] }