Send traces with OpenTelemetry (OTLP)

OpenTelemetry is the spec-compliant way to send traces to Mibo. If your stack — agent framework, app server, LLM SDK, or workflow tool — emits OTel traces, you can point its exporter at Mibo’s ingestion endpoint and start evaluating real interactions immediately. No glue code, no custom body shape.

The other supported path is Your API — Mibo’s canonical {spans:[...]} JSON posted directly. Pick OTLP when you have an exporter; Your API when you don’t (curl scripts, n8n Cloud, Flowise Cloud, any managed platform). See Sending Traces for the decision matrix.

Note

OTLP works across agent types. You don’t need a specific agent type to use it — your agent can be n8n, Flowise, or HTTP API, and OTLP traces are accepted the same way on all of them. The agent type only controls how active tests are driven; ingestion is decoupled.

How it works

You configure your OTel exporter to send traces to Mibo’s HTTP endpoint. Mibo receives the OTLP envelope, stores it as a trace, and runs your test cases against it (passive testing).

[your instrumented system] --(OTLP/HTTP JSON)--> [Mibo] --> [test cases run]

Exporter configuration

Most OTel SDKs accept these environment variables (Node, Python, Go, Java, etc.):

OTEL_EXPORTER_OTLP_TRACES_PROTOCOL=http/json
OTEL_EXPORTER_OTLP_TRACES_ENDPOINT=https://api.mibo-ai.com/public/traces
OTEL_EXPORTER_OTLP_TRACES_HEADERS=x-api-key=<your_api_key>

Caution

Only OTLP/HTTP JSON is accepted. If your exporter defaults to protobuf or gRPC, switch it to http/json. Protobuf support is not on the roadmap.

Language support matrix

JSON encoding (http/json) is a spec-level OTLP option, but not every language’s first-party exporter implements it. The table below covers the SDKs we’ve tested against Mibo’s endpoint.

Language / SDK	OTLP/HTTP JSON support	Notes
Node.js — @opentelemetry/exporter-trace-otlp-http	✅ Built-in	The env vars above work as-is.
Go — go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp	✅ Built-in	Pass otlptracehttp.WithProtocol("http/json") or set the env var.
Java — io.opentelemetry:opentelemetry-exporter-otlp	✅ Built-in (autoconfigure ≥ 1.31)	Honors OTEL_EXPORTER_OTLP_TRACES_PROTOCOL=http/json.
.NET — OpenTelemetry.Exporter.OpenTelemetryProtocol	✅ Built-in	Default is protobuf; for JSON, set Protocol = HttpJson or use the OTEL_EXPORTER_OTLP_TRACES_PROTOCOL=http/json env var.
Python — opentelemetry-exporter-otlp-proto-http	❌ Protobuf only	This package always sends Content-Type: application/x-protobuf and ignores OTEL_EXPORTER_OTLP_TRACES_PROTOCOL=http/json. See Python workaround below.

Python — sending JSON to Mibo

Python’s first-party OTel HTTP exporter only emits protobuf, which Mibo rejects with OTLP_PROTOBUF_UNSUPPORTED (415). Until a first-party JSON exporter ships, two paths work:

1. Custom SpanExporter (recommended for in-process tracing). A ~40-line SpanExporter that serializes ReadableSpans to OTLP JSON resourceSpans and POSTs with Content-Type: application/json. You keep the real OTel SDK — tracer, context, semconv constants, BatchSpanProcessor — and only swap the wire-format step. This approach keeps all OTel instrumentation benefits (auto context propagation, semantic conventions, batching).

2. Your API trace shape (recommended for one-shot scripts). Skip OTLP entirely and POST the Mibo Your API JSON shape ({platformId, spans:[...]}) directly. See Your API. You lose the OTel SDK ergonomics (auto context propagation, batching) but the request is a single httpx.post.

Both paths land in the same internal representation — your test cases don’t change.

Configuring trace routing

OTLP traces are routed to a Mibo agent by their service.name resource attribute. Each agent in Mibo has an OTLP service name field; when an OTLP trace arrives, Mibo looks up the agent in the request’s project whose OTLP service name equals service.name.

1. Set OTLP service name on your agent. In the dashboard, open the agent and set its OTLP service name to a value of your choice (e.g. support-agent-prod). The value must be unique within the project.

2. Configure your exporter to send the same value. Either of these works:

   OTEL_SERVICE_NAME=support-agent-prod
   # or
   OTEL_RESOURCE_ATTRIBUTES=service.name=support-agent-prod

3. Use an API key that’s allowed to write to that agent (a per-agent key or a multi-agent key whose allowlist includes it).

Note

mibo.platform_id resource attribute is not used for OTLP routing — only service.name is. (Custom API / Your API uses metadata.mibo.platform_id or top-level platformId instead; see Your API.)

Failure modes:

• 400 OTLP_SERVICE_NAME_REQUIRED — service.name resource attribute is missing.
• 404 OTLP_PLATFORM_NOT_FOUND — no agent in this project has the matching OTLP service name.
• 403 OTLP_PLATFORM_NOT_ALLOWED — match found, but the API key isn’t authorized for that agent.

Special platform setups

n8n self-hosted (≥ 2.19.0)

n8n self-hosted has a built-in OTel tracer. See Send traces from n8n for the full setup — env vars, docker-compose, multi-backend config, and all n8n options.

Flowise self-hosted

Flowise self-hosted exposes the same OTel env vars. The same five lines above (with service.name=flowise) point it at Mibo.

Advanced configuration

W3C traceparent propagation

W3C trace-context propagation headers (traceparent, tracestate) are not currently supported for cross-service tracing. If the upstream caller injects a traceparent header into your service, Mibo ignores it — span hierarchy is built from parentSpanId on each span you export. Configure your SDK to extract traceparent into local spans within your service, establishing the parent-child relationship before exporting to Mibo.

What Mibo reads from your spans

Mibo’s assertions look up well-known OTel GenAI semantic-convention attributes. If your instrumentation follows the OTel GenAI conventions, everything works out of the box.

Attribute mapping

Span attribute	Mibo behavior	Assertion target
gen_ai.response.text	Default text for semantic assertions; Mibo reads from the root span first, then falls back to the most-recent descendant span if absent	semantic assertions, response_regex
gen_ai.tool.name	Treats the span as a tool call; nested under the parent span’s tool_calls	tool_call assertion
gen_ai.tool.call.arguments	Parsed as JSON if possible, else stored raw under value	tool_call.expected_arguments
gen_ai.usage.input_tokens / gen_ai.usage.output_tokens	Pulled from root, summed across descendants if absent	token_limit
http.response.status_code	Pulled from root, fallback to any descendant span	http_status
Span name	Every non-tool span (at any depth) becomes a node_call keyed by name (substring, case-insensitive match). Nested spans match too — scope your expected_name accordingly.	node_call.expected_name
Root span start/end_time_unix_nano	Wall-clock duration of the trace, in milliseconds	response_time.max_ms
parent_span_id	Defines the span tree; tool-spans attach to their parent’s tool_calls	(structural — not a target)

Common assertion examples

When you write a test case, Mibo uses the attributes above to evaluate assertions:

• Response time: response_time { max_ms: 5000 } — Mibo computes (root.end - root.start) / 1e6 in milliseconds.
• HTTP status: http_status { expected_status: 200 } — Mibo reads http.response.status_code from the root span, falling back to descendant spans if absent.
• Token usage: token_limit { max_total_tokens: 10000 } — Mibo reads gen_ai.usage.input_tokens and gen_ai.usage.output_tokens, summing them across the trace.
• Span name matching: node_call { expected_name: "search" } — matches any span whose name contains “search” (substring match, case-insensitive).
• Tool calls: tool_call { expected_name: "create_booking" } — matches a child span whose gen_ai.tool.name is exactly create_booking.
• Text-based assertions: Any semantic or regex assertion uses gen_ai.response.text as the default source. Override per-assertion with target_node to point at a specific span name instead.

Asserting on your own span attributes

Beyond the standard semconv keys, you can attach arbitrary attributes to your spans and assert on them directly with json_match and json_schema. Combine target_node (substring match on span name) with field (literal attribute key) to point at any span’s attribute.

For example, if your agent’s orchestrator span carries an attribute like myco.workflow.status with values such as "READY" / "WAITING_FOR_USER", you can gate other assertions on it:

{
  "target": "json_match",
  "target_node": "orchestrator",
  "field": "myco.workflow.status",
  "expected_value": "READY",
  "id": "ready_gate"
}

And validate the shape of a structured payload on another span:

{
  "target": "json_schema",
  "target_node": "generator",
  "field": "myco.output_payload",
  "depends_on": "ready_gate",
  "schema": { "type": "array", "minItems": 1 }
}

Notes:

• field is the literal attribute key, not a dot-path — OTel attribute names commonly contain dots (http.request.method, myco.foo.bar) and are looked up verbatim.
• OTel attribute values can’t be dicts or lists-of-dicts. JSON-stringify structured payloads (e.g. myco.output_payload = JSON.stringify(payload)) — json_schema parses the string before validating.
• target_node is a substring, case-insensitive match against span.name. Pick distinctive names to avoid collisions.
• This works with any OTel-instrumented system, not just GenAI semconv — pick a namespace under your own company prefix (e.g. myco.*) to avoid clashing with future semconv keys.

Trace identity and incremental exports

Mibo uses the first span’s traceId (or an x-request-id header, if you set one) as the trace’s external identifier. The endpoint upserts by traceId: the first POST creates the trace and fires your test cases, every subsequent POST with the same traceId merges its spans into the existing trace by span_id.

How batching affects trace assembly

This is how OTel exporters work in practice:

• SimpleSpanProcessor ships each span as it ends — a 10-span trace = 10 POSTs
• BatchSpanProcessor flushes every scheduledDelayMillis (5s default) — a long-running trace splits across multiple batches

Either way, Mibo stitches them together on the server, so a single logical trace ends up as a single record in your dashboard.

Timing of passive test execution

Passive evaluation is debounced: Mibo waits ~5 s after the latest batch lands before running test cases, with a 60 s cap from the first batch. So a multi-batch trace is normally evaluated once, after all batches have arrived (idempotent retries that add no new spans don’t trigger another execution).

You don’t have to change SimpleSpanProcessor / BatchSpanProcessor settings to get the right behavior — the server handles assembly. Only consider tuning the exporter if your trace genuinely takes longer than ~60 s of wall-clock to finish.

Error handling and resilience

Response codes

The Mibo ingestion endpoint returns standard HTTP status codes. Your exporter should handle them as follows:

• 200 OK / 201 Created — trace accepted. Passive evaluation is queued automatically.
• 400 OTLP_SERVICE_NAME_REQUIRED — service.name resource attribute is missing.
• 400 OTLP_NO_SPANS — the OTLP envelope contained zero spans.
• 400 VALIDATION_ERROR — invalid OTLP shape (missing traceId, invalid spans, etc.).
• 401 UNAUTHORIZED — API key is missing, invalid, revoked, or expired.
• 403 OTLP_PLATFORM_NOT_ALLOWED — service.name matched an agent, but the API key isn’t authorized for it.
• 404 OTLP_PLATFORM_NOT_FOUND — no agent in this project has an OTLP service name matching service.name. Set it in the dashboard or fix OTEL_SERVICE_NAME.
• 413 PAYLOAD_TOO_LARGE — trace exceeds 30 MB decompressed. Reduce batch size via maxExportBatchSize.
• 415 OTLP_PROTOBUF_UNSUPPORTED — you sent protobuf instead of JSON. Ensure OTEL_EXPORTER_OTLP_TRACES_PROTOCOL=http/json is set.
• 500+ errors — transient server issue. Most OTel exporters retry automatically with exponential backoff — no action needed. The trace will be retried.

Idempotency

OTel’s built-in batching and merging provide natural idempotency:

• Mibo merges spans by (traceId, span_id) across multiple POSTs.
• If a batch POST times out, resending it is safe — duplicate spans are deduplicated by span_id.
• Optionally, set x-request-id as an HTTP header in your exporter config for explicit deduplication by external id in addition to traceId.

Operational concerns

Payload size limits

The endpoint accepts up to 30 MB decompressed per request. Verbose GenAI instrumentation (full prompts, completions, and tool arguments in attributes) can hit this ceiling with batches of 50–100 spans.

If you see HTTP 413 (Payload Too Large):

1. Reduce maxExportBatchSize in your BatchSpanProcessor config — 10–20 spans per batch is a healthy default
2. Alternatively, filter verbose attributes before export if your SDK supports it
3. Consider sampling to export only a subset of traces to production if payload size is persistent

PII and sensitive data

OTel GenAI instrumentation packages (@opentelemetry/instrumentation-openai, traceloop/instrumentation-langchain, and similar) capture request/response payloads by default — including prompts, completions, and tool arguments. If your system handles user PII:

• Review your instrumentation SDK config before sending traces to Mibo — most packages support disabling request/response capture via environment variables or config flags
• Understand what’s captured — prompts, tool arguments, and LLM responses can all contain sensitive user data
• Know the visibility model — traces are encrypted at rest and in transit, but attribute contents are visible in the Mibo dashboard to any team member with access
• Consider sampling strategies — in production, you can sample only a subset of traces to reduce PII exposure while still evaluating real interactions

Testing your setup

Quick sanity check via curl

Send a minimal OTLP trace to verify your endpoint and API key are configured correctly:

curl -X POST https://api.mibo-ai.com/public/traces \
  -H "x-api-key: $MIBO_API_KEY" \
  -H "content-type: application/json" \
  -d '{
    "resourceSpans": [{
      "resource": {
        "attributes": [
          { "key": "service.name", "value": { "stringValue": "my-agent" } }
        ]
      },
      "scopeSpans": [{
        "spans": [{
          "traceId": "5b8aa5a2d2c872e8321cf37308d69df2",
          "spanId": "root-span-0001",
          "name": "agent.run",
          "startTimeUnixNano": "1717000000000000000",
          "endTimeUnixNano": "1717000001500000000",
          "attributes": [
            { "key": "gen_ai.response.text", "value": { "stringValue": "Hello!" } }
          ],
          "status": { "code": 1 }
        }]
      }]
    }]
  }'

Understanding the response

• 2xx with {} — The trace was accepted. It will appear in your Mibo dashboard within seconds.
• 400 — Missing or invalid API key, or mibo.platform_id is required but missing
• 413 — Payload is too large; reduce batch size (see Payload size limits)
• 415 OTLP_PROTOBUF_UNSUPPORTED — You sent protobuf instead of JSON (common with Python; see Python workaround)

Once accepted, the trace will be evaluated against your test cases on the next test run (passive testing is scheduled, not immediate).

Troubleshooting

Problem	Likely cause	Solution
415 OTLP_PROTOBUF_UNSUPPORTED	You sent protobuf instead of JSON (common in Python)	Set OTEL_EXPORTER_OTLP_TRACES_PROTOCOL=http/json. For Python, use a custom SpanExporter or switch to Your API. See Python workaround.
400 VALIDATION_ERROR	Your OTLP shape is malformed or missing required fields	Check that you’re sending valid OTLP/HTTP JSON with resourceSpans, not protobuf or gRPC. See Error handling.
400 platform resolution failed	Multi-agent API key without mibo.platform_id resource attribute	Set OTEL_RESOURCE_ATTRIBUTES=mibo.platform_id=<platform_uuid> to specify the target agent.
401 UNAUTHORIZED	API key is missing, invalid, revoked, or expired	Verify your key with echo $OTEL_EXPORTER_OTLP_TRACES_HEADERS and check the dashboard for expiration or revocation.
413 Payload Too Large	Your batch contains too many spans or very large attributes	Reduce maxExportBatchSize to 10–20 spans. Filter verbose attributes before export if possible. See Payload size limits.
Traces appear but tests don’t run	Tests run asynchronously, not on every POST	Refresh the dashboard after a few seconds. Check that your test cases are enabled. Passive evaluation only runs on the first POST for a traceId.
Traces missing gen_ai.* attributes	Your instrumentation doesn’t follow OTel GenAI conventions	Review your SDK config and enable the relevant instrumentation packages. See Attribute mapping.

API Keys & Trace Routing Create per-platform keys and configure routing.

Passive Testing How traces flow through your test cases.

Assertion Reference All assertion types and matchers.