Install Relay

Use this page as the implementation handoff. Both paths use a project-scoped Relay key, but they change different parts of the stack.

Send this section to	Choose	What changes
Application or AI platform owner	Live	The LLM client uses Relay’s base URL and project key.
Observability or platform owner	Observe	The existing trace pipeline adds Relay as another destination.

A workspace admin can open Relay → Get Started, choose Live or Observe, and select Copy AI prompt. That prompt includes a newly created project-scoped key and the workspace’s exact endpoint. The public prompts below use placeholders so this page is safe to share.

Before you install

Sign in to Oximy and open Relay → Get Started.
Select the automatically created Default project, or choose the project that should own this workload’s keys, traffic, limits, and reports.
Create a project key. Store the full ox_... value in the application’s existing ignored secret store; Relay only reveals it once.
Choose Live or Observe below. A project can use both.

Live: route through Relay

Live is for teams that want Relay to execute the model request. Keep the application’s prompts, request options, and control flow; change the compatible client’s base URL and authentication. Use https://relay.oximy.com/v1 for OpenAI-compatible clients. Start with a provider-qualified model configured in the project, or use oximy/auto after enabling automatic routing.

import os
from openai import OpenAI

client = OpenAI(
    base_url="https://relay.oximy.com/v1",
    api_key=os.environ["OXIMY_API_KEY"],
)

response = client.chat.completions.create(
    model="<PROVIDER/MODEL>",
    messages=[{"role": "user", "content": "Hello from Relay"}],
)
print(response.choices[0].message.content)

import OpenAI from 'openai';

const client = new OpenAI({
  baseURL: 'https://relay.oximy.com/v1',
  apiKey: process.env.OXIMY_API_KEY,
});

const response = await client.chat.completions.create({
  model: '<PROVIDER/MODEL>',
  messages: [{ role: 'user', content: 'Hello from Relay' }],
});
console.log(response.choices[0].message.content);

import os
from anthropic import Anthropic

client = Anthropic(
    base_url="https://relay.oximy.com",
    api_key=os.environ["OXIMY_API_KEY"],
)

message = client.messages.create(
    model="anthropic/<MODEL>",
    max_tokens=256,
    messages=[{"role": "user", "content": "Hello from Relay"}],
)
print(message.content[0].text)

import { createOpenAI } from '@ai-sdk/openai';
import { generateText } from 'ai';

const relay = createOpenAI({
  baseURL: 'https://relay.oximy.com/v1',
  apiKey: process.env.OXIMY_API_KEY,
});

const { text } = await generateText({
  model: relay('<PROVIDER/MODEL>'),
  prompt: 'Hello from Relay',
});
console.log(text);

import os
from langchain_openai import ChatOpenAI

llm = ChatOpenAI(
    base_url="https://relay.oximy.com/v1",
    api_key=os.environ["OXIMY_API_KEY"],
    model="<PROVIDER/MODEL>",
)
print(llm.invoke("Hello from Relay").content)

curl https://relay.oximy.com/v1/chat/completions \
  -H "Authorization: Bearer $OXIMY_API_KEY" \
  -H "Content-Type: application/json" \
  -H "x-session-id: first-session" \
  -d '{
    "model": "<PROVIDER/MODEL>",
    "messages": [{"role": "user", "content": "Hello from Relay"}]
  }'

Use a stable x-session-id when the client supports custom headers so related requests appear as one Agent Run. Open Relay → Traffic after the first call and confirm the requested model, resolved model, status, cost, and latency.

Live AI install prompt

Replace <YOUR_RELAY_KEY> and paste this prompt into the coding agent that can edit the application:

Integrate this existing application with Oximy Relay in Live mode.

Relay base URL: https://relay.oximy.com/v1
Relay project key: <YOUR_RELAY_KEY>

First inspect the repository for its language, package manager, LLM SDK, client construction, model call sites, existing configuration, and ignored secret mechanism. Do not scaffold a new project.

Make the smallest focused change that routes compatible LLM calls through Relay:
1. Store the Relay key only in the application's existing ignored secret mechanism. Never commit it, put it in source, print it, or include it in your response.
2. Change the existing OpenAI-compatible client's base URL to https://relay.oximy.com/v1 and use the Relay key for that client. If the app uses Anthropic's SDK directly, preserve its Messages API shape and use https://relay.oximy.com as its base URL.
3. Preserve prompts, model IDs, request options, streaming, retries, tools, and application control flow. Do not rewrite working model logic.
4. If the app has a stable conversation or job ID and the SDK supports custom headers, send it as x-session-id.
5. Run the narrowest existing validation, then report the files changed and the exact command for one identifiable request. Do not echo the key.

Observe: keep the current provider path

Observe is for teams that want Relay traffic, cost, latency, and trace analysis without putting Relay in the model execution path. The application keeps its current provider or gateway. Add Relay as another OTLP/HTTP trace destination. The exact trace endpoint is:

https://relay.oximy.com/v1/traces

Python
TypeScript
OTEL Collector

Install the standard OpenTelemetry packages if they are not already present:

pip install opentelemetry-sdk opentelemetry-exporter-otlp-proto-http

Add Relay alongside the processors already attached to the tracer provider:

import os
from opentelemetry import trace
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor

provider = trace.get_tracer_provider()
if provider.__class__.__name__ == "ProxyTracerProvider":
    provider = TracerProvider()
    trace.set_tracer_provider(provider)
elif not hasattr(provider, "add_span_processor"):
    raise RuntimeError("Add Relay through the existing provider's extension point.")

relay_exporter = OTLPSpanExporter(
    endpoint="https://relay.oximy.com/v1/traces",
    headers={"Authorization": f"Bearer {os.environ['OXIMY_API_KEY']}"},
)
provider.add_span_processor(BatchSpanProcessor(relay_exporter))

Install the standard Node packages if they are not already present:

npm install @opentelemetry/sdk-node @opentelemetry/sdk-trace-base @opentelemetry/exporter-trace-otlp-http

Add the Relay processor to the existing NodeSDK configuration. Preserve every processor already in spanProcessors:

import { OTLPTraceExporter } from '@opentelemetry/exporter-trace-otlp-http';
import { NodeSDK } from '@opentelemetry/sdk-node';
import { BatchSpanProcessor } from '@opentelemetry/sdk-trace-base';

const relayExporter = new OTLPTraceExporter({
  url: 'https://relay.oximy.com/v1/traces',
  headers: { Authorization: `Bearer ${process.env.OXIMY_API_KEY}` },
});

const sdk = new NodeSDK({
  // Keep the application's existing resource, instrumentation, and processors.
  spanProcessors: [
    ...existingSpanProcessors,
    new BatchSpanProcessor(relayExporter),
  ],
});

existingSpanProcessors represents the processors already configured by the application. Do not replace them with this example.

When an OpenTelemetry Collector owns export, fan out from the Collector instead of changing every application:

exporters:
  otlphttp/oximy_relay:
    traces_endpoint: https://relay.oximy.com/v1/traces
    headers:
      Authorization: Bearer ${env:OXIMY_API_KEY}

service:
  pipelines:
    traces:
      exporters: [<EXISTING_EXPORTERS>, otlphttp/oximy_relay]

Keep every existing exporter in the pipeline. The Collector’s OTLP/HTTP exporter sends traces to the exact traces_endpoint above.

Do not set or replace global OTEL_EXPORTER_* variables in an application that already exports telemetry; that can redirect its existing observability pipeline. Preserve the current provider, exporters, processors, resources, propagation, instrumentation, and shutdown behavior.

Observe AI install prompt

Replace <YOUR_RELAY_KEY> and send this prompt to the application owner, or to the observability owner when a Collector manages export:

Add Oximy Relay Observe telemetry to this existing application.

OTLP/HTTP trace endpoint: https://relay.oximy.com/v1/traces
Relay project key: <YOUR_RELAY_KEY>

First inspect the repository for its language, package manager, existing OpenTelemetry setup, tracer provider, processors, exporters, LLM instrumentation, resource attributes, propagation, and shutdown lifecycle. Also determine whether an OpenTelemetry Collector or another platform component owns export. Do not scaffold a new project.

Keep every LLM request on its current provider or gateway. Make the smallest focused change that adds Relay as another trace destination:
1. Store the Relay key only in the application's existing ignored secret mechanism. Never commit it, put it in source, print it, or include it in your response.
2. Use the standard OTLP/HTTP trace exporter with endpoint https://relay.oximy.com/v1/traces and Authorization: Bearer <YOUR_RELAY_KEY>. Configure it in code; do not set or replace OTEL_EXPORTER_* environment variables.
3. Add that exporter through a new BatchSpanProcessor on the existing tracer provider. Preserve every existing provider, processor, exporter, resource, propagator, and instrumentation. If no SDK provider exists, create one once. If a Collector owns export, add Relay to the Collector's trace exporters instead.
4. Reuse existing LLM semantic-convention instrumentation. If none exists, instrument the real call sites with provider, model, token, latency, status, and error attributes. Do not capture prompts or responses unless the application's privacy policy explicitly permits sending them to Oximy.
5. Preserve the current flush and shutdown behavior and ensure the new batch processor is flushed on shutdown.
6. Run the narrowest existing validation, then report the files changed and the exact command for one identifiable request. Do not echo the key.

Generate that request, then open Relay → Traffic and filter the source to OTEL. First confirm that the trace arrived; then validate model, token, cost, latency, status, and LLM attributes supplied by the instrumentation.

Live and Observe tokens count toward Oximy metering. Relay-created Shadow candidate rows do not, although the model provider may still charge for the candidate request.

Understand integration modes

Compare Live, Observe, existing-gateway, and route-only responsibilities.

Troubleshoot installation

Diagnose authentication, request, trace, and source problems.

​Before you install

​Live: route through Relay

​Live AI install prompt

​Observe: keep the current provider path

​Observe AI install prompt

Understand integration modes

Troubleshoot installation

Before you install

Live: route through Relay

Live AI install prompt

Observe: keep the current provider path

Observe AI install prompt