> ## Documentation Index
> Fetch the complete documentation index at: https://oximy.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Install Relay

> Share one guide for routing live requests through Relay or observing existing requests with OpenTelemetry.

Use this page as the implementation handoff. Both paths use a project-scoped
Relay key, but they change different parts of the stack.

| Send this section to             | Choose                                             | What changes                                                   |
| -------------------------------- | -------------------------------------------------- | -------------------------------------------------------------- |
| Application or AI platform owner | [Live](#live-route-through-relay)                  | The LLM client uses Relay's base URL and project key.          |
| Observability or platform owner  | [Observe](#observe-keep-the-current-provider-path) | The existing trace pipeline adds Relay as another destination. |

<Note>
  A workspace admin can open **Relay → Get Started**, choose **Live** or
  **Observe**, and select **Copy AI prompt**. That prompt includes a newly
  created project-scoped key and the workspace's exact endpoint. The public
  prompts below use placeholders so this page is safe to share.
</Note>

## Before you install

1. Sign in to Oximy and open **Relay → Get Started**.
2. Select the automatically created **Default** project, or choose the project
   that should own this workload's keys, traffic, limits, and reports.
3. Create a project key. Store the full `ox_...` value in the application's
   existing ignored secret store; Relay only reveals it once.
4. Choose **Live** or **Observe** below. A project can use both.

## Live: route through Relay

Live is for teams that want Relay to execute the model request. Keep the
application's prompts, request options, and control flow; change the compatible
client's base URL and authentication.

Use `https://relay.oximy.com/v1` for OpenAI-compatible clients. Start with a
provider-qualified model configured in the project, or use `oximy/auto` after
enabling automatic routing.

<Tabs>
  <Tab title="OpenAI Python">
    ```python theme={null}
    import os
    from openai import OpenAI

    client = OpenAI(
        base_url="https://relay.oximy.com/v1",
        api_key=os.environ["OXIMY_API_KEY"],
    )

    response = client.chat.completions.create(
        model="<PROVIDER/MODEL>",
        messages=[{"role": "user", "content": "Hello from Relay"}],
    )
    print(response.choices[0].message.content)
    ```
  </Tab>

  <Tab title="OpenAI TypeScript">
    ```typescript theme={null}
    import OpenAI from 'openai';

    const client = new OpenAI({
      baseURL: 'https://relay.oximy.com/v1',
      apiKey: process.env.OXIMY_API_KEY,
    });

    const response = await client.chat.completions.create({
      model: '<PROVIDER/MODEL>',
      messages: [{ role: 'user', content: 'Hello from Relay' }],
    });
    console.log(response.choices[0].message.content);
    ```
  </Tab>

  <Tab title="Anthropic Python">
    ```python theme={null}
    import os
    from anthropic import Anthropic

    client = Anthropic(
        base_url="https://relay.oximy.com",
        api_key=os.environ["OXIMY_API_KEY"],
    )

    message = client.messages.create(
        model="anthropic/<MODEL>",
        max_tokens=256,
        messages=[{"role": "user", "content": "Hello from Relay"}],
    )
    print(message.content[0].text)
    ```
  </Tab>

  <Tab title="Vercel AI SDK">
    ```typescript theme={null}
    import { createOpenAI } from '@ai-sdk/openai';
    import { generateText } from 'ai';

    const relay = createOpenAI({
      baseURL: 'https://relay.oximy.com/v1',
      apiKey: process.env.OXIMY_API_KEY,
    });

    const { text } = await generateText({
      model: relay('<PROVIDER/MODEL>'),
      prompt: 'Hello from Relay',
    });
    console.log(text);
    ```
  </Tab>

  <Tab title="LangChain Python">
    ```python theme={null}
    import os
    from langchain_openai import ChatOpenAI

    llm = ChatOpenAI(
        base_url="https://relay.oximy.com/v1",
        api_key=os.environ["OXIMY_API_KEY"],
        model="<PROVIDER/MODEL>",
    )
    print(llm.invoke("Hello from Relay").content)
    ```
  </Tab>

  <Tab title="cURL">
    ```bash theme={null}
    curl https://relay.oximy.com/v1/chat/completions \
      -H "Authorization: Bearer $OXIMY_API_KEY" \
      -H "Content-Type: application/json" \
      -H "x-session-id: first-session" \
      -d '{
        "model": "<PROVIDER/MODEL>",
        "messages": [{"role": "user", "content": "Hello from Relay"}]
      }'
    ```
  </Tab>
</Tabs>

Use a stable `x-session-id` when the client supports custom headers so related
requests appear as one Agent Run. Open **Relay → Traffic** after the first call
and confirm the requested model, resolved model, status, cost, and latency.

### Live AI install prompt

Replace `<YOUR_RELAY_KEY>` and paste this prompt into the coding agent that can
edit the application:

```text theme={null}
Integrate this existing application with Oximy Relay in Live mode.

Relay base URL: https://relay.oximy.com/v1
Relay project key: <YOUR_RELAY_KEY>

First inspect the repository for its language, package manager, LLM SDK, client construction, model call sites, existing configuration, and ignored secret mechanism. Do not scaffold a new project.

Make the smallest focused change that routes compatible LLM calls through Relay:
1. Store the Relay key only in the application's existing ignored secret mechanism. Never commit it, put it in source, print it, or include it in your response.
2. Change the existing OpenAI-compatible client's base URL to https://relay.oximy.com/v1 and use the Relay key for that client. If the app uses Anthropic's SDK directly, preserve its Messages API shape and use https://relay.oximy.com as its base URL.
3. Preserve prompts, model IDs, request options, streaming, retries, tools, and application control flow. Do not rewrite working model logic.
4. If the app has a stable conversation or job ID and the SDK supports custom headers, send it as x-session-id.
5. Run the narrowest existing validation, then report the files changed and the exact command for one identifiable request. Do not echo the key.
```

## Observe: keep the current provider path

Observe is for teams that want Relay traffic, cost, latency, and trace analysis
without putting Relay in the model execution path. The application keeps its
current provider or gateway. Add Relay as another OTLP/HTTP trace destination.

The exact trace endpoint is:

```text theme={null}
https://relay.oximy.com/v1/traces
```

<Tabs>
  <Tab title="Python">
    Install the standard OpenTelemetry packages if they are not already present:

    ```bash theme={null}
    pip install opentelemetry-sdk opentelemetry-exporter-otlp-proto-http
    ```

    Add Relay alongside the processors already attached to the tracer provider:

    ```python theme={null}
    import os
    from opentelemetry import trace
    from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
    from opentelemetry.sdk.trace import TracerProvider
    from opentelemetry.sdk.trace.export import BatchSpanProcessor

    provider = trace.get_tracer_provider()
    if provider.__class__.__name__ == "ProxyTracerProvider":
        provider = TracerProvider()
        trace.set_tracer_provider(provider)
    elif not hasattr(provider, "add_span_processor"):
        raise RuntimeError("Add Relay through the existing provider's extension point.")

    relay_exporter = OTLPSpanExporter(
        endpoint="https://relay.oximy.com/v1/traces",
        headers={"Authorization": f"Bearer {os.environ['OXIMY_API_KEY']}"},
    )
    provider.add_span_processor(BatchSpanProcessor(relay_exporter))
    ```
  </Tab>

  <Tab title="TypeScript">
    Install the standard Node packages if they are not already present:

    ```bash theme={null}
    npm install @opentelemetry/sdk-node @opentelemetry/sdk-trace-base @opentelemetry/exporter-trace-otlp-http
    ```

    Add the Relay processor to the existing `NodeSDK` configuration. Preserve
    every processor already in `spanProcessors`:

    ```typescript theme={null}
    import { OTLPTraceExporter } from '@opentelemetry/exporter-trace-otlp-http';
    import { NodeSDK } from '@opentelemetry/sdk-node';
    import { BatchSpanProcessor } from '@opentelemetry/sdk-trace-base';

    const relayExporter = new OTLPTraceExporter({
      url: 'https://relay.oximy.com/v1/traces',
      headers: { Authorization: `Bearer ${process.env.OXIMY_API_KEY}` },
    });

    const sdk = new NodeSDK({
      // Keep the application's existing resource, instrumentation, and processors.
      spanProcessors: [
        ...existingSpanProcessors,
        new BatchSpanProcessor(relayExporter),
      ],
    });
    ```

    `existingSpanProcessors` represents the processors already configured by
    the application. Do not replace them with this example.
  </Tab>

  <Tab title="OTEL Collector">
    When an OpenTelemetry Collector owns export, fan out from the Collector
    instead of changing every application:

    ```yaml theme={null}
    exporters:
      otlphttp/oximy_relay:
        traces_endpoint: https://relay.oximy.com/v1/traces
        headers:
          Authorization: Bearer ${env:OXIMY_API_KEY}

    service:
      pipelines:
        traces:
          exporters: [<EXISTING_EXPORTERS>, otlphttp/oximy_relay]
    ```

    Keep every existing exporter in the pipeline. The Collector's OTLP/HTTP
    exporter sends traces to the exact `traces_endpoint` above.
  </Tab>
</Tabs>

Do not set or replace global `OTEL_EXPORTER_*` variables in an application that
already exports telemetry; that can redirect its existing observability
pipeline. Preserve the current provider, exporters, processors, resources,
propagation, instrumentation, and shutdown behavior.

### Observe AI install prompt

Replace `<YOUR_RELAY_KEY>` and send this prompt to the application owner, or to
the observability owner when a Collector manages export:

```text theme={null}
Add Oximy Relay Observe telemetry to this existing application.

OTLP/HTTP trace endpoint: https://relay.oximy.com/v1/traces
Relay project key: <YOUR_RELAY_KEY>

First inspect the repository for its language, package manager, existing OpenTelemetry setup, tracer provider, processors, exporters, LLM instrumentation, resource attributes, propagation, and shutdown lifecycle. Also determine whether an OpenTelemetry Collector or another platform component owns export. Do not scaffold a new project.

Keep every LLM request on its current provider or gateway. Make the smallest focused change that adds Relay as another trace destination:
1. Store the Relay key only in the application's existing ignored secret mechanism. Never commit it, put it in source, print it, or include it in your response.
2. Use the standard OTLP/HTTP trace exporter with endpoint https://relay.oximy.com/v1/traces and Authorization: Bearer <YOUR_RELAY_KEY>. Configure it in code; do not set or replace OTEL_EXPORTER_* environment variables.
3. Add that exporter through a new BatchSpanProcessor on the existing tracer provider. Preserve every existing provider, processor, exporter, resource, propagator, and instrumentation. If no SDK provider exists, create one once. If a Collector owns export, add Relay to the Collector's trace exporters instead.
4. Reuse existing LLM semantic-convention instrumentation. If none exists, instrument the real call sites with provider, model, token, latency, status, and error attributes. Do not capture prompts or responses unless the application's privacy policy explicitly permits sending them to Oximy.
5. Preserve the current flush and shutdown behavior and ensure the new batch processor is flushed on shutdown.
6. Run the narrowest existing validation, then report the files changed and the exact command for one identifiable request. Do not echo the key.
```

Generate that request, then open **Relay → Traffic** and filter the source to
OTEL. First confirm that the trace arrived; then validate model, token, cost,
latency, status, and LLM attributes supplied by the instrumentation.

<Info>
  Live and Observe tokens count toward Oximy metering. Relay-created Shadow
  candidate rows do not, although the model provider may still charge for the
  candidate request.
</Info>

<CardGroup cols={2}>
  <Card title="Understand integration modes" icon="git-branch" href="/docs/relay/integration-modes">
    Compare Live, Observe, existing-gateway, and route-only responsibilities.
  </Card>

  <Card title="Troubleshoot installation" icon="stethoscope" href="/docs/relay/troubleshooting">
    Diagnose authentication, request, trace, and source problems.
  </Card>
</CardGroup>