Skip to main content

Getting Started with Real-Time Error Tracking

Building reliable LLM and agentic applications isn’t just about preventing crashes. It’s about catching the quiet failures—when your agent technically responds, but does the wrong thing. That could mean making up a refund policy, skipping required steps, or going off on an inappropriate tangent with overconfidence. These errors don’t show up in logs or crash reports. They just happen—and they affect real users.

Okareo helps you catch and fix those invisible failures. It acts as an always-on watchdog that monitors every interaction with your agents and monitors what they say, decide, and do. Whether your app is fielding support requests, guiding internal workflows, or responding to sales leads, Okareo helps you spot when things silently go off the rails.

In this guide, we’ll walk through how to connect your app to Okareo in minutes—so you can start detecting issues like hallucinated answers, broken flows, or safety violations in real time.

Why Teams Use Error Tracking?​

Real-Time Detection Prompt evals and offline tests miss the moment.

  • A customer-facing agent responded with “we’ll cover that cost,” even though the refund policy said otherwise. There was no alert—until it was flagged as a scope violation.

Replay + Root Cause See the full reasoning chain: tools, memory, prompts, and all decision steps.

  • A customer reported getting stuck during onboarding. Nothing showed up in logs. Replaying the agent revealed it skipped an identity verification step—because the memory state was overwritten mid-flow. Silent failure, invisible without a replay.

Failure Pattern Detection Surface patterns like off-topic responses, stuck conversations, or silent fallbacks.

  • A sales assistant started looping back to the same question whenever users asked about compliance.

Scope & Safety Monitoring Define what your agent is allowed to say and do. Get alerted the moment it deviates.

  • A support bot started suggesting legal advice after a retrieval miss.

How It Works​

To enable error tracking Okareo will inspect your LLM requests and responses and evaluate them for issues in real time. There are several options to collect your LLM requests and responses. You can use OpenTelemetry standard, cloud or self-hosted proxy, or send data points directly to Okareo via APIs. You control what data is shared and stored.

Okareo Proxy in Context

Issues vs. Errors: What’s the Difference?​

Throughout this guide, we will talk about "issues” and “errors” in the context of LLM/Agent behavior. Distinction between the two helps developers prioritize and understand problems:

  • Issues (Behavioral Issues): An Issue represents any notable unwanted behavior or outcome from the AI, even if the system didn’t technically fail. These are the subtle logic or content problems that traditional monitoring would miss. For example, if your customer support bot suddenly starts discussing financial advice when it’s not supposed to, that’s an issue. The app didn’t crash and the model gave a response – but it’s a behavior you consider incorrect or off-policy. Okareo would detect this condition and provide an explanation along with the dialog history showing where the finance topic came up. You can quickly grasp why it was flagged and decide on a fix (e.g., adjust the prompt or add a content filter for finance terms).

Issue Example

  • Errors (System/Configuration Errors): An Error usually refers to a more concrete failure in the model interaction – something that is undeniably wrong in execution. These often correspond to exceptions, misconfigurations, or invalid model responses. For example, using a wrong parameter or model setting or hitting a rate limit can be considered errors. Imagine you set the model’s temperature to an out-of-bounds value, or sent context that is too large for a given model – those are errors.

Error Example

In practice, the Okareo dashboard and notifications may label certain model responses as issues and others as errors. Issues are derived from checks failing, whereas errors might come from the model or system events (like exceptions or explicit refusals). Both are important: issues tell you where your model's behavior could be improved or corrected, and errors tell you where your request completely failed. Okareo’s real-time tracking covers both sides – catching silent logic issues and explicit errors – giving you a full picture of your system’s reliability.

Setting Up Error Tracking​

Step 1: Connect Your App to the Okareo Proxy​

To start tracking your LLM's behavior, easiest way is to route your LLM API calls through the Okareo proxy. This proxy sits between your application and the LLM, enabling Okareo to inspect each request/response and evaluate it for issues in real time. Getting set up is quick:

Sign Up and Get an Okareo API Token: Export it as an environment variable (e.g. OKAREO_API_KEY) or have it handy for the next step.

Configure Your LLM Client to Use the Okareo Proxy: Modify your application’s API calls so they go through Okareo. In practice, this means pointing the base URL to https://proxy.okareo.com (or your self-hosted proxy address) and adding your Okareo API key in the headers. You’ll still use your original LLM provider API key for the actual model access. For example, using different client types:

curl https://proxy.okareo.com/v1/chat/completions \
-H "Content-Type: application/json" \
-H "Authorization: Bearer <YOUR_LLM_PROVIDER_KEY>" \
-H "api-key: <OKAREO_API_KEY>" \
-d '{
    "model": "gpt-4o",
    "messages": [
        {
            "role": "system",
            "content": "This is just a test."
        }
    ]
    }'
tip

Using Okareo proxy and unified OpenAI based API layer you can access most LLM providers and models, including self-hosted and open source models. Under the hood, Okareo will forward the request to the LLM provider and get the response, all while evaluating the interaction. (For other languages or SDKs, the concept is the same: set the base URL to the Okareo cloud proxy and include the api-key header.) Code snippets above are for the cloud hosted proxy and is the easiest way to get started. Okareo also provides a self-hosted option. For self-hosted proxy learn more in CLI docs.

Once this configuration is in place, every LLM call from your app will be monitored by Okareo. There’s no complex setup beyond these URL and header changes – your requests still reach the provider normally, but now Okareo can observe and analyze them in transit. As soon as responses come back, Okareo’s backend will automatically classify each completion with various Checks (built-in evaluation metrics) and flag any notable issues. In other words, simply routing through the proxy instruments your app for live error detection.