Deploying and running agents on Blaxel gives you total observability by design, without needing to install any additional library. When you either deploy from the console’s low-code builder or by using the Blaxel SDK to wrap your code for deployment, Blaxel will automatically implement logging & tracing on your requests.
Monitor from the Blaxel console
There are three ways you can observe and monitor your running workloads:
Metrics
Metrics are aggregated data about workload executions. They include:
- Number and rate of requests
- End-to-end latency: average, p50, p90, p99
- Number of tokens generated by model APIs: input, output and total
- City and country of origin of all requests
Logs
Logs are timestamped data about what happens with your agents, functions and model APIs. Such data includes:
- Status of all requests on agents, functions and model APIs
- Logs generated when building your deployments
- Custom logs generated by your agents and functions
Traces
Tracing helps you understand how your agent works by showing you all its parts in action. When you make a request, it creates a trace that contains multiple spans.
Think of a span as a building block - it shows when something starts and ends, what went in, what came out, and other useful details. Spans can contain other spans, like when one function calls another. You’ll often see spans for things like LLM calls, tool calls, or steps in an agent’s process. You can click on any trace to see all its spans laid out clearly. This makes it much easier to follow how your agent works and fix any problems you find.
Blaxel collects and saves the traces of a sampled 10% of all your executions. In order to force saving the trace on an execution, call the run API and add query parameter debug:true.
Opt out
This feature is controlled by the BL_ENABLE_OPENTELEMETRY environment variable.
When you deploy an agent to Blaxel, the platform automatically injects BL_ENABLE_OPENTELEMETRY=true into the environment.
When developing locally, this environment variable is not set and therefore defaults to false.
To explicitly disable telemetry:
export BL_ENABLE_OPENTELEMETRY=false
Setting DO_NOT_TRACK=1 has no effect on OpenTelemetry.
