Deploy agents

Blaxel provides a serverless infrastructure to instantly deploy AI agents. You receive a global inference endpoint for each deployment, and your workloads are served optimally to dramatically accelerate cold-start and latency.

The main way to deploy an agent on Blaxel is by using Blaxel CLI. This method is detailed down below on the page. Alternatively you can deploy agents from a variety of pre-built templates on the Blaxel web console, or connect a GitHub repository: any push to the main branch will automatically update the deployment on Blaxel.

​

Deploy an agent with Blaxel CLI

This section assumes you have developed an agent locally, as explained in this documentation, and are ready to deploy it.

The Blaxel SDK allows you to connect to and orchestrate other resources (such as model APIs, tool servers, multi-agents) during development, and ensures telemetry, secure connections to third-party systems or private networks, smart global placement of workflows, and much more when agents are deployed.

This packaging makes Blaxel fully agnostic of the framework used to develop your agent and doesn’t prevent you from deploying your software on another platform.

​

Serve locally

You can serve the agent locally in order to make the main.py / main.ts main function available on a local endpoint.

Run the following command to serve the agent:

bl serve

Calling the provided endpoint will execute the agent locally while sandboxing the core agent logic, function calls and model API calls exactly as it would be when deployed on Blaxel. Add the flag --hotreload to get live changes.

bl serve --hotreload

​

Deploy on production

You can deploy the agent in order to make the main.py / main.ts main function callable on a global endpoint. When deploying to Blaxel, you get a dedicated endpoint that enforces your deployment policies.

Run the following command to build and deploy a local agent on Blaxel:

bl deploy

​

Overview of deployment life-cycle

​

Deploying an agent

Deploying an agent will create the associated agent deployment. At this time:

• it is reachable through a specific endpoint
• it does not consume resources until it is actively being invoked and processing inferences
• its status can be monitored either on the console or using the CLI/APIs

​

Managing revisions

As you iterate on software development, you will need to update the version of an agent that is currently deployed and used by your consumers. Every time you build a new version of your agent, this creates a revision. Blaxel stores the 10 latest revisions for each object.

Revisions are atomic builds of your deployment that can be either deployed (accessible via the inference endpoint) or not. This system enables you to:

• rollback a deployment to its exact state from an earlier date
• create a revision without immediate deployment to prepare for a future release
• implement progressive rollout strategies, such as canary deployments

Important: Revisions are not the same as versions. You cannot use revisions to return to a previous configuration and branch off from it. For version control, use your preferred system (such as GitHub) alongside Blaxel.

Deployment revisions are updated following a blue-green paradigm. The Global Inference Network will wait for the new revision to be completely up and ready before routing requests to the new deployment. You can also set up a canary deployment to split traffic between two revisions (maximum of two).

​

Executions and inference requests

Executions (a.k.a inference executions) are ephemeral invocations of agent deployments by a consumer. Because Blaxel is serverless, an agent deployment is only materialized onto one of the execution locations when it actively receives and processes requests. Workload placement and request routing is fully managed by the Global Inference Network, as defined by your environment policies.

Read more about querying agents in this documentation.

​

Deactivating an agent deployment

Any agent deployment can be deactivated at any time. When deactivated, it will no longer be reachable through the inference endpoint and will stop consuming resources.

Agents can be deactivated and activated at any time from the Blaxel console, or via API or CLI.

​

Agent deployment reference

​

Deploy agent from configuration file (advanced usage)

The bl deploy command generates a YAML configuration manifest automatically and deploys it to Blaxel’s hosting infrastructure. You can also create custom manifest files in the .blaxel folder and deploy them using the following command:

bl apply -f ./my-agent-deployment.yaml

Read our reference for agent deployments.

​

Model API

You must choose one action model, which will be the reasoning and talking core of the agent. The model must be a model API referenced on Blaxel.

Read about the API parameters in the reference.

​

Functions

Select one or multiple functions to equip your agent with the ability to run custom code. This is optional, in which case your agent will only be able to talk.

Read about the API parameters in the reference.

​

Chaining and multi-agents

Multi-agent systems allow to better specialize each agent, with their specific set of tools and instructions.

You can chain other agents to an agent on Blaxel. When processing a consumer query, the agent will be able to handover the request to another agent that is chained to it if the action model considers it the best way to address the query.

Read about the API parameters in the reference.

​

Policies

Policies can be optionally attached to an agent deployment directly.

Read about the API parameters in the reference.

​

Resources

Select the memory size to allocate to the execution of the agent.

Read about the API parameters in the reference.