Develop agents in TypeScript

You can bring agents developed in TypeScript in any framework (LangChain, CrewAI, or any custom framework) and deploy them on Blaxel by integrating a few lines of the Blaxel SDK and leveraging our other developer tools (Blaxel CLI, GitHub action, etc.).

​

Blaxel agent file structure

​

Overview

To deploy a custom AI agent on Blaxel, you need the following file in your repository:

• agent.ts (or src/agent.ts): This file must contain an async function that runs your agent. This function serves as the default entry point during serving and deployment.
  • You can specify a different module to serve using the flag bl serve --module your-function-path.your-function-name
  • The function accepts these arguments, which Blaxel passes when a request is made to your deployed agent:
    • request: The Fastify Request that triggers the agent’s execution.
    • args: (Optional) Argument object containing the core agent algorithm. By default, it uses a LangChain ReAct agent.

Mark your agent’s main function with Blaxel’s wrapAgent wrapper to serve it. You can override the default argument values that Blaxel passes upon requesting the agent by using the wrapper parameters (see below).

Some folders are optional but designed to handle specific parts of the agent code on Blaxel:

• src/functions: all files listed in this folder will be interpreted as custom tools that will all be made available to the agent automatically when serving or deploying the agent. The tool’s main TypeScript function itself is identified with the wrapFunction wrapper.

​

Quickstart

You can quickly initialize a new project from scratch by using CLI command bl create-agent-app. This will create a pre-scaffolded local repo where your entire code can be added. By default, this command creates a boilerplate agent that has:

• A LangChain ReAct agent as the core agent logic. This agent can be overridden by passing a custom agent as overrideAgent in the options of the wrapAgent in /src/agent.ts
• A hello-world tool (a function) that returns a placeholder text string. This function can be customized by changing the code in the helloworld function in /src/functions/helloworld.ts, and additional functions can be created in the /src/functions folder.

​

The wrapAgent higher-order function

The wrapAgent decorator identifies the main function that handles your core agent logic such as how to handle queries and responses, tool calls and such.

import { wrapAgent } from "@blaxel/sdk";

export const agent = wrapAgent(async (request, args) => { ... }, {
  agent: {
    metadata: { name: "example-agent" }
  }
});

Inside the wrapper, you can pass an object of options:

• agent: this can be either the Blaxel SDK’s Agent object or equivalent dict, and allows to specify all the parameters for the deployment of your agent on Blaxel. Examples include:
  • metadata.name: the name of the agent on Blaxel
  • spec.description: the description of the agent on Blaxel
  • spec.model: the name of a model API deployed in your Blaxel workspace, to use as core chat model for the agent
• overrideAgent: You can override the default LangChain ReAct agent by passing any custom agent instance. When the deployed agent receives a request, Blaxel will pass this as an argument to the agent’s handler function.
• overrideModel: You can query an LLM provider directly instead of routing through Blaxel by passing a BaseChatModel object from LangChain. When the deployed agent receives a request, Blaxel will pass this as the llm to the default LangChain ReAct agent. This is used as a more granular alternative to overrideAgent.
• remoteFunctions: You can pass on additional Blaxel functions that are already deployed so your agent can access them too, in addition to the functions from src/functions/. This parameter takes in an array of string representing the names of the functions on Blaxel. This is used as a more granular alternative to overrideAgent.

​

Managing secrets

You can add secrets to your code using $secrets.KEY_NAME. When you run the bl deploy command, Blaxel will attempt to retrieve the secret values and export them:

• if the key exists in your environment variables, Blaxel will use that value
• if not, you’ll be prompted to enter the secret value in the terminal

  const handleRequest = async (request: FastifyRequest, args: AgentType) => {
  ...
  }
  
  return wrapAgent(handleRequest, {
    agent: {
      metadata: {
        name: "test-thread2",
      },
      spec: {
        runtime: {
          envs: [
            {
              name: "OPENAI_API_KEY",
              value: "$secrets.API_KEY",
            },
          ],
        },
      },
    },
  };

​

Example

Here’s an example that demonstrates overriding the default agent object.

import { FastifyRequest } from 'fastify';
import { ChatOpenAI } from '@langchain/openai';
import { createReactAgent } from '@langchain/langgraph/prebuilt';
import { wrapAgent } from '@blaxel/sdk';
import helloworld from './customfunctions/helloworld';

// Initialize the chat model
const chat = new ChatOpenAI();

// Create a ReAct agent with custom tools
const customAgent = createReactAgent({
  llm: chat,
  tools: [helloworld]
});

// Create the main agent handler
const handler = async (request: FastifyRequest, args: { agent: any }) => {
  // Your main agent logic here
  // ...
};

// Export the wrapped agent
export const agent = wrapAgent(handler, {
  agent: {
    metadata: {
      name: "langchain-example",
    },
    spec: {
      model: "sandbox-openai",
    },
  },
  overrideAgent: customAgent,
}); 

When developing custom agents, you may also need to retrieve instantiated model APIs and functions in your code. The following section shows you how to.

​

Retrieve existing functions and model APIs

​

Get model APIs

You can use model APIs deployed on Blaxel by calling getChatModel(). This function returns a model API in LangChain format BaseChatModel

import { getChatModel } from '@blaxel/sdk';
import { createReactAgent } from "@langchain/langgraph/prebuilt";

const chat = await getChatModel("model-name-on-Blaxel");

const customAgent = createReactAgent({ llm: chat, tools: [] });

​

Get functions

You can use functions defined both in your code and deployed on Blaxel by calling getFunctions(). This function returns a list of tools in LangChain format StructuredTool.

import { getFunctions } from "@blaxel/sdk";
import { createReactAgent } from "@langchain/langgraph/prebuilt";

const functions = await getFunctions();

const customAgent = createReactAgent({ llm: chat, tools: functions });

By default, this returns the list of functions defined in the local folder src/functions/. You can pass several parameters:

• remoteFunctions: to retrieve additional Blaxel functions that are already deployed so your agent can access them too, in addition to the functions from src/functions/. This parameter takes in an array of string representing the names of the functions on Blaxel

​

Manage threads for sessions

To simulate multi-turn conversations, agent consumers can generate a thread ID and include it in the agent request using either the X-Blaxel-Thread-Id or Thread-Id header. You can access this thread by calling getDefaultThread() in your agent code.

import { getDefaultThread } from "@blaxel/sdk";

const handleRequest = async (request: FastifyRequest, args: AgentType) => {
  const { agent } = args;
  const body = (await request.body) as InputType;
  const input = body.inputs || body.input || "";
  
  const thread_id = getDefaultThread(request) || uuidv4();

  const stream = await agent.stream(
    { messages: [new HumanMessage(input)] },
    { configurable: { thread_id } }
  );
  
 ...
 
}