Building AI agents on Cloudflare

This blog post explores building AI agents on Cloudflare. We will delve into leveraging Cloudflare’s platform, including Workers, Workers AI, and the AI Agent SDK, to easily create AI applications. We will also cover the resources available to get started, such as starter packs, server/tools, and React SDKs. We will build a chatbot agent that can run custom tools for specific prompts.

What is Cloudflare?

Cloudflare is a leading global cloud services provider that delivers a broad range of web infrastructure and website security services.

Cloudflare offerings

• AI Gateway: Observe and control your AI applications with caching, rate limiting, request retries, model fallback, and more.

• Vectorize: Build full-stack AI applications with Vectorize, Cloudflare’s vector database. Adding Vectorize enables you to perform tasks such as semantic search, recommendations, anomaly detection or can be used to provide context and memory to an LLM.

• Workers: Build serverless applications and deploy instantly across the globe for exceptional performance, reliability, and scale.

• Workers AI: Run machine learning models, powered by serverless GPUs, on Cloudflare’s global network.

• Pages: Create full-stack applications that are instantly deployed to the Cloudflare global network.

• R2: Store large amounts of unstructured data without the costly egress bandwidth fees associated with typical cloud storage services.

• D1: Create new serverless SQL databases to query from your Workers and Pages projects.

• Durable Objects: A globally distributed coordination API with strongly consistent storage.

• KV: Create a global, low-latency, key-value data storage.

What are Cloudflare Workers?

Cloudflare Workers is a serverless computing platform that allows developers to build and deploy applications at the edge of the network, closer to users. This technology represents a shift from traditional server-based architectures to a more distributed computing model.

Cloudflare Workers run on Cloudflare’s global network spanning 275+ cities worldwide. Instead of executing code in a centralized data center, Workers execute at the “edge” - the point closest to your users. This dramatically reduces latency and improves performance.

Comparison to Other Serverless Platforms

Compared to AWS Lambda, Azure Functions, or Google Cloud Functions:

• ● Faster cold starts: Milliseconds vs. seconds

• ● Global by default: No need to deploy to multiple regions

• ● Simpler pricing: Based on requests rather than execution time

• ● Edge-first: Designed for the edge rather than adapted for it

• ● More limited runtime: Less flexibility but better performance

Cloudflare Workers represent a powerful shift in how web applications are built and deployed, bringing computation closer to users and enabling new patterns for performance, security, and scalability.

What are Cloudflare Workers AI?

Workers AI allows you to run AI models in a serverless way, without having to worry about scaling, maintaining, or paying for unused infrastructure. You can invoke models running on GPUs on Cloudflare’s network from your own code, from Workers, Pages, or anywhere via the Cloudflare API.

Workers AI gives you access to:

• ● 50+ open-source models

• ● Serverless, pay-for-what-you-use pricing model

What are Agents?

Agents are Autonomous AI systems that dynamically select tools and adapt processes based on real-time context. Characterized by:

• ● Non-linear decision making

• ● Non-deterministic outcomes

• ● Full task automation

• ● Adaptive tool selection

The difference between agents and co-pilots is that agents do not require human intervention.

Cloudflare agents-sdk

Cloudflare Agents SDK, is a javascript/typescript SDK, which enables you to build and deploy AI-powered agents that can autonomously perform tasks, communicate with clients in real time, call AI models, persist state, schedule tasks, run asynchronous workflows, browse the web, query data from your database, support human-in-the-loop interactions and more.

Benefits of building AI agents on Cloudflare;

• ● State management included - ability to automatically sync state between an Agent and clients, trigger events on state changes, and read+write to each Agent’s SQL database.

• ● Communicative - agents can be connected via websockets and they can stream updates back to the client in real time

• ● Extensible - use any AI model that you want, read data from databases hosted in another cloud, add your own methods to the agent and call them

Cloudflare agents starter template

For the rest of this blog we will be focusing on the cloudflare agents starter template . This project provides a foundation for creating interactive chat experiences with AI and cloudflare agents sdk, it uses react and the agents-sdk react plugin for the UI and features tool integrations for agent tool usage.

To run it locally the only thing we need is an api key for openai/anthropic or cloudlflare in order to access a LLM.

Lets dive into the code for the starter template. Inside src/server.ts we can see the creation of the agent.

This is extending the AIChatAgent class that is provided by the agents sdk.