Most AI agent tutorials stop at a Python script running on a laptop. Production agents look different. They sit behind an HTTP endpoint, call tools, wait several seconds on a model API, and need to survive pod restarts and rolling updates like any other service.
This guide builds a small tool-calling agent in TypeScript with the Anthropic SDK, packages it in a multi-stage Docker image, and deploys it to Kubernetes with a Secret for the API key, resource limits, health probes, and a shutdown setup that doesn’t cut off slow model calls.
Prerequisites
- Node.js 22 or later and npm
- Docker
- A Kubernetes cluster (kind, minikube, or a managed cluster) and
kubectl - An Anthropic API key
The same pattern works with the OpenAI Node SDK. Only the request and response shapes change.
Step 1: Set Up the Project
mkdir ts-agent && cd ts-agent
npm init -y
npm pkg set type=module
npm install @anthropic-ai/sdk
npm install --save-dev typescript @types/node
Add a tsconfig.json:
{
"compilerOptions": {
"target": "ES2022",
"module": "NodeNext",
"moduleResolution": "NodeNext",
"rootDir": "src",
"outDir": "dist",
"strict": true,
"skipLibCheck": true
},
"include": ["src"]
}
Step 2: Write the Agent Loop
The agent gets one tool, check_url, which sends a GET request and reports the status code and response time. It’s a small tool, but it shows the full loop: the model asks for a tool call, your code runs it, and the result goes back to the model.
Create src/agent.ts:
import Anthropic from "@anthropic-ai/sdk";
const client = new Anthropic(); // reads ANTHROPIC_API_KEY from the environment
const MODEL = process.env.MODEL ?? "set-a-current-model-id";
const MAX_TURNS = 5;
const tools: Anthropic.Tool[] = [
{
name: "check_url",
description: "Send a GET request to a URL and return the HTTP status code and response time in ms.",
input_schema: {
type: "object",
properties: { url: { type: "string", description: "Full URL, including https://" } },
required: ["url"],
},
},
];
async function checkUrl(url: string) {
const start = Date.now();
try {
const res = await fetch(url, { signal: AbortSignal.timeout(5000) });
return { status: res.status, ms: Date.now() - start };
} catch (err) {
return { error: String(err), ms: Date.now() - start };
}
}
export async function runAgent(question: string): Promise<string> {
const messages: Anthropic.MessageParam[] = [{ role: "user", content: question }];
for (let turn = 0; turn < MAX_TURNS; turn++) {
const response = await client.messages.create({
model: MODEL,
max_tokens: 1024,
tools,
messages,
});
if (response.stop_reason !== "tool_use") {
return response.content
.filter((block): block is Anthropic.TextBlock => block.type === "text")
.map((block) => block.text)
.join("\n");
}
messages.push({ role: "assistant", content: response.content });
const results: Anthropic.ToolResultBlockParam[] = [];
for (const block of response.content) {
if (block.type === "tool_use" && block.name === "check_url") {
const { url } = block.input as { url: string };
results.push({
type: "tool_result",
tool_use_id: block.id,
content: JSON.stringify(await checkUrl(url)),
});
}
}
messages.push({ role: "user", content: results });
}
return "Stopped after reaching the tool-call limit.";
}
Three details matter here. MAX_TURNS keeps a confused model from looping forever and running up your API bill. AbortSignal.timeout(5000) stops one slow URL from hanging the whole request. And each tool_result must carry the matching tool_use_id, or the API rejects the next call.
The model ID comes from an environment variable, so you can switch models in Kubernetes without rebuilding the image. Pick a current ID from the provider’s model list.
Step 3: Expose It Over HTTP
Create src/server.ts with two routes: /ask for questions and /healthz for Kubernetes probes.
import http from "node:http";
import { runAgent } from "./agent.js";
const server = http.createServer(async (req, res) => {
if (req.url === "/healthz") {
res.writeHead(200).end("ok");
return;
}
if (req.method === "POST" && req.url === "/ask") {
let body = "";
for await (const chunk of req) body += chunk;
try {
const { question } = JSON.parse(body);
const answer = await runAgent(question);
res.writeHead(200, { "Content-Type": "application/json" });
res.end(JSON.stringify({ answer }));
} catch (err) {
res.writeHead(500).end(JSON.stringify({ error: String(err) }));
}
return;
}
res.writeHead(404).end();
});
server.listen(3000, () => console.log("agent listening on :3000"));
process.on("SIGTERM", () => {
console.log("SIGTERM received, finishing in-flight requests");
server.close(() => process.exit(0));
});
The SIGTERM handler is easy to skip and painful to miss. A single agent request can take 20 to 40 seconds when the model chains several tool calls. Without the handler, every rolling update kills those requests halfway through.
Step 4: Build a Small Container Image
FROM node:22-slim AS build
WORKDIR /app
COPY package*.json tsconfig.json ./
RUN npm ci
COPY src ./src
RUN npx tsc -p .
FROM node:22-slim
WORKDIR /app
ENV NODE_ENV=production
COPY package*.json ./
RUN npm ci --omit=dev
COPY --from=build /app/dist ./dist
USER node
EXPOSE 3000
CMD ["node", "dist/server.js"]
The first stage compiles TypeScript. The second ships only the compiled JavaScript and production dependencies, so the TypeScript compiler never reaches the runtime image. USER node runs the process without root, and the exec-form CMD makes Node PID 1 so it receives SIGTERM directly instead of through a shell.
docker build -t your-registry/ts-agent:0.1.0 .
docker push your-registry/ts-agent:0.1.0
Step 5: Deploy to Kubernetes
Store the API key in a Secret rather than in the manifest:
kubectl create secret generic agent-secrets \
--from-literal=ANTHROPIC_API_KEY=your-key-here
Then apply the Deployment and Service:
apiVersion: apps/v1
kind: Deployment
metadata:
name: ts-agent
spec:
replicas: 2
selector:
matchLabels:
app: ts-agent
template:
metadata:
labels:
app: ts-agent
spec:
terminationGracePeriodSeconds: 90
containers:
- name: agent
image: your-registry/ts-agent:0.1.0
ports:
- containerPort: 3000
env:
- name: MODEL
value: "your-model-id"
- name: ANTHROPIC_API_KEY
valueFrom:
secretKeyRef:
name: agent-secrets
key: ANTHROPIC_API_KEY
resources:
requests:
cpu: 100m
memory: 128Mi
limits:
cpu: 500m
memory: 256Mi
readinessProbe:
httpGet:
path: /healthz
port: 3000
periodSeconds: 5
livenessProbe:
httpGet:
path: /healthz
port: 3000
periodSeconds: 15
lifecycle:
preStop:
exec:
command: ["sleep", "5"]
---
apiVersion: v1
kind: Service
metadata:
name: ts-agent
spec:
selector:
app: ts-agent
ports:
- port: 80
targetPort: 3000
Test it with a port-forward:
kubectl port-forward svc/ts-agent 8080:80
curl -X POST localhost:8080/ask \
-d '{"question": "Is https://collabnix.com up, and how fast did it respond?"}'
Why These Settings
The resource numbers look small because the model runs on the provider’s servers. The pod spends most of its time waiting on network I/O, so 128Mi of memory and a tenth of a CPU core cover a modest load.
terminationGracePeriodSeconds: 90 gives slow agent calls time to finish after SIGTERM. The default is 30 seconds, which is shorter than a long multi-tool request. The preStop sleep gives the Service a few seconds to stop sending new traffic to the pod before shutdown begins.
Common Pitfalls
- Scaling on CPU. Because agent pods are I/O-bound, CPU stays low even when every pod is saturated with waiting requests. A HorizontalPodAutoscaler on CPU won’t react. Scale on in-flight requests or request latency instead, using KEDA or the Prometheus Adapter.
- Keys in the image. Never bake the API key into the Dockerfile or a ConfigMap. Anyone with access to the image or the namespace can read it.
- Unbounded tool input. The model chooses the URL for
check_url. In a real deployment, restrict tools to an allowlist of hosts and apply a NetworkPolicy, so a crafted prompt can’t make the pod probe internal services. - Rate limits. Two replicas share one API key and one rate limit. Catch 429 responses and retry with backoff instead of returning a 500 to the user.
Who Builds and Runs This in Practice
Look back at the code. The model is one API call. Everything else is TypeScript: JSON schemas for the tools, async control flow, timeouts, error handling, an HTTP server, and a Dockerfile. Once the prototype works, most of the remaining work is regular web and backend engineering.
That changes how teams staff agent projects. Many expect to need machine learning specialists, then find the real backlog is API integration, streaming responses to a React front end, and keeping services healthy in the cluster. Engineers who handle those tasks well usually know:
- async/await, streams, and cancellation in Node.js
- Schema validation with JSON Schema or a library like Zod
- Writing Dockerfiles and reading
kubectl describe podoutput when a probe fails
Teams without those skills in-house can train existing front-end developers or hire javascript developers who already work across Node.js and containers. Either way, it helps to see agent development as software engineering first.
Conclusion
You now have a TypeScript agent with a capped tool-calling loop, a non-root multi-stage image, and a Kubernetes Deployment that keeps the API key in a Secret, sets resource limits, and shuts down without dropping slow requests. From here, add more tools, stream tokens to the client with the SDK’s streaming API, and wire up request-based autoscaling before real traffic arrives.