Skip to content
Multigrid

AI at the edge: Workers, Vercel and Netlify

What an edge runtime can and cannot do with a model call, and where the limits — CPU time, body size, function duration — actually bite.

An edge runtime is not a small server. It is a different execution model with a different clock, and most of the trouble people have running model calls on one comes from carrying server assumptions across. The thing being metered is usually not the thing you think: Cloudflare bills Workers on CPU time and not on how long you waited for a provider, so a forty-second completion can fit inside a ten-millisecond budget, while a base64 decode of the image that completion produced can blow it. Vercel and Netlify meter differently again, and each has a body size and a function duration that will stop a streaming proxy before any model does.

These pages work outward from those constraints to the code. Where a limit is a real number Cloudflare, Vercel or Netlify publishes, it is quoted with its publisher and marked as something that moves; where it is not published, the page says which console pane or quota name to read rather than guessing. The code is written against bindings, CLI flags and manifest fields, because those survive the redesigns that screenshots do not.

Running Text Generation on Cloudflare Workers AI

Wire the AI binding into a Worker, make one text-generation call, and read every field of the object it hands back.

9 min read

Generating Embeddings With Cloudflare Workers AI

Turn text into vectors with a Workers AI embedding model and write them into a Vectorize index in the same request.

9 min read

Streaming Responses From Cloudflare Workers AI

Enable streaming on the AI binding and pass the events through to the client without buffering them in the Worker.

9 min read

Cloudflare Workers AI Pricing and Neuron Limits

What a neuron is, what Cloudflare charges for one, and how to turn a model's published neuron rate into a cost per request.

9 min read

Generating Images on Cloudflare Workers AI

Call an image model from a Worker and return the result as a real image response rather than a base64 blob.

8 min read

Rate Limits on Cloudflare Workers AI

The two different limits Workers AI enforces, the two different 429s they produce, and what to do about each.

9 min read

Using a Durable Object to Hold Agent State

Build a Durable Object that keeps an agent's conversation history across requests, using its SQL storage rather than a key-value blob.

10 min read

WebSocket Coordination With Durable Objects for a Streaming Chat

Accept a WebSocket into a Durable Object with the hibernation API so an idle connection stops accruing duration charges.

10 min read

Rate Limiting Requests Per User With a Durable Object

Use one Durable Object per user as a strongly-consistent counter, and know when the built-in rate-limiting binding is the better answer.

10 min read

Creating a Vectorize Index on Cloudflare

Create an index with the dimension and distance metric your embedding model requires, both of which are fixed at creation.

9 min read

Querying Cloudflare Vectorize From a Worker

Run a similarity query through the Vectorize binding and read the matches, their scores and the ceilings on topK.

9 min read

Metadata Filtering in Cloudflare Vectorize

Attach metadata at insert time, create the metadata index that makes it filterable, and filter a query by it in the same call.

9 min read

Cloudflare Vectorize Pricing and Dimension Limits

How Vectorize bills on queried and stored vector dimensions, worked through to a monthly figure, and the dimension ceiling that constrains model choice.

9 min read

Setting Up Cloudflare AI Gateway in Front of a Model Provider

Point an existing provider call at a gateway URL, get logging without changing your code, and avoid the header mistake that returns 401.

9 min read

Caching Responses With Cloudflare AI Gateway

Set a per-request cache TTL on a model call, confirm the second call is served from cache, and work out in advance whether your traffic can hit at all.

9 min read

Rate Limiting a Model Provider With Cloudflare AI Gateway

Configure a gateway-side request limit so one caller cannot exhaust a shared provider key, and understand what the gateway limit does and does not isolate.

9 min read

Reading Cost Analytics in Cloudflare AI Gateway

What the gateway's cost and token charts are actually computing, when the cost number is missing or wrong, and how to query the same data with GraphQL.

9 min read

Configuring a Fallback Provider in Cloudflare AI Gateway

Chain a second provider behind the first, force a failure to prove the chain works, and understand which failures a gateway fallback does not catch.

10 min read

Binding a D1 Database to a Worker for AI Request Logging

Write each model request's latency, token counts and outcome to a D1 table from the Worker that made the call, without adding latency to the response.

10 min read

Storing an API Key as a Cloudflare Worker Secret

Set a provider key with wrangler secret put, read it from env at runtime, and run local development without the key ever entering your repository.

9 min read

CPU Time Limits on Cloudflare Workers for AI Calls

Why a Worker waiting three minutes for a model response is not close to any limit, and which parts of an AI Worker actually consume CPU time.

9 min read

Cloudflare Queues for Background Model Processing

Produce a job from a request-handling Worker and consume it in a batch handler, with the retry, delay and dead-letter behaviour that model calls need.

10 min read

What Workers AI Is Good For Inside a Cloudflare Stack

An argument that Workers AI earns its place through colocation and billing shape rather than model quality, and what that implies about where to use it.

10 min read

Smart Placement on Cloudflare Workers for Backend-Heavy AI Calls

Enable Smart Placement on a Worker that mostly waits on a model API, and work out in advance whether your request shape can benefit at all.

9 min read

Hyperdrive for Connecting Cloudflare Workers to a Vector Database

Pool a Postgres-backed vector database connection through Hyperdrive so a Worker can run a pgvector similarity search without paying connection setup on every request.

10 min read

Edge Function Timeout Limits on Vercel

The Edge runtime's 25-second rule for the first byte and 300-second streaming ceiling, dated, and exactly where a long model call runs into them.

9 min read

Streaming a Model Response From a Vercel Edge Function

Build a streamed response from an edge function, read it incrementally in the browser, and confirm the chunks are arriving one at a time rather than all at once.

10 min read

Vercel Fluid Compute for AI Workloads

How Active CPU billing and in-function concurrency change the cost of a function that spends most of its time waiting on a model API, and what that changes in your code.

10 min read

Function Duration Limits on Vercel by Plan

The documented default, maximum and extended maximum durations for Vercel Functions on each plan, and what the clock is actually measuring.

9 min read

Storing a Model Provider API Key as a Vercel Environment Variable

Adds a provider key as a sensitive, environment-scoped variable with the Vercel CLI, reads it from a function, and pulls it for local development.

9 min read

Fixing a Netlify Function Timeout on a Slow Model Call

Works from the 502 and the Task timed out log line to whether you are actually hitting Netlify's ceiling or an upstream call with no deadline of its own.

9 min read

Background Functions on Netlify for Long AI Jobs

Moves a slow model job into a 15-minute background function that returns 202 immediately, and covers the part that follows: how the caller ever finds out it finished.

10 min read

Edge Functions on Netlify for Streaming a Model Response

Builds a Netlify Edge Function that forwards a provider's token stream to the browser, and proves the client receives it incrementally rather than at the end.

10 min read

Netlify Function Timeout and Payload Limits

Every documented Netlify function execution and payload limit in one table, with the correction that the timeouts are flat across plans and what actually is plan-gated.

9 min read

Setting Up Netlify Environment Variables for a Model API Key

Sets a provider key scoped to the production deploy context and the functions scope, so a preview build cannot read it or spend against it.

9 min read

Body Size Limits on Vercel Functions for Large AI Payloads

The 4.5 MB request and response cap on Vercel Functions, the two error codes it produces, and the arithmetic that puts a much smaller real document over it.

9 min read

Fixing FUNCTION_INVOCATION_TIMEOUT on Vercel

Works from the 504 and the exact error code to the specific cause, separating a genuine plan ceiling from the far more common hung upstream call.

9 min read

Deploying a Python Function on Vercel for a Model Call

Deploys a Python function that calls a model on Vercel, covering the entrypoint rules, the version pin, and the bundling step that decides whether it deploys at all.

10 min read

Cron Jobs on Vercel to Trigger a Scheduled Model Task

Configures a cron schedule in vercel.json that invokes a function running a scheduled model task, secures it, and makes it survive duplicate and missed runs.

10 min read

Netlify Scheduled Functions for a Recurring Model Job

Configures a cron-triggered Netlify function for a recurring model job, and works around the 30-second limit that makes a scheduled function a dispatcher rather than a worker.

9 min read

Why an Edge Runtime Can't Use a Node SDK for a Model Provider

Starts from the module-resolution and Node-API errors an edge build emits, explains what an isolate genuinely cannot provide, and locates the dependency that is actually at fault.

10 min read

Fixing a Streaming Response Buffered Instead of Chunked on Vercel

Locates which layer is collecting the whole body — the function type, the response headers, an intermediary, or the client — instead of guessing at headers.

10 min read

Other topics