Modal

Modal is a serverless platform where you write Python, attach a GPU with one argument, and pay per second while it runs. On 31 October 2026 an H100 cost $0.001097 a second, about $3.95 an hour, and the free Starter plan includes $30 of credits a month. It suits bursty inference and batch jobs. It is Python-first, which is its main constraint.

Some links on this page go through /go/ and may earn us a fee; it never changes our order.

Facts at a glance

The interactive hosting compare lets you line Modal up against GPU clouds and per-token APIs.

How Modal works

You decorate a Python function and tell Modal what it needs:

import modal

app = modal.App("example")

@app.function(gpu="H100", timeout=600)
def generate(prompt: str) -> str:
    ...

Modal builds a container image from your definition, boots a container when a call arrives, runs the function, and shuts the container down after an idle period. You are billed for the seconds the container is up. The docs say the default maximum idle time is 60 seconds, which you can configure between two seconds and twenty minutes. Functions can also serve web endpoints, run on a schedule, or process a queue of inputs.

The tradeoff against renting a pod is control. You do not log in to a machine. You define the environment in code, and Modal runs it. For some teams that is the appeal, since there is no server to patch. For others it is the barrier, since the platform is Python-centred.

The prices

Modal's pricing page lists GPU rates per second. We converted to hours by multiplying by 3,600. Checked on 31 October 2026.

CPU is $0.0000131 per core per second and memory is $0.00000222 per GiB per second on the standard rows. Volumes are $0.09 per GiB a month, and the page lists network egress at $0.04 per GiB. Compute is billed on top of the plan.

Modal also lists credit grants: early-stage startups can apply for free compute credits, and graduate students, labs and researchers can get up to $10,000. Usage on Starter and Team is pay-as-you-go and does not need prepaid credits.

What you pay in practice

Volumes here are examples we chose.

Example one: image generation. Modal's own pricing page works an example: a Stable Diffusion run on an A10 taking about 1.5 seconds per image, with 4 GiB of memory and 1 CPU, costs about $0.000491 per image in total, or $0.491 per 1,000 images. That is useful as a template for your own per-request cost.

Example two: a nightly summarisation batch. 20,000 documents, 3 hours of H100 time. 10,800 seconds at $0.001097 is $11.85, and the GPU is off afterwards. With the Starter plan's $30 credit, this batch is free for about two and a half runs a month.

Example three: an inference endpoint on an A100 80 GB. If the endpoint is busy 60 hours in a month, the GPU cost is 60 × $2.50, about $150. Compare that with a Runpod Secure A100 on a pod at $1.79 an hour: left on for 720 hours it is about $1,289. At 60 busy hours, Modal's serverless billing is cheaper than an always-on pod. The break-even is near 515 hours of use a month at those two rates.

Example four: a team's coding model. An always-busy 80 GB GPU is a bad fit for pay-per-second, since the rate per hour is higher than a pod's. Keep it on a Runpod pod or a Lambda Cloud instance.

Modal's page also shows a scale example: 75 GPUs for 24 hours at a flat $3 per GPU-hour is $5,400, while an average of 50 GPUs for 24 hours at $3.95 is $4,740. The point is that elastic use can beat a fixed reservation even at a higher hourly rate.

Limits

Modal's docs say it may automatically upgrade a GPU request, for example H100 to H200, to give you more capacity, so pin the card if reproducibility matters.

Cold starts

Cold starts are the cost of scale-to-zero. The cold start guide names two sources of delay: getting the container up and loading your model. It says the whole thing can range from seconds to minutes. The levers it describes:

• Keep containers warm by setting a longer idle timeout or a minimum number of containers, at the cost of paying for idle time.
• Memory snapshots capture the container's memory state after initialisation, which the docs say can cut boot times from minutes to seconds.
• Store weights on a Volume instead of downloading at every start.

The trade-off, in the docs' own framing, is cold-start latency against cost. For an interactive chat endpoint you may want a warm container. For a batch job, nobody cares about a 90-second start.

Regions and data

Modal lets you pin container regions per function, such as a US West region, and separately choose a routing region for inputs and outputs. The docs advise broader regions where you can, since they enlarge the pool of capacity and improve cold starts, and note that pinning to specific regions can apply price multipliers.

On data, the security page says:

• Inference endpoints have zero data retention: request and response payloads are never written to disk and pass through Modal only as in-flight traffic.
• Other data types have retention periods set out in a table on the page, and all stored data is encrypted at rest. Data in transit uses TLS 1.3.
• Modal lists SOC 2 Type II, GDPR with a Data Processing Addendum, and HIPAA.
• Each container runs in its own sandbox. Features include single sign-on, role-based access, audit logs and customer-supplied encryption keys.

The page also describes a shared responsibility model, where you decide which endpoints to expose and how they authenticate. Read it with your counsel for regulated data.

What else Modal offers

The docs navigation shows more than GPU functions, and some of it matters for agent builders.

Sandboxes. Isolated environments to run untrusted or generated code, with docs on running commands, networking and security, file access and snapshots. This is relevant if your agent writes and runs code, since the sandbox gives each run its own boundary. The security page says Modal provides isolation primitives for running untrusted code, and lists sandbox network access controls.

Endpoints. A section on supported models, shared endpoints and dedicated endpoints suggests a managed inference route alongside bring-your-own-code functions. We have not tested it, and the pages we read do not give prices for it.

Training. A quickstart for training, a Tinker-compatible API in beta, and multi-node training. For fine-tuning runs, per-second billing suits short jobs.

Scheduled and web functions. Cron-style schedules and HTTP endpoints, so a function can be an API as well as a batch step.

Volumes and cloud bucket mounts. Persistent storage for weights and datasets, with an S3-style mount option.

Serving an open model on Modal

The common pattern for a language model is to run vLLM inside a Modal function. You define an image with vLLM installed, put weights on a Volume so they do not download each time, attach a GPU such as an H100 or an A100 80 GB, and expose an OpenAI-compatible web endpoint. The function scales from zero when requests arrive and back to zero after the idle timeout. Cold starts are the cost: the model has to load into GPU memory. Memory snapshots and a warm container reduce that.

A sensible first sizing exercise: take the listed size of your model, add about a quarter for cache, and pick the smallest card that fits. gpt-oss 120B is listed at 65 GB and fits an 80 GB card, so an A100 80 GB at about $2.50 an hour is the capacity fit, and an H100 at about $3.95 is the speed fit. For a 20B model at 14 GB, an L4 at about $0.80 an hour or an A10 at about $1.10 may do, depending on the context you need. See GPU requirements for LLMs and vLLM.

If the endpoint sees steady traffic around the clock, work out whether the always-on cost beats a pod. At about $2.50 an hour, an A100 80 GB on Modal costs $1,800 for 720 hours against about $1,289 on a Runpod Secure pod. If it is busy only a few hours a day, Modal wins.

Billing details to know

• Per second, per resource. GPU, CPU and memory each bill per second. A function that asks for a lot of memory pays for it while the container is up.
• Idle time bills. A container that stays warm for 60 seconds after the last request bills for those seconds. Setting a long idle timeout trades money for latency.
• Credits first. Plan credits apply to compute before you pay. The pages we read do not say what happens to unused credits.
• Egress and storage. The page lists $0.04 per GiB of network egress and $0.09 per GiB a month for volumes, which matter if you move large models or datasets.
• Concurrency. More containers means more spend per second. Cap it with the scaling settings if your input could spike.

Security responsibilities

Modal's security page describes a shared responsibility model, and it is worth reading before you put customer data through it. Modal covers the platform: encryption, container isolation, monitoring. You cover the application: which endpoints you expose, how they authenticate, whether sensitive fields are encrypted before they reach Modal, backups of data you store, and compliance with the laws that apply to you. A function exposed as a public web endpoint without authentication is exposed, whatever the platform does underneath.

Who should pick Modal

Alternatives

• Runpod: pods and serverless, lower hourly rates on Community Cloud.
• Lambda Cloud: plain on-demand GPU instances, 1x H100 SXM at $4.29.
• Replicate: run public models by the second without writing infrastructure.
• Hugging Face Inference: dedicated endpoints.
• AWS Bedrock: managed models for AWS users.
• The AI hosting overview maps the whole field.

Getting started

1. Install the client with pip install modal and run modal setup to authenticate.
2. Write a function with a GPU argument, and run it with modal run.
3. Put model weights on a Volume.
4. Deploy with modal deploy to get a web endpoint.
5. Watch the usage page against your $30 of credits.

Check the commands against Modal's current docs before you run them. For self-run model servers, see vLLM.

Our take

Modal is a good answer for work that comes in bursts and for teams that live in Python. Per-second pricing makes a 3-hour batch cost $11.85 and idle time cost nothing. The same pricing is a poor deal for a GPU that is busy all day. The security page is more detailed than most, with zero retention on inference endpoints and a SOC 2 Type II report, which helps privacy-minded buyers. Our conclusions rest on Modal's pricing page and docs. We did not run a performance test.

Related pages

• Hosting hub and AI hosting overview
• Modal vs Runpod
• Private AI hub