Serving

A serving deployment puts one model behind one OpenAI-compatible endpoint. Replicas (the GPU workers behind it) come and go — the endpoint URL never changes.

You control four things. GPUs are always picked for you: cheapest fit under your cap, re-assigned automatically.

Setting	Meaning
Price cap ($/replica-hour)	Hard ceiling — GPUs above it are never assigned, at deploy time or later
Min / max replicas	Allowed range (min ≥ 1 — scale-to-zero isn’t supported yet)
Autoscale	On: scale within the range by load · Off: hold at min
Output storage (image/video only)	Default bucket for generated files (details)

Deploying

Click Deploy on a model card. The panel shows:

• GPU candidates — every GPU type that fits, cheapest first, with its hourly price. Currently unavailable types are dimmed but listed.
• Suggested cap — pre-filled at ~1.5× the cheapest available candidate.

Set the four controls and confirm. The card appears as Creating; first deployments show weight-download progress (% and bytes) per replica, cache hits start without it.

Before you confirm:

You might see	It means	Do
Availability shows an error state	Couldn’t verify right now — not “unavailable”	Re-run the check
Host-RAM surcharge note	Some GPU types add a small RAM cost for image/video models	Shown in the panel when it applies
400 No GPU Available	Nothing matching the model’s VRAM is free right now	Retry later, or raise the cap
Conflict on deploy	The model already has an active deployment in this workspace	Edit the existing one

Status badges

Badge	Meaning	What to do
Creating	First replica being prepared	Wait — watch the download %
Running	≥ 1 replica serving	Use the endpoint
Updating	Replica count changing	Nothing — the endpoint stays up
Stopping	Shutting down	—
Stopped	Terminated, or paused and fully wound down	Re-deploy to serve again
Error	Deployment failed, diagnostic badge attached	Troubleshooting

Two rules the badges follow

• If one replica is serving, the card stays Running. Scale-ups, replacements, and single-replica restarts appear as an inline banner — a live endpoint is never hidden behind a transition screen.
• Changes apply within ~30 seconds. Until then the card shows the real in-between state (e.g. Stopping), not the end state.

Pause and resume

Action	What happens	Endpoint	Billing
Pause	Replicas wind down (~30s); config, URL, and registration are kept	503 while paused	Stops — nothing is running
Resume	Replicas come back (cold start; may re-download weights)	Live once Running	Resumes
Delete	Deployment torn down permanently	Gone — re-deploying mints a new URL	Stops

Editing a deployment

Edit on the card changes the same four controls:

• Min / max / autoscale — applied within ~30 seconds.
• Price cap, raised — immediately widens the GPU pool (this fixes a price cap scale block).
• Price cap, lowered — over-cap replicas are replaced gradually, one at a time, with progress on the card. The endpoint never dips.
• Output storage — applies to subsequent requests.

The replica drawer

Click a deployment card to see its replicas. Each row:

• Status — pending / creating / running / terminating / failed. Freshly acquired capacity shows a single “preparing” phase until ready.
• GPU — N× MODEL · VRAM GB (e.g. 2× RTX 4090 · 24 GB), known once placed.
• Hourly price — one price system: always the listed Meshive rate, never above your cap, regardless of where capacity comes from.
• Download progress — only while creating, only when weights actually need downloading.
• Step timeline — each provisioning step with status and timestamps. A failed step carries the diagnostic: exit code, one-line cause, and an OOM classification (VRAM vs RAM — see Troubleshooting).

Note

Failure evidence survives cleanup: when a failed replica is rolled back, its diagnostic is carried at the deployment level for as long as the status is Error.

Autoscaling

Load is evaluated every 30 seconds, always within your min/max range. Autoscale off = hold at min. Full rules, with examples — Autoscaling:

Model type	Scales up when	Scales down when
LLM / embedding	Sustained GPU-memory load above target (~60s), or a queue burst (≤30s)	Sustained low load (~2.5 min) and the remaining replicas can safely absorb it
Image	Several concurrent jobs per replica (~4)	Zero in-flight
Video	Fewer concurrent jobs per replica (~2) — jobs run minutes	Zero in-flight

When scale-up is blocked

The card shows the blocked target (“Can’t scale to N”) and one of two causes:

Banner	Cause	Whose move
Price cap	GPUs exist, but none at or under your cap	Yours — Raise cap opens the Edit drawer
Capacity	No matching GPUs available right now	Ours — retried automatically, nothing to do

The banner clears on its own once the condition resolves.

Live metrics

Deployment cards show four metric cells matched to the model type, plus GPU utilization:

Model type	Cell 1	Cell 2	Cell 3	Cell 4
LLM	TTFT p95	Total tok/s	Concurrent	Error rate
Embedding	Latency p95	Queries/s	Concurrent	Error rate
Image	Latency p95	Images/s	In-flight	Error rate
Video	Frames/s	Queue	In-flight	Error rate

Per-replica cells in the drawer: LLM/embedding → TTFT · tok/s · concurrency · KV-cache; image/video → concurrency · GPU · reachability.

Reading the numbers

• — means “not measured right now” — it is not zero.
• Metrics are ~30-second display snapshots. Billing comes from replica runtime, not from these.

Billing

• Billing for a replica starts when its model weights begin downloading — image pull, extraction, and warmup before that are free. If the weights are already cached on the node, billing starts only when the replica is running.
• The card and replica drawer show exactly when billing is active: the $/hr figure covers running replicas plus any replica already in its billed download phase.
• GPU-hours accrue per replica and roll up under Serverless in workspace billing.
• No per-request or per-token charge on top.

Troubleshooting

Symptom	Cause	Fix
OOM badge (VRAM)	The model or job exceeded GPU memory	Smaller size/steps (diffusion), a more quantized model, or raise the cap for a bigger GPU
OOM badge (RAM)	Host memory exhausted	Contact support — platform side
”Model retiring” warning	The official model behind this deployment is scheduled for removal	Stopping it means no redeploy — keep it running, or line up a replacement
Scale-up blocked (price cap)	No GPU under your cap	Raise cap (Edit drawer)
Scale-up blocked (capacity)	Temporary GPU shortage	Wait — retried automatically
Creating/Starting for a long time, % moving	Large model’s first download	Wait — progress is real
Endpoint returns 503	Paused, or no replica ready yet	Resume / wait for Running
Endpoint returns 404	Deleted, or wrong endpoint URL	Check the URL on the Serving page