Shared serving
Shared serving connects two sides of the marketplace:
- Guests call models other teams host — no GPU to rent, nothing to deploy, billed per token.
- Owners open their deployment’s spare throughput and are credited for every token guests spend — a discount on the deployment they’re already paying for.
Using shared models (guest)
Section titled “Using shared models (guest)”Serverless › Shared lists every model currently open for guest traffic. Each entry shows the per-token price and a live availability state. Pick one to see exactly how to call it — View code gives a ready-to-paste snippet, Try in Playground works without any setup.
- Your existing workspace API key works — nothing new to create.
- Billing is per token, charged to workspace credit at the listed input/output rates per 1M tokens. This is the one place Serverless bills per token rather than per GPU-hour (how deployments are billed).
- Call it through the shared endpoint with the model id shown on the page:
curl https://inference.meshive.ai/v1/chat/completions \ -H "Authorization: Bearer $MESHIVE_API_KEY" \ -d '{"model": "shared/<model-id>", "messages": [{"role": "user", "content": "Hi"}]}'GET /v1/models on the same endpoint lists everything currently available — handy for SDKs.
What guests should expect
Section titled “What guests should expect”Shared capacity is spare capacity — the catalog reflects what owners have open right now:
| You might see | It means | Do |
|---|---|---|
| Available / Degraded / Unavailable state | Live pool health for that model | Degraded = slower but succeeding; retry a timeout |
429 — Shared pool is busy right now | Guest demand exceeds idle capacity — temporary, not an outage | Retry shortly (the Playground retries automatically) |
402 — You’re out of credit | Shared usage draws on workspace credit | Add credit |
503 — This model just went offline | Every provider left the pool for that model | Notify me on the model row — your key and code stay valid when it returns |
Guest requests are isolated: per-request engine logs are off, so not even the deployment’s owner sees your request contents.
Sharing your deployment (owner)
Section titled “Sharing your deployment (owner)”Eligible deployments show a Sharing section with a Share idle capacity toggle — in the deploy panel and in Edit deployment. Currently eligible: deployments of official text (LLM) models on vLLM; other deployments don’t show the section.
Turning it on opens a one-time consent covering the ground rules:
- Your requests always come first — guests only use idle capacity.
- Guest traffic never scales your deployment — autoscaling follows your own load only.
- 100% of what guests pay is credited to your balance.
- Guests run on this deployment’s hardware, and per-request engine logs are turned off — even you won’t see guests’ request details.
Enabling performs a rolling restart of your replicas to activate priority scheduling. With a single replica, the endpoint is briefly unavailable (up to a few minutes while the model reloads); sharing starts automatically once the restart completes — the card shows Sharing: Activating…, then a Shared badge.
Earnings appear on the deployment card as Earned this month, credited to your balance. Turn sharing off anytime in Edit deployment — the deployment simply stops serving guest traffic.