Tutorial: Deploy a custom Hugging Face model

This tutorial deploys a model that is not in the official catalog — any Hugging Face repo — and ends with a working OpenAI-compatible endpoint. We’ll use an LLM as the running example; image/video repos follow the same flow with one extra field each (noted inline).

You need: a workspace with credits, and a Hugging Face repo id (e.g. Qwen/Qwen2.5-7B-Instruct).

1. Register the repo.

   Serverless › Models → Register custom model. Paste the repo id and submit — that’s the entire required input. Meshive reads the repo’s metadata and fills in modality, framework, file size, parameter count, quantization, and a VRAM estimate.

   Worth overriding only if you know better (every derived value is editable):

   • Image models must list Supported sizes (e.g. 1024x1024) and video models Supported resolutions — these become the validation lists for the API and Playground. If the repo’s metadata couldn’t answer something required, registration tells you exactly which field to fill instead of guessing.
   • Video models: requests will run at whatever duration/fps callers send — check the model card for the native spec, and put it in your client defaults (why this matters).

   Gated repo?

   Paste a Hugging Face token in the form (one-off), or save tokens once under the workspace’s HF token list and pick by label. Tokens are only used for metadata reads and weight downloads.

2. Deploy it.

   Click Deploy on your new model. The panel lists every GPU type that can hold the model, cheapest first — pick nothing; just set the knobs:

   • Price cap: keep the suggested value (≈1.5× the cheapest fitting GPU) unless you have a budget number in mind.
   • Min/max replicas: 1 / 3 is a sensible start; autoscale on.
   • (Image/video models) Output storage: Managed is fine to start — switch to your own bucket when you need permanence.

   Common rejections at this step, and what they mean:

   Response	Meaning
   No GPU Available	Nothing matching the model’s VRAM is free right now — retry later or raise the cap
   Conflict (already deployed)	This model already has an active deployment in the workspace — edit that one instead

3. Watch it come up.

   The card starts in Creating. A first-time deployment downloads the weights — the replica row shows live progress (% and bytes; a large model takes a while, and the progress bar is the proof things are moving). Cached weights skip the download entirely.

   When the badge is Running, copy the endpoint URL from the card:

   https://<your-deployment>.inference.meshive.ai/v1

4. Call it.

   Create a key under Models → API Keys if you haven’t (guide), then:

   Python

   from openai import OpenAI
   
   client = OpenAI(
       base_url="https://<your-deployment>.inference.meshive.ai/v1",
       api_key="mk-...",
   )
   
   resp = client.chat.completions.create(
       model="",  # the endpoint knows its model
       messages=[{"role": "user", "content": "Summarize what you are in one line."}],
   )
   print(resp.choices[0].message.content)

   curl

   curl https://<your-deployment>.inference.meshive.ai/v1/chat/completions \
     -H "Authorization: Bearer $MESHIVE_API_KEY" \
     -H "Content-Type: application/json" \
     -d '{"messages":[{"role":"user","content":"Summarize what you are in one line."}]}'

   For an image model, the first call is POST /v1/images/generations with one of the sizes you registered; for video, POST /v1/videos (reference). Or skip code entirely — the Playground runs the same requests through the same gateway.

If something fails

Symptom	What it means	Fix
Replica fails with an OOM (VRAM) badge	The model needs more GPU memory than the assigned GPU had — usually an underestimated VRAM requirement	Raise Min VRAM on the registration, or raise the price cap so bigger GPUs qualify
OOM (RAM) badge	Host memory, not GPU — platform-side	Contact support
Creating seems stuck, download % moving	Big model, first download	Wait it out — the % is real
Stuck without download progress, then Error	Check the replica’s step timeline — the failing step carries the diagnostic	The timeline names the step and reason; the troubleshooting table covers the rest
Registration rejected	Unsupported/ambiguous modality, missing required field, engine incompatibility, or missing VRAM info (diffusion)	The error names the exact problem — see registration errors

Cleaning up

Serving bills GPU-hours per running replica — when you’re done experimenting, Pause (keeps URL & config, stops the meter within ~30s) or Delete the deployment. A registration with no active deployment can then be deleted from the Models page.