Chat completions

POST /v1/chat/completions

OpenAI-compatible chat completion against your LLM deployment. Supports streaming (SSE) and multimodal messages content (e.g. image_url parts) when the deployed model is a vision-language model.

Request body

Field	Type	Default	Notes
model	string	(the deployment’s model)	Optional. If set, must match the endpoint’s model id
messages	array	required	OpenAI message objects; content parts (vision) pass through
stream	bool	false	SSE token stream
temperature	float	engine default
max_tokens	int	engine default
top_p	float	engine default
frequency_penalty / presence_penalty	float	engine default
stop	string | string[]	—
n	int	1

Any other OpenAI field (tools, tool_choice, response_format, seed, logprobs, user, …) is passed through to the inference engine rather than rejected — support depends on the deployed model/engine.

Example

curl

curl https://<your-deployment>.inference.meshive.ai/v1/chat/completions \
  -H "Authorization: Bearer $MESHIVE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "messages": [
      {"role": "system", "content": "You are a concise assistant."},
      {"role": "user", "content": "Explain KV cache in one paragraph."}
    ],
    "stream": true
  }'

Python

stream = client.chat.completions.create(
    model="",  # endpoint serves exactly one model
    messages=[
        {"role": "system", "content": "You are a concise assistant."},
        {"role": "user", "content": "Explain KV cache in one paragraph."},
    ],
    stream=True,
)
for chunk in stream:
    print(chunk.choices[0].delta.content or "", end="")

Node.js

const stream = await client.chat.completions.create({
  messages: [
    { role: "system", content: "You are a concise assistant." },
    { role: "user", content: "Explain KV cache in one paragraph." },
  ],
  stream: true,
});
for await (const chunk of stream) {
  process.stdout.write(chunk.choices[0]?.delta?.content ?? "");
}

The response (and stream chunk) schema is OpenAI’s — choices, usage, finish_reason, etc.

Behavior notes

• Rate limits: counts toward the key’s RPM, and usage tokens count toward its TPD. Success responses carry X-RateLimit-Remaining / X-RateLimit-Reset; over-limit returns 429 + Retry-After (details).
• Timeout: a single request times out after 120s at the gateway.
• Mid-stream failures emit a final data: {"error": ...} event before the stream closes — handle it when consuming SSE manually.
• Paused / no capacity: 503 (paused deployments say so; credit exhaustion returns insufficient_quota).

Errors

Status · code	Meaning
400	model set but doesn’t match this endpoint’s model
401 / 402	Bad key / credit exhausted
404	Not a deployment endpoint, or model not found
429 rate_limit_error	RPM or TPD exceeded — wait Retry-After
503	Deployment paused or no replica available