Manage model masters

List, get, add, update, and archive model masters through the CosmicAC API or the Models page.

Model masters hold the default serving parameters for each model. For what a model master is, see Model masters.

This guide uses curl to list, get, add, update, and archive them through the cosmicac-app-node API. You can also update a model master from the Models page in the web interface.

Editing a model master doesn't change a job that already serves the model. CosmicAC applies the new values to the next job someone creates for it.

Prerequisites

You need the following before you start:

A running CosmicAC deployment. See Installation.
Your CosmicAC base URL, the address where your CosmicAC UI is reachable. Ask your administrator if you need it. CosmicAC serves the API under /api, so each request uses <base-url>/api.
A terminal with curl.

List model masters

List the active model masters. To list archived ones instead, set status=archived:

curl "<base-url>/api/v1/model-masters?status=active&limit=20&offset=0"

Each entry includes an id. Use that id to get, update, or archive the model master in the requests that follow.

Get a model master

To get a model master, send a GET request with its id:

curl <base-url>/api/v1/model-masters/<id>

Add a model master

To add a model master, send a POST request with its serving parameters, in the same format as Set up model masters. This example adds MiniMax M2.5:

curl -X POST <base-url>/api/v1/model-masters \
  -H "Content-Type: application/json" \
  -d '{
    "job_type": "INFERENCE_VLLM",
    "base_os_image": "Ubuntu 22.04 + CUDA 13.0",
    "disk_gb": 500,
    "cuda_driver_version": "CUDA 13.0",
    "model_name": "MiniMaxAI/MiniMax-M2.5",
    "runtime_image": "vllm/vllm-openai:v0.15.1",
    "data_type": "Auto",
    "quantisation": null,
    "tensor_parallel": 4,
    "per_replica_gpu_count": 4,
    "gpu_memory_utilisation": 0.85,
    "max_model_length": 27000,
    "max_concurrent_sequences": 256,
    "reasoning_parser": "deepseek_r1",
    "multimodal": true,
    "replica": 1,
    "require_auth_header": true,
    "min": ["base_os_image"],
    "max": ["gpu_memory_utilisation"],
    "inference_param_overrides": {
      "root_disk_size_gb": 500,
      "env": [
        { "name": "TRUST_REMOTE_CODE", "value": "true" },
        { "name": "SWAP_SPACE", "value": "0" },
        { "name": "ENABLE_EXPERT_PARALLEL", "value": "true" },
        { "name": "ENFORCE_EAGER", "value": "false" }
      ]
    }
  }'

For the recommended values, see Recommended model parameters.

Runtime image format

Set runtime_image to a Docker image reference, such as vllm/vllm-openai:v0.15.1. CosmicAC serves the model on the image you name here.

The earlier label format, such as vLLM 0.15.0 + CUDA 12.9, no longer works. If a model master still uses that format, CosmicAC serves the model on a fallback image from your deployment rather than the version you set.

Update a model master

To update a model master, send a PATCH request with only the fields to change:

curl -X PATCH <base-url>/api/v1/model-masters/<id> \
  -H "Content-Type: application/json" \
  -d '{ "disk_gb": 750, "replica": 2 }'

To change a model's runtime image, send a PATCH request with runtime_image:

curl -X PATCH <base-url>/api/v1/model-masters/<id> \
  -H "Content-Type: application/json" \
  -d '{ "runtime_image": "vllm/vllm-openai:v0.15.1" }'

Set the job creation warnings

CosmicAC warns when a new job's value differs from the value this model master stores for the same parameter. Two lists of parameter names control which parameters CosmicAC checks, and in which direction.

List	CosmicAC warns when the new job's value is
min	Lower than the model master's value.
max	Higher than the model master's value.

CosmicAC compares numeric parameters only, so a name such as base_os_image never warns.

A PATCH request replaces each list, so send the full list:

curl -X PATCH <base-url>/api/v1/model-masters/<id> \
  -H "Content-Type: application/json" \
  -d '{ "min": ["max_model_length"], "max": ["gpu_memory_utilisation", "replica"] }'

The web interface shows these warnings, because it calls the job validation endpoint before creating a job. When you create a job directly through the API, CosmicAC returns no warnings.

Archive a model master

To archive a model master, send a DELETE request with its id:

curl -X DELETE <base-url>/api/v1/model-masters/<id>

The request archives the model master rather than deleting it. Archived model masters still appear when you list with status=archived.

Update a model master in the web interface

A model appears on the Models page only while a job serves it.

In the left sidebar, click Models.
Find the model's row, then click the pencil icon in the Recommended Config column. A row that reads Set configuration has no model master yet, so saving creates one.
Under Default parameters, edit the values CosmicAC prefills into a new job's Serving configuration. For the recommended values, see Recommended model parameters.
Under Job creation warnings, choose one setting for each parameter.
Click Save.

Setting	CosmicAC warns when the new job's value is
No warning	Never.
Warn below	Lower than the model master's value.
Warn above	Higher than the model master's value.
Warn when different	Different from the model master's value.

To add a model master for a model you don't serve yet, see Add a model master. To archive one, see Archive a model master.