Version 2.22 has reached end of support. Upgrade to a newer supported version.
Ctrlk
Contact support
For the complete documentation index, see llms.txt. This page is also available as Markdown.

Workloads

Workloads are trainings, workspaces, and inferences that are fully controlled by NVIDIA Run:ai. Workloads can be native, third party integrations, and typical Kubernetes workload types. For more information, see Workloads overview.

List workloads.

get

Retrieve a list of active workloads with details.

Authorizations
AuthorizationstringRequired

Bearer authentication

Query parameters
deletedbooleanOptional

Return only deleted resources when true.

offsetinteger · int32Optional

The offset of the first item returned in the collection.

Example: 100
limitinteger · int32 · min: 1 · max: 500Optional

The maximum number of entries to return.

Default: 50
sortOrderstring · enumOptional

Sort results in descending or ascending order.

Default: ascPossible values:
sortBystring · enumOptional

Sort results by a parameter.

Possible values:
filterBystring[]Optional

Filter results by a parameter. Use the format field-name operator value. Operators are == Equals, != Not equals, <= Less than or equal, >= Greater than or equal, =@ contains, !@ Does not contain, =^ Starts with and =$ Ends with. Dates are in ISO 8601 timestamp format and available for operators ==, !=, <= and >=.

Example: ["name!=some-workload-name","allocatedGPU>=2","createdAt>=2021-01-01T00:00:00Z"]
searchstringOptional

Filter results by a free text search.

Example: test project
Responses
200

Executed successfully.

application/json
nextintegerOptionalExample: 1
get/api/v1/workloads
GET /api/v1/workloads HTTP/1.1
Host: app.run.ai
Authorization: Bearer YOUR_SECRET_TOKEN
Accept: */*

{
  "next": 1,
  "workloads": [
    {
      "tenantId": 1001,
      "runningPods": 1,
      "phaseUpdatedAt": "2022-06-08T11:28:24.131Z",
      "k8sPhaseUpdatedAt": "2022-06-08T11:28:24.131Z",
      "updatedAt": "2022-06-08T11:28:24.131Z",
      "source": "CLI",
      "deletedAt": "2022-08-12T19:28:24.131Z",
      "type": "runai-job",
      "name": "very-important-job",
      "id": "123e4567-e89b-12d3-a456-426614174000",
      "priority": 50,
      "priorityClassName": "high-priority",
      "submittedBy": "researcher@run.ai",
      "clusterId": "71f69d83-ba66-4822-adf5-55ce55efd210",
      "projectName": "proj-1",
      "projectId": "1",
      "departmentName": "department-1",
      "departmentId": "1",
      "namespace": "runai-proj-1",
      "createdAt": "2022-01-01T03:49:52.531Z",
      "workloadRequestedResources": {
        "gpuRequestType": "portion",
        "gpu": {
          "limit": 1.5,
          "request": 1
        },
        "gpuMemory": {
          "limit": "2G",
          "request": "200M"
        },
        "cpu": {
          "limit": 1.5,
          "request": 1
        },
        "cpuMemory": {
          "limit": "2G",
          "request": "200M"
        },
        "extendedResources": [
          {
            "resource": "hardware-vendor.example/foo",
            "quantity": 2,
            "exclude": false
          }
        ]
      },
      "podsRequestedResources": {
        "gpuRequestType": "portion",
        "gpu": {
          "limit": 1.5,
          "request": 1
        },
        "gpuMemory": {
          "limit": "2G",
          "request": "200M"
        },
        "cpu": {
          "limit": 1.5,
          "request": 1
        },
        "cpuMemory": {
          "limit": "2G",
          "request": "200M"
        },
        "extendedResources": [
          {
            "resource": "hardware-vendor.example/foo",
            "quantity": 2,
            "exclude": false
          }
        ]
      },
      "allocatedResources": {
        "gpu": 1.5,
        "gpuMemory": "200Mi",
        "cpu": 0.5,
        "cpuMemory": "0B",
        "extendedResources": [
          {
            "resource": "hardware-vendor.example/foo",
            "quantity": 2,
            "exclude": false
          }
        ]
      },
      "actionsSupport": {
        "delete": true,
        "suspend": true
      },
      "phase": "Creating",
      "conditions": [
        {
          "type": "Ready",
          "status": "False",
          "message": "Resource validation failed: ...",
          "reason": "ErrorConfig",
          "lastTransitionTime": "2022-01-01T03:49:52.531Z"
        }
      ],
      "phaseMessage": "Not enough resources in the requested nodepool",
      "k8sPhase": "Pending",
      "requestedPods": {
        "number": 1,
        "min": 2,
        "max": 5,
        "parallelism": 3,
        "completions": 5
      },
      "requestedNodePools": [
        "default"
      ],
      "currentNodePools": [
        "default"
      ],
      "completedAt": "2022-01-01T03:49:52.531Z",
      "images": [
        "alpine:latest"
      ],
      "urls": [
        "text"
      ],
      "datasources": [
        {
          "type": "pvc",
          "name": "my-pvc-datasource-1",
          "id": "123e4567-e89b-12d3-a456-426614174000"
        }
      ],
      "environments": [
        {
          "connections": [
            {
              "name": "my-pytorch-env",
              "toolType": "pytorch",
              "connectionType": "ExternalUrl",
              "url": "http://wandb.com/yourproject",
              "authorizationType": "public",
              "authorizedUsers": [
                "user@company.ai",
                "another@company.ai"
              ],
              "authorizedGroups": [
                "group-a",
                "group-b"
              ],
              "containerPort": 8080
            }
          ],
          "name": "pytorch",
          "id": "123e4567-e89b-12d3-a456-426614174000",
          "replicaType": "Master"
        }
      ],
      "externalConnections": [
        {
          "name": "my-pytorch-env",
          "toolType": "pytorch",
          "connectionType": "ExternalUrl",
          "url": "http://wandb.com/yourproject",
          "authorizationType": "public",
          "authorizedUsers": [
            "user@company.ai",
            "another@company.ai"
          ],
          "authorizedGroups": [
            "group-a",
            "group-b"
          ],
          "containerPort": 8080
        }
      ],
      "distributedFramework": "Pytorch",
      "additionalFields": {
        "ANY_ADDITIONAL_PROPERTY": "anything"
      },
      "preemptible": true,
      "environmentVariables": {
        "ANY_ADDITIONAL_PROPERTY": "text"
      },
      "command": "sleep",
      "arguments": "1000",
      "phaseReason": "text",
      "idleAllocatedGpus": 1,
      "category": "Train"
    }
  ]
}

List workload categories.

get

Retrieves a list of workload categories. These categories are used to classify and monitor different types of workloads within the NVIDIA Run:ai platform.

Authorizations
AuthorizationstringRequired

Bearer authentication

Responses
200

List of categories retrieved successfully.

application/json
get/api/v1/workload-categories
GET /api/v1/workload-categories HTTP/1.1
Host: app.run.ai
Authorization: Bearer YOUR_SECRET_TOKEN
Accept: */*

{
  "categories": [
    {
      "id": "123e4567-e89b-12d3-a456-426614174000",
      "name": "text",
      "description": "text",
      "createdAt": "2026-01-01T00:00:00.000Z",
      "createdBy": "text"
    }
  ]
}

Get workload category by id.

get

Retrieves a specific workload category by its ID. Workload categories are used to classify and monitor different types of workloads within the NVIDIA Run:ai platform.

Authorizations
AuthorizationstringRequired

Bearer authentication

Path parameters
categoryIdstring · uuidRequired

The unique identifier of the workload category.

Responses
200

Category retrieved successfully

application/json
idstring · uuidRequired

The unique identifier of the workload category.

namestringRequired

A unique, human-readable name for the workload category.

descriptionstringOptional

An optional description providing additional details about the category.

createdAtstring · date-timeOptional

The date and time when the workload category was created.

createdBystringOptional

The user who created the workload category.

get/api/v1/workload-categories/{categoryId}
GET /api/v1/workload-categories/{categoryId} HTTP/1.1
Host: app.run.ai
Authorization: Bearer YOUR_SECRET_TOKEN
Accept: */*

{
  "id": "123e4567-e89b-12d3-a456-426614174000",
  "name": "text",
  "description": "text",
  "createdAt": "2026-01-01T00:00:00.000Z",
  "createdBy": "text"
}

List workload types.

get

Retrieves a list of workload types with their configurations - their corresponding workload categories and priorities.

Authorizations
AuthorizationstringRequired

Bearer authentication

Query parameters
externalTypesOnlybooleanOptional

Return only external workload types.

Default: false
Responses
200

List of types retrieved successfully

application/json
get/api/v1/workload-types
GET /api/v1/workload-types HTTP/1.1
Host: app.run.ai
Authorization: Bearer YOUR_SECRET_TOKEN
Accept: */*

{
  "types": [
    {
      "id": "123e4567-e89b-12d3-a456-426614174000",
      "categoryName": "Build",
      "priorityName": "medium",
      "updatedAt": "2026-01-01T00:00:00.000Z",
      "updatedBy": "text"
    }
  ]
}

List workload type by id.

get

Retrieves a specific workload type by its ID.

Authorizations
AuthorizationstringRequired

Bearer authentication

Path parameters
workloadTypeIdstring · uuidRequired

The unique identifier of the workload type.

Responses
200

Type retrieved successfully

application/json
idstring · uuidOptional

The unique identifier of the workload type.

categoryNamestringOptional

The name of the workload category.

Example: Build
priorityNamestringOptional

The name of the workload priority.

Example: medium
updatedAtstring · date-timeOptional

The time at which the type has been updated

updatedBystringOptional

The recent user who updated the type

get/api/v1/workload-types/{workloadTypeId}
GET /api/v1/workload-types/{workloadTypeId} HTTP/1.1
Host: app.run.ai
Authorization: Bearer YOUR_SECRET_TOKEN
Accept: */*

{
  "id": "123e4567-e89b-12d3-a456-426614174000",
  "categoryName": "Build",
  "priorityName": "medium",
  "updatedAt": "2026-01-01T00:00:00.000Z",
  "updatedBy": "text"
}

Update a workload type by id.

put

Update the default category or priority assigned to a workload type.

Authorizations
AuthorizationstringRequired

Bearer authentication

Path parameters
workloadTypeIdstring · uuidRequired

The unique identifier of the workload type to update.

Body
categoryIdstring · uuidOptional

The unique identifier of the workload category.

priorityIdstring · uuidOptional

The unique identifier of the workload priority.

Responses
200

Updated successfully

application/json
idstring · uuidOptional

The unique identifier of the workload type.

categoryNamestringOptional

The name of the workload category.

Example: Build
priorityNamestringOptional

The name of the workload priority.

Example: medium
updatedAtstring · date-timeOptional

The time at which the type has been updated

updatedBystringOptional

The recent user who updated the type

put/api/v1/workload-types/{workloadTypeId}
PUT /api/v1/workload-types/{workloadTypeId} HTTP/1.1
Host: app.run.ai
Authorization: Bearer YOUR_SECRET_TOKEN
Content-Type: application/json
Accept: */*
Content-Length: 105

{
  "categoryId": "123e4567-e89b-12d3-a456-426614174000",
  "priorityId": "123e4567-e89b-12d3-a456-426614174000"
}
{
  "id": "123e4567-e89b-12d3-a456-426614174000",
  "categoryName": "Build",
  "priorityName": "medium",
  "updatedAt": "2026-01-01T00:00:00.000Z",
  "updatedBy": "text"
}

Get a workload.

get

Retrieve workload data using a workloadId.

Authorizations
AuthorizationstringRequired

Bearer authentication

Path parameters
workloadIdstring · uuidRequired

The Universally Unique Identifier (UUID) of the workload.

Responses
200

Executed successfully.

application/json
tenantIdinteger · int32Required

The id of the tenant.

Example: 1001
runningPodsinteger · int32RequiredExample: 1
phaseUpdatedAtstring · date-timeRequiredExample: 2022-06-08T11:28:24.131Z
k8sPhaseUpdatedAtstring · date-timeRequiredExample: 2022-06-08T11:28:24.131Z
updatedAtstring · date-timeRequiredExample: 2022-06-08T11:28:24.131Z
sourcestring · enumRequiredPossible values:
deletedAtstring · date-time · nullableRequiredExample: 2022-08-12T19:28:24.131Z
typestringRequiredExample: runai-job
namestringRequiredExample: very-important-job
idstring · uuidRequired
priorityinteger · int32 · nullableRequiredExample: 50
priorityClassNamestringRequiredExample: high-priority
submittedBystringOptionalExample: researcher@run.ai
clusterIdstring · uuidRequired

The id of the cluster.

Example: 71f69d83-ba66-4822-adf5-55ce55efd210
projectNamestringRequiredExample: proj-1
projectIdstringRequiredExample: 1
departmentNamestringRequiredExample: department-1
departmentIdstringRequiredExample: 1
namespacestringRequiredExample: runai-proj-1
createdAtstring · date-timeRequiredExample: 2022-01-01T03:49:52.531Z
phasestring · enumRequiredPossible values:
phaseMessagestringOptionalExample: Not enough resources in the requested nodepool
k8sPhasestringRequiredExample: Pending
requestedNodePoolsstring[]OptionalExample: default
currentNodePoolsstring[]OptionalExample: default
completedAtstring · date-time · nullableOptionalExample: 2022-01-01T03:49:52.531Z
imagesstring[]OptionalExample: alpine:latest
urlsstring[]Optional
distributedFrameworkstringOptionalExample: Pytorch
preemptibleboolean · nullableOptional
commandstringOptionalExample: sleep
argumentsstringOptionalExample: 1000
phaseReasonstring · enumOptionalPossible values:
idleGpusinteger · nullableOptionalDeprecated

deprecated. use idleAllocatedGpus instead

Example: 3
idleAllocatedGpusnumber · nullableOptional

sum of idle allocated gpus in the workload

Example: 1
categorystringRequired

Category Description

Example: Train
get/api/v1/workloads/{workloadId}
GET /api/v1/workloads/{workloadId} HTTP/1.1
Host: app.run.ai
Authorization: Bearer YOUR_SECRET_TOKEN
Accept: */*

{
  "tenantId": 1001,
  "runningPods": 1,
  "phaseUpdatedAt": "2022-06-08T11:28:24.131Z",
  "k8sPhaseUpdatedAt": "2022-06-08T11:28:24.131Z",
  "updatedAt": "2022-06-08T11:28:24.131Z",
  "source": "CLI",
  "deletedAt": "2022-08-12T19:28:24.131Z",
  "type": "runai-job",
  "name": "very-important-job",
  "id": "123e4567-e89b-12d3-a456-426614174000",
  "priority": 50,
  "priorityClassName": "high-priority",
  "submittedBy": "researcher@run.ai",
  "clusterId": "71f69d83-ba66-4822-adf5-55ce55efd210",
  "projectName": "proj-1",
  "projectId": "1",
  "departmentName": "department-1",
  "departmentId": "1",
  "namespace": "runai-proj-1",
  "createdAt": "2022-01-01T03:49:52.531Z",
  "workloadRequestedResources": {
    "gpuRequestType": "portion",
    "gpu": {
      "limit": 1.5,
      "request": 1
    },
    "gpuMemory": {
      "limit": "2G",
      "request": "200M"
    },
    "cpu": {
      "limit": 1.5,
      "request": 1
    },
    "cpuMemory": {
      "limit": "2G",
      "request": "200M"
    },
    "extendedResources": [
      {
        "resource": "hardware-vendor.example/foo",
        "quantity": 2,
        "exclude": false
      }
    ]
  },
  "podsRequestedResources": {
    "gpuRequestType": "portion",
    "gpu": {
      "limit": 1.5,
      "request": 1
    },
    "gpuMemory": {
      "limit": "2G",
      "request": "200M"
    },
    "cpu": {
      "limit": 1.5,
      "request": 1
    },
    "cpuMemory": {
      "limit": "2G",
      "request": "200M"
    },
    "extendedResources": [
      {
        "resource": "hardware-vendor.example/foo",
        "quantity": 2,
        "exclude": false
      }
    ]
  },
  "allocatedResources": {
    "gpu": 1.5,
    "gpuMemory": "200Mi",
    "cpu": 0.5,
    "cpuMemory": "0B",
    "extendedResources": [
      {
        "resource": "hardware-vendor.example/foo",
        "quantity": 2,
        "exclude": false
      }
    ]
  },
  "actionsSupport": {
    "delete": true,
    "suspend": true
  },
  "phase": "Creating",
  "conditions": [
    {
      "type": "Ready",
      "status": "False",
      "message": "Resource validation failed: ...",
      "reason": "ErrorConfig",
      "lastTransitionTime": "2022-01-01T03:49:52.531Z"
    }
  ],
  "phaseMessage": "Not enough resources in the requested nodepool",
  "k8sPhase": "Pending",
  "requestedPods": {
    "number": 1,
    "min": 2,
    "max": 5,
    "parallelism": 3,
    "completions": 5
  },
  "requestedNodePools": [
    "default"
  ],
  "currentNodePools": [
    "default"
  ],
  "completedAt": "2022-01-01T03:49:52.531Z",
  "images": [
    "alpine:latest"
  ],
  "urls": [
    "text"
  ],
  "datasources": [
    {
      "type": "pvc",
      "name": "my-pvc-datasource-1",
      "id": "123e4567-e89b-12d3-a456-426614174000"
    }
  ],
  "environments": [
    {
      "connections": [
        {
          "name": "my-pytorch-env",
          "toolType": "pytorch",
          "connectionType": "ExternalUrl",
          "url": "http://wandb.com/yourproject",
          "authorizationType": "public",
          "authorizedUsers": [
            "user@company.ai",
            "another@company.ai"
          ],
          "authorizedGroups": [
            "group-a",
            "group-b"
          ],
          "containerPort": 8080
        }
      ],
      "name": "pytorch",
      "id": "123e4567-e89b-12d3-a456-426614174000",
      "replicaType": "Master"
    }
  ],
  "externalConnections": [
    {
      "name": "my-pytorch-env",
      "toolType": "pytorch",
      "connectionType": "ExternalUrl",
      "url": "http://wandb.com/yourproject",
      "authorizationType": "public",
      "authorizedUsers": [
        "user@company.ai",
        "another@company.ai"
      ],
      "authorizedGroups": [
        "group-a",
        "group-b"
      ],
      "containerPort": 8080
    }
  ],
  "distributedFramework": "Pytorch",
  "additionalFields": {
    "ANY_ADDITIONAL_PROPERTY": "anything"
  },
  "preemptible": true,
  "environmentVariables": {
    "ANY_ADDITIONAL_PROPERTY": "text"
  },
  "command": "sleep",
  "arguments": "1000",
  "phaseReason": "text",
  "idleAllocatedGpus": 1,
  "category": "Train",
  "pendingSchedulingMessages": [
    {
      "nodePool": "default",
      "phaseReason": "NonPreemptibleOverQuota",
      "reason": "Non-preemptible over quota",
      "orgType": "text",
      "userMessage": "You have reached the limit of non-preemptible resources"
    }
  ]
}

Count workloads.

get

Retrieve the number of workloads.

Authorizations
AuthorizationstringRequired

Bearer authentication

Query parameters
deletedbooleanOptional

Return only deleted resources when true.

filterBystring[]Optional

Filter results by a parameter. Use the format field-name operator value. Operators are == Equals, != Not equals, <= Less than or equal, >= Greater than or equal, =@ contains, !@ Does not contain, =^ Starts with and =$ Ends with. Dates are in ISO 8601 timestamp format and available for operators ==, !=, <= and >=.

Example: ["name!=some-workload-name","allocatedGPU>=2","createdAt>=2021-01-01T00:00:00Z"]
searchstringOptional

Filter results by a free text search.

Example: test project
Responses
200

Executed successfully.

application/json
countinteger · int64RequiredExample: 1
get/api/v1/workloads/count
GET /api/v1/workloads/count HTTP/1.1
Host: app.run.ai
Authorization: Bearer YOUR_SECRET_TOKEN
Accept: */*

{
  "count": 1
}

Get the workloads telemetry.

get

Retrieves workload data by telemetry type.

Authorizations
AuthorizationstringRequired

Bearer authentication

Query parameters
clusterIdstring · uuidOptional

Filter using the Universally Unique Identifier (UUID) of the cluster.

Example: d73a738f-fab3-430a-8fa3-5241493d7128
nodepoolNamestringOptional

Filter using the nodepool.

Example: default
departmentIdstringOptional

Filter using the department id.

Example: 1
telemetryTypestring · enumRequired

Select a telemetry type.

Possible values:
Responses
200

Executed successfully.

typestringRequired

specifies what data returned

Example: ALLOCATION_RATIO
timestampstring · date-timeRequiredExample: 2023-06-06 12:09:18.211
get/api/v1/workloads/telemetry
GET /api/v1/workloads/telemetry?telemetryType=WORKLOADS_COUNT HTTP/1.1
Host: app.run.ai
Authorization: Bearer YOUR_SECRET_TOKEN
Accept: */*

{
  "type": "ALLOCATION_RATIO",
  "timestamp": "2023-06-06 12:09:18.211",
  "values": [
    {
      "value": "85",
      "groups": [
        {
          "key": "department",
          "value": "1",
          "name": "department-A"
        }
      ]
    }
  ]
}

Get workload metrics data.

get

Retrieves workloads data metrics from the metrics database. Use in reporting and analysis tools.

Authorizations
AuthorizationstringRequired

Bearer authentication

Path parameters
workloadIdstring · uuidRequired

The Universally Unique Identifier (UUID) of the workload.

Query parameters
startstring · date-timeRequired

Start date of time range to fetch data in ISO 8601 timestamp format.

Example: 2023-06-06T12:09:18.211Z
endstring · date-timeRequired

End date of time range to fetch data in ISO 8601 timestamp format.

Example: 2023-06-07T12:09:18.211Z
numberOfSamplesinteger · max: 1000Optional

The number of samples to take in the specified time range.

Default: 20Example: 20
Responses
200

Executed successfully.

get/api/v1/workloads/{workloadId}/metrics
GET /api/v1/workloads/{workloadId}/metrics?metricType=GPU_UTILIZATION&start=2023-06-06T12%3A09%3A18.211Z&end=2023-06-07T12%3A09%3A18.211Z HTTP/1.1
Host: app.run.ai
Authorization: Bearer YOUR_SECRET_TOKEN
Accept: */*

{
  "measurements": [
    {
      "type": "ALLOCATED_GPU",
      "labels": "{'gpu': '3'}",
      "values": [
        {
          "value": "85",
          "timestamp": "2023-06-06 12:09:18.211"
        }
      ]
    }
  ]
}
PreviousPods
NextWorkspaces

Last updated