Launching Workloads with GPU Fractions

This quick start provides a step-by-step walkthrough for running a Jupyter Notebook workspace using GPU fractions.

NVIDIA Run:ai’s GPU fractions provides an agile and easy-to-use method to share a GPU or multiple GPUs across workloads. With GPU fractions, you can divide the GPU/s memory into smaller chunks and share the GPU/s compute resources between different workloads and users, resulting in higher GPU utilization and more efficient resource allocation.

Prerequisites

Before you start, make sure:

You have created a project or have one created for you.
The project has an assigned quota of at least 0.5 GPU.

Note

Flexible workload submission is disabled by default. If unavailable, your administrator must enable it under General Settings → Workloads → Flexible workload submission.

Step 1: Logging In

Browse to the provided NVIDIA Run:ai user interface and log in with your credentials.

Step 2: Submitting a Workspace

Go to the Workload manager → Workloads
Click +NEW WORKLOAD and select Workspace
Select under which cluster to create the workload
Select the project in which your workspace will run
Select Start from scratch to launch a new workspace quickly
Enter a name for the workspace (if the name already exists in the project, you will be requested to submit a different name)
Under Submission, select Flexible and click CONTINUE
Click the load icon. A side pane appears, displaying a list of available environments. Select the ‘jupyter-lab’ environment for your workspace (Image URL: jupyter/scipy-notebook)
- If ‘jupyter-lab’ is not displayed in the gallery, follow the below steps to create a one-time environment configuration:
  - Enter the jupyter-lab Image URL - jupyter/scipy-notebook
  - Tools - Set the connection for your tool
    Click +TOOL
    Select Jupyter tool from the list
  - Set the runtime settings for the environment. Click +COMMAND & ARGUMENTS and add the following:
    Enter the command - start-notebook.sh
    Enter the arguments - --NotebookApp.base_url=/${RUNAI_PROJECT}/${RUNAI_JOB_NAME} --NotebookApp.token=''
    Note: If host-based routing is enabled on the cluster, enter the --NotebookApp.token='' only.
Click the load icon. A side pane appears, displaying a list of available compute resources. Select the ‘small-fraction’ compute resource for your workspace.
- If ‘small-fraction’ is not displayed in the gallery, follow the below steps to create a one-time compute resource configuration:
  - Set GPU devices per pod - 1
  - Set GPU memory per device
    Select % (of device) - Fraction of a GPU device’s memory
    Set the memory Request - 10 (the workload will allocate 10% of the GPU memory)
  - Optional: set the CPU compute per pod - 0.1 cores (default)
  - Optional: set the CPU memory per pod - 100 MB (default)
Click CREATE WORKSPACE

Copy the following command to your terminal. Make sure to update the below with the name of your project and workload. For more details, see CLI reference:

runai project set "project-name"
runai workspace submit "workload-name" --image jupyter/scipy-notebook \
--gpu-devices-request 0.1 --command --external-url container=8888 \
--name-prefix jupyter --command -- start-notebook.sh \
--NotebookApp.base_url=/${RUNAI_PROJECT}/${RUNAI_JOB_NAME} --NotebookApp.token=

Copy the following command to your terminal. Make sure to update the below with the name of your project and workload. For more details, see CLI reference:

runai config project "project-name"
runai submit "workload-name" --jupyter -g 0.1

Copy the following command to your terminal. Make sure to update the below parameters. For more details, see Workspaces API:

curl -L 'https://<COMPANY-URL>/api/v1/workloads/workspaces' \ 
-H 'Content-Type: application/json' \
-H 'Authorization: Bearer <TOKEN>' \ 
-d '{ 
    "name": "workload-name", 
    "projectId": "<PROJECT-ID>",  
    "clusterId": "<CLUSTER-UUID>", 
    "spec": {
        "command" : "start-notebook.sh",
        "args" : "--NotebookApp.base_url=/${RUNAI_PROJECT}/${RUNAI_JOB_NAME} --NotebookApp.token=''",
        "image": "jupyter/scipy-notebook",
        "compute": {
            "gpuDevicesRequest": 1,
            "gpuRequestType": "portion",
            "gpuPortionRequest": 0.1

        },
        "exposedUrls" : [
            { 
                "container" : 8888,
                "toolType": "jupyter-notebook", 
                "toolName": "Jupyter" 
            }
        ]
    }
}

<COMPANY-URL> - The link to the NVIDIA Run:ai user interface
<TOKEN> - The API access token obtained in Step 1
<PROJECT-ID> - The ID of the Project the workload is running on. You can get the Project ID via the Get Projects API.
<CLUSTER-UUID> - The unique identifier of the Cluster. You can get the Cluster UUID via the Get Clusters API.
toolType will show the Jupyter icon when connecting to the Jupyter tool via the user interface.
toolName will show when connecting to the Jupyter tool via the user interface.

Note

The above API snippet runs with NVIDIA Run:ai clusters of 2.18 and above only.

Step 3: Connecting to the Jupyter Notebook

Select the newly created workspace with the Jupyter application that you want to connect to
Click CONNECT
Select the Jupyter tool. The selected tool is opened in a new tab on your browser.

Next Steps

Manage and monitor your newly created workload using the Workloads table.

PreviousQuick Starts NextLaunching Workloads with Dynamic GPU Fractions

Last updated 3 months ago