NGC: NVIDIA's Catalog and Container Registry

What NGC is, how nvcr.io authentication works, how to pull and run NGC containers with Docker, Enroot/Pyxis (Slurm), Apptainer, and Kubernetes, how image tags relate to drivers, and where the NGC CLI, private registry, NVIDIA AI Enterprise, and NIM fit in.

1. The big picture: what NGC is

NGC (originally "NVIDIA GPU Cloud") is NVIDIA's hub for GPU-optimized software. Despite the name, it isn't a cloud you rent GPUs from. It's mainly a catalog and a container registry:

Part	What it is	Where you meet it
NGC Catalog	The website (`catalog.ngc.nvidia.com`) listing containers, models, Helm charts, and resources, with docs and release notes	Browsing, reading release notes, copying pull commands
NGC container registry	A Docker-compatible registry at `nvcr.io`	`docker pull nvcr.io/...`, Enroot, Apptainer, Kubernetes
NGC Helm repository	Helm charts at `helm.ngc.nvidia.com`	Installing the GPU Operator, Network Operator, and other charts
NGC CLI	The `ngc` command-line tool	Downloading models and resources, managing a private registry
Private registry	Your organization's own space inside NGC (`nvcr.io/<org>/<team>/...`)	Sharing internal images with your team

NGC is an app store for GPU software. The catalog is the storefront, nvcr.io is the warehouse the downloads come from, and your API key is your store account.

Why NGC containers matter to infrastructure engineers: NVIDIA builds, tests, and tunes them monthly with a matched stack (CUDA, cuDNN, NCCL, framework, and often HPC-X/UCX). Running a known-good NGC container separates "is the cluster broken?" from "is the user's software environment broken?". That's why validation tools such as HPL and nccl-tests are commonly run from NGC images.

2. What's in the catalog

Asset type	Examples	How you get it
Containers	PyTorch, TensorFlow, Triton Inference Server, NeMo, CUDA base images, HPC Benchmarks, DCGM Exporter	`docker pull nvcr.io/...` (or Enroot / Apptainer / Kubernetes)
Models	Pretrained checkpoints (for example NeMo or TAO models)	`ngc registry model download-version`
Helm charts	GPU Operator, Network Operator, Triton	`helm repo add nvidia https://helm.ngc.nvidia.com/nvidia`
Resources	Scripts, sample datasets, training recipes	`ngc registry resource download-version`
Collections	Curated bundles of the above for one use case	Browsing the catalog

Many public NVIDIA containers can be pulled anonymously today, but some content (NVIDIA AI Enterprise, early-access, NIM, and private-registry images) requires a login. Exam questions and real-world runbooks usually assume you log in, so learn that flow.

3. Image names and tags

nvcr.io / nvidia / pytorch : 24.05-py3
───┬───   ──┬───   ───┬───   ────┬────
registry  org      image     tag = YY.MM release + variant

nvcr.io/nvidia/k8s/dcgm-exporter:<version>      # org "nvidia", team "k8s"
nvcr.io/<your-org>/<your-team>/myimage:1.0     # private registry image

Piece	Meaning
`nvcr.io`	The NGC registry hostname. Not `docker.io`.
org / team	Namespaces. NVIDIA's public images live under the `nvidia` org, sometimes with a team (for example `nvidia/k8s`).
`24.05`	Release month: year 2024, month 05. Framework containers are released roughly monthly.
`-py3`, `-tf2`, `-devel`, `-runtime`, `-base`	Variant suffixes (Python 3 build, framework version, or how much of the toolkit is included)

The tag is a release date, not a framework version. pytorch:24.05-py3 doesn't mean PyTorch 24.05. To find which PyTorch, CUDA, NCCL, and minimum driver it contains, read that release's release notes / support matrix.

Avoid :latest for cluster validation. Pin an exact tag so results are reproducible and every node runs the same stack.

4. Authentication: API keys and `$oauthtoken`

To pull protected content or use the NGC CLI, you generate an API key in the NGC web UI (under your account's setup / API keys page). Then you log in to the registry:

export NGC_API_KEY=<paste-your-key>        # keep it out of shell history and git

echo "$NGC_API_KEY" | docker login nvcr.io --username '$oauthtoken' --password-stdin
# Login Succeeded

Field	Value	Why
Registry	`nvcr.io`	The NGC registry
Username	The literal string `$oauthtoken`	It's not a variable. It tells NGC "the password is an API key."
Password	Your API key	Not your NGC account password

The username is literally $oauthtoken, typed exactly like that. In a shell, put it in single quotes ('$oauthtoken') so the shell doesn't expand it to an empty variable. "Use your email as the username" and "use your NGC password" are both wrong.

"Dollar-oauth-token, single-quoted; the key is the password."

NGC has offered more than one key type over time (legacy API keys and newer personal keys, which typically start with nvapi-). Personal keys can be scoped to specific services, such as the registry. If a key works in the CLI but not for docker login (or vice versa), check which services the key was granted.

Treat the API key like a password: store it in a secrets manager, a Kubernetes Secret, or a root-only credentials file. Never bake it into an image, a Dockerfile, a job script in git, or a public study guide.

5. Pulling and running with Docker

docker pull nvcr.io/nvidia/pytorch:24.05-py3

docker run --gpus all -it --rm \
  --ipc=host --ulimit memlock=-1 --ulimit stack=67108864 \
  -v /raid/datasets:/data \
  nvcr.io/nvidia/pytorch:24.05-py3

Option	Why it's there
`--gpus all`	Exposes the GPUs through the NVIDIA Container Toolkit. Without it the container sees no GPUs. Use `--gpus '"device=0,1"'` for specific GPUs.
`--ipc=host` (or `--shm-size=16g`)	Docker's default `/dev/shm` is only 64 MB. PyTorch data loaders and NCCL's shared-memory transport need much more. Too little shows up as bus errors or NCCL shm failures.
`--ulimit memlock=-1`	Lets RDMA (InfiniBand, GPUDirect) pin memory
`--ulimit stack=67108864`	The larger stack NVIDIA recommends for framework containers
`-v host:container`	Mounts datasets and output. Containers are disposable, so keep data outside.

At startup, NGC framework containers print a banner with the container version and, if relevant, a driver warning, for example that the container needs a newer driver or is running in forward-compatibility mode. Read it: it's the fastest compatibility check you have.

The container toolkit is configured on the host (nvidia-ctk runtime configure --runtime=docker, then restart Docker). DGX OS ships with it set up. On a fresh non-DGX host, a missing toolkit gives could not select device driver "" with capabilities: [[gpu]].

6. Slurm clusters: Enroot and Pyxis

On DGX SuperPOD / BasePOD clusters, users usually don't run Docker directly. Instead:

Enroot (NVIDIA) turns container images into unprivileged sandboxes. There's no root daemon, so it's safe on shared multi-user clusters.
Pyxis (NVIDIA) is a Slurm plugin (SPANK) that adds --container-image and related options to srun/sbatch, using Enroot underneath.

# Credentials for nvcr.io (root-only readable file, one per user)
# ~/.config/enroot/.credentials
machine nvcr.io login $oauthtoken password <your-api-key>

# Run a container through Slurm + Pyxis (note the # between registry and image)
srun -N 2 --ntasks-per-node=8 --gpus-per-node=8 \
     --container-image=nvcr.io#nvidia/pytorch:24.05-py3 \
     --container-mounts=/raid/datasets:/data \
     python train.py

# Or import once into a squashfs file and reuse it
enroot import docker://nvcr.io#nvidia/pytorch:24.05-py3     # → nvidia+pytorch+24.05-py3.sqsh
srun --container-image=./nvidia+pytorch+24.05-py3.sqsh ...

Enroot and Pyxis separate the registry from the image path with #: nvcr.io#nvidia/pytorch:24.05-py3. Docker uses /. Mixing them up is a classic mistake.

Importing to a .sqsh file on shared storage means 100 nodes don't each pull a 10+ GB image from nvcr.io at the same moment. Big clusters often pre-stage images this way, or use a local registry mirror or pull-through cache.

Recent Pyxis versions accept several image reference formats. The # form is the one shown in NVIDIA's Enroot and Pyxis documentation and always works.

7. Apptainer / Singularity

Many HPC sites use Apptainer (formerly Singularity), which also runs unprivileged and converts images into a single .sif file.

export APPTAINER_DOCKER_USERNAME='$oauthtoken'
export APPTAINER_DOCKER_PASSWORD="$NGC_API_KEY"
apptainer pull pytorch_24.05.sif docker://nvcr.io/nvidia/pytorch:24.05-py3

apptainer exec --nv pytorch_24.05.sif python -c "import torch; print(torch.cuda.device_count())"

With Apptainer, GPUs come from --nv, not --gpus all. Forgetting --nv gives a container that can't see the driver.

Older Singularity releases use SINGULARITY_DOCKER_USERNAME / SINGULARITY_DOCKER_PASSWORD instead of the APPTAINER_ names.

8. Kubernetes: pull secrets, GPU Operator, Helm charts

# A pull secret for nvcr.io
kubectl create secret docker-registry ngc-secret \
  --docker-server=nvcr.io \
  --docker-username='$oauthtoken' \
  --docker-password="$NGC_API_KEY"

# Reference it from the pod spec
spec:
  imagePullSecrets:
    - name: ngc-secret
  containers:
    - name: train
      image: nvcr.io/nvidia/pytorch:24.05-py3
      resources:
        limits:
          nvidia.com/gpu: 8

# NVIDIA's Helm charts come from NGC too
helm repo add nvidia https://helm.ngc.nvidia.com/nvidia && helm repo update
helm install gpu-operator nvidia/gpu-operator -n gpu-operator --create-namespace

Symptom	Likely cause
`ErrImagePull` / `ImagePullBackOff` with "unauthorized"	Missing or wrong `imagePullSecrets`, a secret in the wrong namespace, or a bad key
Pod pending: `Insufficient nvidia.com/gpu`	Not an NGC problem. The device plugin (GPU Operator) isn't advertising GPUs, or they're all in use.

The GPU Operator itself pulls its components (driver container, container toolkit, device plugin, DCGM Exporter) from nvcr.io. In air-gapped clusters, mirror those images to a local registry and point the chart's repository values at it.

9. Driver compatibility: will this container run here?

A container brings its own CUDA toolkit and libraries, but it uses the host's kernel driver. The NVIDIA Container Toolkit mounts the host driver's user-space libraries (libcuda.so) into the container. So the question is always: is the host driver new enough for the CUDA version inside the container?

Situation	Result
Host driver ≥ the container's required driver	Works normally
Host driver is older, on a data-center GPU with a supported driver branch	Often works through CUDA forward compatibility (NGC containers include the compat libraries). The banner says so.
Host driver is older and forward compatibility doesn't apply	Fails: "CUDA driver version is insufficient for CUDA runtime version" (error 35), or the banner warns that the container needs a newer driver

Each NGC release's support matrix lists the CUDA version and the minimum driver. Before rolling a new container out to a cluster, check it against the driver version in your BCM software image.

For more on driver vs runtime versions, compatibility modes, and CUDA error codes, see CUDA for infrastructure.

10. The NGC CLI

ngc config set          # prompts for API key, org, team, output format → ~/.ngc/config
ngc config current      # show the active configuration

ngc registry image list "nvidia/*"                          # browse images
ngc registry image info nvidia/pytorch                      # tags and details
ngc registry model list "nvidia/*"
ngc registry model download-version nvidia/<model>:<version>
ngc registry resource download-version nvidia/<resource>:<version>

Use the NGC CLI for…	Use docker / enroot / apptainer for…
Models, resources, datasets, browsing, private-registry admin	Pulling and running container images

Command names and flags change between CLI releases. Run ngc --help or ngc registry --help on your version. The config file stores your API key, so keep ~/.ngc/config readable only by you.

The NGC CLI isn't how you run containers. ngc downloads models and resources. docker pull / enroot import / apptainer pull fetch images.

11. Orgs, teams, and the private registry

An NGC org is your company's account. Teams are subdivisions inside it. Admins assign users roles (for example registry read vs write).
The private registry lets the org host its own images and models at nvcr.io/<org>/<team>/..., visible only to members.

docker tag my-train:1.0 nvcr.io/<your-org>/<your-team>/my-train:1.0
docker push nvcr.io/<your-org>/<your-team>/my-train:1.0

The usual pattern: start FROM nvcr.io/nvidia/pytorch:<tag>, add your code and dependencies, and push the result to your private registry. You inherit NVIDIA's tuned stack and add only what you need.

12. NVIDIA AI Enterprise, NIM, and Base Command

Name	What it is	Relation to NGC
NVIDIA AI Enterprise (NVAIE)	NVIDIA's licensed, supported software suite: production branches with security patches, enterprise support, certified for NVIDIA-Certified Systems	Its containers are delivered through NGC to entitled orgs. DGX systems include an entitlement.
NIM (NVIDIA Inference Microservices)	Prebuilt, optimized inference containers for specific models, with a standard API	Pulled from `nvcr.io`. At runtime they typically need `NGC_API_KEY` to download model weights.
Base Command Platform	NVIDIA's hosted job, workspace, and dataset service for DGX Cloud and SuperPOD-style environments	Uses NGC orgs, teams, and the registry
Base Command Manager (BCM)	Cluster provisioning and management software	A different product. It manages the nodes that then run NGC containers.

# Typical NIM start (the model name is an example; check the NIM's catalog page)
docker run --gpus all --rm -e NGC_API_KEY -p 8000:8000 \
  -v ~/.cache/nim:/opt/nim/.cache \
  nvcr.io/nim/<publisher>/<model>:<tag>

Don't confuse Base Command Platform (hosted, NGC-based job service) with Base Command Manager (on-premises cluster management, formerly Bright Cluster Manager).

13. Containers infra engineers actually use

Image	Used for	Related guide
`nvcr.io/nvidia/hpc-benchmarks`	HPL, HPL-MxP, HPCG, STREAM for acceptance testing	HPL mastery
`nvcr.io/nvidia/pytorch` (and others)	Includes NCCL; common for nccl-tests and real-workload burn-in	NCCL deep dive
`nvcr.io/nvidia/cuda` (`-base` / `-runtime` / `-devel`)	Minimal images: quick `nvidia-smi` test inside a container, or a base to build tools on	CUDA for infrastructure
`nvcr.io/nvidia/k8s/dcgm-exporter`	GPU metrics for Prometheus / Grafana	DCGM deep dive
`nvcr.io/nvidia/cloud-native/dcgm`	DCGM in a container (diagnostics, `dcgmproftester`)	gpu-burn & dcgmproftester
`nvcr.io/nvidia/tritonserver`	Inference serving	(none)

# Quick check: can a container see the GPUs?
docker run --rm --gpus all nvcr.io/nvidia/cuda:12.4.1-base-ubuntu22.04 nvidia-smi

Exact image paths and tags change between releases. Copy the current pull command from the image's catalog page.

14. Pull command builder

Runtime
Image
Tag

15. "How do I…?" chooser

Pick a question above.

16. Sample outputs

What the commands in this guide print. Expand a command to see a typical result and what to check in it.

Illustrative output, not captured from a real system. Hostnames, serial numbers, GUIDs, MAC and IP addresses are invented, and columns and wording vary between versions. Learn the shape of each output and the fields called out under it.

echo "$NGC_API_KEY" | docker login nvcr.io --username '$oauthtoken' --password-stdinAuthenticate Docker to the NGC registry

WARNING! Your password will be stored unencrypted in /root/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store

Login Succeeded

What to look at: 'Login Succeeded' is the only thing that matters; the warning just says the key is stored base64-encoded in config.json, so protect that file or use a credential helper. 'unauthorized: authentication required' usually means the username wasn't literally $oauthtoken (for example it was double-quoted and expanded to empty) or the key lacks registry scope.

docker pull nvcr.io/nvidia/pytorch:24.05-py3Download an NGC framework container

24.05-py3: Pulling from nvidia/pytorch
a8b1c5f80c2d: Pull complete
2b7c3e5a1f04: Pull complete
6f2d4a9e8c31: Pull complete
0c5e7b2d9a13: Pull complete
...
9e4a1d7c3b52: Pull complete
4f8b2e6a0d97: Pull complete
Digest: sha256:6a1f4c2e9b8d7a3f5c0e1d2b4a6c8e0f1a3b5c7d9e2f4a6b8c0d1e3f5a7b9c1d
Status: Downloaded newer image for nvcr.io/nvidia/pytorch:24.05-py3
nvcr.io/nvidia/pytorch:24.05-py3

What to look at: The Digest line identifies exactly which build you got; compare it across nodes when results differ. 'unauthorized' or 'manifest unknown' means a login problem or a mistyped tag (tags are YY.MM-py3, not a PyTorch version).

docker run --gpus all -it --rm --ipc=host --ulimit memlock=-1 --ulimit stack=67108864 -v /raid/datasets:/data nvcr.io/nvidia/pytorch:24.05-py3

Start the container with GPUs


=============
== PyTorch ==
=============

NVIDIA Release 24.05 (build 92629224)
PyTorch Version 2.4.0a0+07cecf4

Container image Copyright (c) 2024, NVIDIA CORPORATION & AFFILIATES. All rights reserved.

Copyright (c) 2014-2024 Facebook Inc.
...
Copyright (c) 2013-2016 The Caffe contributors
All rights reserved.

Various files include modifications (c) NVIDIA CORPORATION & AFFILIATES.  All rights reserved.

This container image and its contents are governed by the NVIDIA Deep Learning Container License.
By pulling and using the container, you accept the terms and conditions of this license:
https://developer.nvidia.com/ngc/nvidia-deep-learning-container-license

root@7c2e9a41b3d5:/workspace#

What to look at: The banner gives the release (24.05) to look up in the support matrix for the minimum driver. Without --ipc=host the banner adds a NOTE that the SHMEM limit is the default 64MB, and a 'CUDA driver version is insufficient' warning here means the host driver is too old for this tag.

srun -N 1 --gpus-per-node=8 --container-image=nvcr.io#nvidia/pytorch:24.05-py3 nvidia-smi -LRun an NGC image through Pyxis

pyxis: importing docker image: nvcr.io#nvidia/pytorch:24.05-py3
pyxis: imported docker image: nvcr.io#nvidia/pytorch:24.05-py3
GPU 0: NVIDIA H100 80GB HBM3 (UUID: GPU-5f1c0a2e-7b3d-4c11-9e02-1a0b0c0d0e00)
GPU 1: NVIDIA H100 80GB HBM3 (UUID: GPU-5f1c1a2e-7b3d-4c11-9e02-1a0b0c0d0e01)
GPU 2: NVIDIA H100 80GB HBM3 (UUID: GPU-5f1c2a2e-7b3d-4c11-9e02-1a0b0c0d0e02)
GPU 3: NVIDIA H100 80GB HBM3 (UUID: GPU-5f1c3a2e-7b3d-4c11-9e02-1a0b0c0d0e03)
GPU 4: NVIDIA H100 80GB HBM3 (UUID: GPU-5f1c4a2e-7b3d-4c11-9e02-1a0b0c0d0e04)
GPU 5: NVIDIA H100 80GB HBM3 (UUID: GPU-5f1c5a2e-7b3d-4c11-9e02-1a0b0c0d0e05)
GPU 6: NVIDIA H100 80GB HBM3 (UUID: GPU-5f1c6a2e-7b3d-4c11-9e02-1a0b0c0d0e06)
GPU 7: NVIDIA H100 80GB HBM3 (UUID: GPU-5f1c7a2e-7b3d-4c11-9e02-1a0b0c0d0e07)

What to look at: Pyxis prints the import lines, then the job output; eight GPUs means Slurm's GPU allocation reached the container. 'pyxis: failed to import docker image' usually means the docker-style nvcr.io/ path was used instead of nvcr.io#, or ~/.config/enroot/.credentials is missing.

enroot import docker://nvcr.io#nvidia/pytorch:24.05-py3Convert the image to a squashfs file

[INFO] Querying registry for permission grant
[INFO] Authenticating with user: $oauthtoken
[INFO] Using credentials from file: /home/alice/.config/enroot/.credentials
[INFO] Authentication succeeded
[INFO] Fetching image manifest list
[INFO] Fetching image manifest
[INFO] Downloading 74 missing layers...

100% 74:0=0s 9e4a1d7c3b52...

[INFO] Extracting image layers...

100% 73:0=0s a8b1c5f80c2d...

[INFO] Converting whiteouts...

100% 73:0=0s a8b1c5f80c2d...

[INFO] Creating squashfs filesystem...

Parallel mksquashfs: Using 224 processors
Creating 4.0 filesystem on /raid/images/nvidia+pytorch+24.05-py3.sqsh, block size 131072.
...
[==========================================================|] 412893/412893 100%

What to look at: Look for 'Authentication succeeded' and the .sqsh path at the end (registry+org+image+tag joined with +). '[ERROR] URL https://nvcr.io/... returned error code: 401 Unauthorized' means the credentials file is missing or its machine line is wrong.

apptainer pull pytorch_24.05.sif docker://nvcr.io/nvidia/pytorch:24.05-py3Build a SIF image from NGC

INFO:    Converting OCI blobs to SIF format
INFO:    Starting build...
Getting image source signatures
Copying blob a8b1c5f80c2d done
Copying blob 2b7c3e5a1f04 done
...
Copying config 3c9d1e7b2a done
Writing manifest to image destination
Storing signatures
...
INFO:    Creating SIF file...
INFO:    Build complete: pytorch_24.05.sif

What to look at: Success ends with 'Build complete' and the .sif name. An 'unauthorized: authentication required' error means APPTAINER_DOCKER_USERNAME/PASSWORD weren't exported in this shell.

apptainer exec --nv pytorch_24.05.sif python -c "import torch; print(torch.cuda.device_count())"GPUs visible inside Apptainer

What to look at: 8 on a DGX H100 means --nv bound the host driver into the container. 0 (often with a 'could not find any nv libraries' warning) means --nv was left out or the host driver isn't loaded.

kubectl create secret docker-registry ngc-secret --docker-server=nvcr.io --docker-username='$oauthtoken' --docker-password="$NGC_API_KEY"

Kubernetes pull secret for nvcr.io

secret/ngc-secret created

What to look at: The secret is created in the current namespace only (default here). Pods in another namespace that reference ngc-secret will sit in ImagePullBackOff until you create it there with -n <namespace>.

helm repo add nvidia https://helm.ngc.nvidia.com/nvidia && helm repo updateAdd NVIDIA's Helm chart repository

"nvidia" has been added to your repositories
Hang tight while we grab the latest from your chart repositories...
...Successfully got an update from the "nvidia" chart repository
Update Complete. ⎈Happy Helming!⎈

What to look at: You want the 'Successfully got an update' line for the nvidia repository. A 'failed to fetch ... index.yaml' error means a proxy or firewall is blocking helm.ngc.nvidia.com.

helm install gpu-operator nvidia/gpu-operator -n gpu-operator --create-namespaceDeploy the GPU Operator chart

NAME: gpu-operator
LAST DEPLOYED: Tue Nov 10 09:30:12 2026
NAMESPACE: gpu-operator
STATUS: deployed
REVISION: 1
TEST SUITE: None

What to look at: STATUS: deployed only means Helm created the objects; the operator still has to roll out its DaemonSets, so follow with kubectl get pods -n gpu-operator. 'cannot re-use a name that is still in use' means a release with that name already exists.

ngc config setConfigure the NGC CLI interactively

Enter API key [no-apikey]. Choices: [<VALID_APIKEY>, 'no-apikey']: <redacted>
Enter CLI output format type [ascii]. Choices: ['ascii', 'csv', 'json']: ascii
Enter org [no-org]. Choices: ['0123456789abcdef (dgx-lab-org)']: 0123456789abcdef
Enter team [no-team]. Choices: ['infra', 'no-team']: infra
Enter ace [no-ace]. Choices: ['no-ace']: no-ace
Validating configuration...
Successfully validated configuration.
Saving configuration...
Successfully saved NGC configuration to /home/alice/.ngc/config

What to look at: The org and team choices offered come from the key itself, so seeing your org listed confirms the key is valid. 'Invalid apikey' means a wrong or revoked key, and choosing the wrong org later shows up as 403/permission errors on private registry paths.

ngc config currentShow active NGC CLI settings

+-------------+------------------------------------+---------------+
| key         | value                              | source        |
+-------------+------------------------------------+---------------+
| apikey      | <redacted>                         | user settings |
| format_type | ascii                              | user settings |
| org         | dgx-lab-org (0123456789abcdef)     | user settings |
| team        | infra                              | user settings |
| ace         | no-ace                             | user settings |
+-------------+------------------------------------+---------------+

What to look at: Check org and team before pulling or pushing private artifacts, and the source column shows whether a value came from ~/.ngc/config or an environment variable. The real CLI masks the key; never paste this output anywhere if it shows more than the last few characters.

ngc registry image list "nvidia/*"Browse NGC container images

+-----------------------+-----------------------+------------+------------+-------------+------------+
| Name                  | Repository            | Latest Tag | Image Size | Updated     | Permission |
|                       |                       |            |            | Date        |            |
+-----------------------+-----------------------+------------+------------+-------------+------------+
| CUDA                  | nvidia/cuda           | 12.4.1-bas | 112.4 MB   | Oct 03,     | unlocked   |
|                       |                       | e-ubuntu22 |            | 2024        |            |
|                       |                       | .04        |            |             |            |
| PyTorch               | nvidia/pytorch        | 24.10-py3  | 9.97 GB    | Oct 29,     | unlocked   |
|                       |                       |            |            | 2024        |            |
| TensorFlow            | nvidia/tensorflow     | 24.10-tf2- | 8.63 GB    | Oct 29,     | unlocked   |
|                       |                       | py3        |            | 2024        |            |
| Triton Inference      | nvidia/tritonserver   | 24.10-py3  | 7.71 GB    | Oct 30,     | unlocked   |
| Server                |                       |            |            | 2024        |            |
...
+-----------------------+-----------------------+------------+------------+-------------+------------+

What to look at: The Repository column (org/image) is what goes after nvcr.io/ in docker pull, and Latest Tag is just the newest release, not a recommendation. Wrapped cells are normal in ascii format; use --format_type csv or json for scripts.

ngc registry image info nvidia/pytorchTags and details for one image

--------------------------------------------------
  Image Repository Information
    Name: pytorch
    Display Name: PyTorch
    Short Description: PyTorch is a GPU accelerated tensor computational framework. Functionality can be extended with common Python libraries such as NumPy and SciPy.
    Built By: NVIDIA
    Publisher: Facebook
    Multinode Support: False
    Multi-Arch Support: True
    Logo: https://assets.nvidiagrid.net/ngc/logos/Pytorch.png
    Labels: NVIDIA AI Enterprise Supported, Deep Learning, Machine Learning
    Public: Yes
    Last Updated: Oct 29, 2024
    Latest Image Size: 9.97 GB
    Latest Tag: 24.10-py3
    Tags:
        24.10-py3
        24.09-py3
        24.08-py3
        24.07-py3
        24.06-py3
        24.05-py3
        ...
--------------------------------------------------

What to look at: Confirm the tag you plan to pin (24.05-py3) is listed. Tags are release dates, so look up the driver requirement for that release in the framework support matrix instead of guessing from the number.

ngc registry model list "nvidia/*"Browse pretrained models

+--------------------+------------------------------+----------------+-------------+-----------+-----------+-----------------+------------+
| Name               | Repository                   | Latest Version | Application | Framework | Precision | Last Modified   | Permission |
+--------------------+------------------------------+----------------+-------------+-----------+-----------+-----------------+------------+
| BERT Large         | nvidia/bert_pyt_ckpt_large_q | 21.03.0        | Language    | PyTorch   | AMP       | Mar 30, 2021    | unlocked   |
| ResNet50 v1.5      | nvidia/resnet50_pyt_amp      | 21.03.0        | Image Class | PyTorch   | AMP       | Apr 01, 2021    | unlocked   |
...
+--------------------+------------------------------+----------------+-------------+-----------+-----------+-----------------+------------+

What to look at: Use the Repository and Latest Version columns to build the <org>/<model>:<version> argument for download-version. An empty table usually means the pattern or the configured org is wrong, not that the catalog is empty.

ngc registry model download-version nvidia/<model>:<version>Download one model version locally

Getting files to download...
  ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ • 1.3/1.3 GiB • Remaining: 0:00:00 • 118.2 MB/s • Elapsed: 0:00:12 • Total: 3 - Completed: 3 - Failed: 0

--------------------------------------------------------------------------------
   Download status: COMPLETED
   Downloaded local path model: /home/alice/<model>_v<version>
   Total files downloaded: 3
   Total downloaded size: 1.3 GiB
   Started at: 2026-11-10 09:41:05
   Completed at: 2026-11-10 09:41:17
   Duration taken: 12s
--------------------------------------------------------------------------------

What to look at: You want 'Download status: COMPLETED' with 'Failed: 0', and the local path tells you where the files landed (resource download-version prints the same summary). FAILED or a non-zero failed count means a partial download you should not use.

docker push nvcr.io/<your-org>/<your-team>/my-train:1.0Push an image to the private registry

The push refers to repository [nvcr.io/<your-org>/<your-team>/my-train]
3b1f9a7c2e40: Pushed
8d4e2c6a1b93: Pushed
5f70bf18a086: Mounted from nvidia/pytorch
...
1.0: digest: sha256:0d7e3b9a5c1f4e2d8b6a0c3e5f7a9b1d3c5e7f9a2b4c6d8e0f1a3b5c7d9e2f4a size: 9217

What to look at: 'Mounted from nvidia/pytorch' means the base layers were reused instead of uploaded again, and the final digest line confirms the push. 'denied: requested access to the resource is denied' means the org/team path is wrong or your key can't write there.

docker run --gpus all --rm -e NGC_API_KEY -p 8000:8000 -v ~/.cache/nim:/opt/nim/.cache nvcr.io/nim/<publisher>/<model>:<tag>

Start a NIM microservice


===========================================
== NVIDIA Inference Microservice LLM NIM ==
===========================================

NVIDIA Inference Microservice LLM NIM Version 1.2.3
Model: <publisher>/<model>

Container image Copyright (c) 2016-2024, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
...
INFO 11-10 09:52:14.112 ngc_profile.py:231] Running NIM without LoRA. Only looking for compatible profiles that do not support LoRA.
INFO 11-10 09:52:14.113 ngc_profile.py:233] Detected 1 compatible profile(s).
INFO 11-10 09:52:14.113 ngc_injector.py:152] Valid profile: 5a3f...(tensorrt_llm-h100-fp8-tp1-throughput) on GPUs [0]
INFO 11-10 09:52:14.114 ngc_injector.py:206] Selected profile: 5a3f...(tensorrt_llm-h100-fp8-tp1-throughput)
INFO 11-10 09:52:15.020 ngc_injector.py:241] Model workspace is now ready. It took 0.906 seconds
...
INFO 11-10 09:53:40.551 server.py:82] Started server process [1]
INFO 11-10 09:53:40.552 on.py:48] Waiting for application startup.
INFO 11-10 09:53:40.560 on.py:62] Application startup complete.
INFO 11-10 09:53:40.561 server.py:214] Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit)

What to look at: Check the selected profile matches your GPU (h100 here) and wait for 'Uvicorn running on http://0.0.0.0:8000' before sending requests. A 401/403 while downloading the model means NGC_API_KEY wasn't passed into the container, and no compatible profile means the GPU or GPU count isn't supported by that NIM.

docker run --rm --gpus all nvcr.io/nvidia/cuda:12.4.1-base-ubuntu22.04 nvidia-smiQuick GPU check from an NGC image


==========
== CUDA ==
==========

CUDA Version 12.4.1

Container image Copyright (c) 2016-2023, NVIDIA CORPORATION & AFFILIATES. All rights reserved.

This container image and its contents are governed by the NVIDIA Deep Learning Container License.
By pulling and using the container, you accept the terms and conditions of this license:
https://developer.nvidia.com/ngc/nvidia-deep-learning-container-license

A copy of this license is made available in this container at /NGC-DL-CONTAINER-LICENSE for your convenience.

Tue Nov 10 09:21:07 2026
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 550.127.08             Driver Version: 550.127.08     CUDA Version: 12.4     |
|-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA H100 80GB HBM3          On  |   00000000:18:00.0 Off |                    0 |
| N/A   31C    P0             69W /  700W |       1MiB /  81559MiB |      0%      Default |
|                                         |                        |             Disabled |
+-----------------------------------------+------------------------+----------------------+
...   (GPUs 1-6 trimmed)
|   7  NVIDIA H100 80GB HBM3          On  |   00000000:DB:00.0 Off |                    0 |
| N/A   33C    P0             72W /  700W |       1MiB /  81559MiB |      0%      Default |
|                                         |                        |             Disabled |
+-----------------------------------------+------------------------+----------------------+

+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI        PID   Type   Process name                              GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
|  No running processes found                                                             |
+-----------------------------------------------------------------------------------------+

What to look at: The CUDA Version in the header is the highest the host driver supports, not the container's CUDA (12.4.1 in the banner); the container's CUDA must not exceed it unless forward compatibility applies. Seeing all eight GPUs proves the toolkit and driver path; 'could not select device driver' means the Container Toolkit isn't configured.

17. Scenarios

18. Searchable reference

Item	Meaning

19. Flashcards (click to flip)

20. Exam traps

The registry is nvcr.io. The catalog website is catalog.ngc.nvidia.com.
docker login nvcr.io: username is the literal $oauthtoken, password is the API key.
Single-quote '$oauthtoken' in shells and YAML-generating commands.
Tags like 24.05 are release dates. Check the release notes for the framework, CUDA, and driver versions.
Enroot / Pyxis image syntax uses #: nvcr.io#nvidia/pytorch:24.05-py3.
GPUs in containers: Docker --gpus all, Apptainer --nv, Kubernetes nvidia.com/gpu limits.
Kubernetes pulls need an imagePullSecrets entry for nvcr.io in the same namespace as the pod.
The container uses the host driver. A newer container on an old driver can fail (error 35) unless forward compatibility applies.
NCCL or data-loader failures in Docker often mean small /dev/shm: add --ipc=host or --shm-size.
The NGC CLI downloads models and resources. It doesn't run containers.
Base Command Platform (NGC-hosted jobs) ≠ Base Command Manager (cluster management).
Never bake an API key into an image or commit it. Use secrets.

21. Practice quiz (22 questions)

Score: 0 / 0