Skip to main content

Troubleshooting NVIDIA GPU

This guide provides steps to troubleshoot and resolve issues when the Jan app does not utilize the NVIDIA GPU on Windows and Linux systems.

1. Ensure GPU Mode Requirements​

NVIDIA Driver​

  • Install an NVIDIA Driver supporting CUDA 11.7 or higher.
    • Use the following command to verify the installation:
nvidia-smi

CUDA Toolkit​

  • Install a CUDA toolkit compatible with your NVIDIA driver.
    • Use the following command to verify the installation:
nvcc --version

2. Switch to GPU Mode​

Jan defaults to CPU mode but automatically switches to GPU mode if your system supports it, selecting the GPU with the highest VRAM. Check this setting in Settings > Advanced Settings.

Troubleshooting Tips​

If GPU mode isn't enabled by default:

  1. Confirm that you have installed an NVIDIA driver supporting CUDA 11.7 or higher. Refer to CUDA compatibility.
  2. Ensure compatibility of the CUDA toolkit with your NVIDIA driver. Refer to CUDA compatibility.
  3. For Linux, add CUDA's .so libraries to the LD_LIBRARY_PATH. For Windows, ensure that CUDA's .dll libraries are in the PATH. Refer to Windows setup.

3. Check GPU Settings​

  1. Navigate to Settings > Advanced Settings > Jan Data Folder to access GPU settings.
  2. Open the settings.json file in the settings folder. Here's an example:
~/jan/settings/settings.json
{
  "notify": true,
  "run_mode": "gpu",
  "nvidia_driver": {
    "exist": true,
    "version": "531.18"
  },
  "cuda": {
    "exist": true,
    "version": "12"
  },
  "gpus": [
    {
      "id": "0",
      "vram": "12282"
    },
    {
      "id": "1",
      "vram": "6144"
    },
    {
      "id": "2",
      "vram": "6144"
    }
  ],
  "gpu_highest_vram": "0"
}

4. Restart Jan​

Restart Jan application to make sure it works.

Troubleshooting Tips​

  • Ensure nvidia_driver and cuda fields indicate installed software.
  • If gpus field is empty or lacks your GPU, check NVIDIA driver and CUDA toolkit installations.
  • For further assistance, share the settings.json file.

Tested Configurations​

  • Windows 11 Pro 64-bit:

    • GPU: NVIDIA GeForce RTX 4070ti
    • CUDA: 12.2
    • NVIDIA driver: 531.18 (Bare metal)

  • Ubuntu 22.04 LTS:

    • GPU: NVIDIA GeForce RTX 4070ti
    • CUDA: 12.2
    • NVIDIA driver: 545 (Bare metal)

  • Ubuntu 20.04 LTS:

    • GPU: NVIDIA GeForce GTX 1660ti
    • CUDA: 12.1
    • NVIDIA driver: 535 (Proxmox VM passthrough GPU)

  • Ubuntu 18.04 LTS:

    • GPU: NVIDIA GeForce GTX 1660ti
    • CUDA: 12.1
    • NVIDIA driver: 535 (Proxmox VM passthrough GPU)

Common Issues and Solutions​

  1. If the issue persists, try installing the Nightly version.
  2. Ensure your (V)RAM is accessible; some users with virtual RAM may require additional configuration.
  3. Seek assistance in Jan Discord.