I have both cudatoolkit and cudnn in my conda environment, but pytorch is still using my CPU

Question

(my_project_env) C:\\Users...\\pythonProject3\>conda list cudatoolkit

# packages in environment at C:\\ProgramData\\anaconda3\\envs\\my_project_env:

# 

# Name                    Version                   Build  Channel

cudatoolkit               11.8.0               hd77b12b_0

(my_project_env) C:\\Users...\\pythonProject3\>conda list cudnn

# packages in environment at C:\\ProgramData\\anaconda3\\envs\\my_project_env:

# 

# Name                    Version                   Build  Channel

cudnn                     8.9.2.26               cuda11_0

(my_project_env) C:\\Users\\Oliver\\PycharmProjects\\pythonProject3\>python asfdsfg.py
Using device: cpu

Here is the relevant code in asfdsfg.py

import torch
import torch.nn as nn

device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
print("Using device:", device)
print(torch.cuda.get_arch_list())

When I look online, it says that these versions of cudatoolkit and cudnn should be compatible. The only oddity I can see is that my cudatoolkit version says 11.8.0, but the installer says 12.2, as does the folder/filepath that was created

C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.2

What am I doing wrong/am I not considering?

Also how would I make my environment recognize a new version of cudNN that's been installed to the local files? I've been overall kinda confused with environments and conda/cuda in general. Thanks!

Answer 1

If you use pip install torch, it may do a cpu-only pytorch install by default.

Instead, you should choose the appropriate installation options on pytorch.org. I recommend choosing conda and install it in a new environment. conda will automatically figure out the versions for you. If this doesn't work or if you want a legacy version, go to download.pytorch.org/whl/torch and look for a version that matches your cuda environment and download the whl file. Then do pip install ***.whl. For example, torch-2.0.0+cu118-cp310-cp310-win_amd64.whl means torch 2.0.0 for cuda 11.8, python 3.10 on windows machines with amd64 architecture.

Answer 2

I faced the same issue, and tried to solve that through reading a lot of resources on the internet, but one trustworthy source has made it work.

https://pytorch.org/get-started/locally/

I'm using Windows, on the link below, you'd be able to choose a few options and type the command provided on the terminal, having activated a specific conda env.

My command was

conda install pytorch torchvision torchaudio pytorch-cuda=11.8 -c pytorch -c nvidia

Then, when I checked through torch.cuda.is_available() inside that env, it gave me True