I am trying to use the sagemaker.huggingface module to run a hugging face estimator as described in this blog, but I encounter the following error:
ModuleNotFoundError: No module named 'sagemaker.huggingface'; 'sagemaker' is not a package
This is the line of code it gets that error on, in the first line of my python file:
from sagemaker.huggingface import HuggingFace
I have installed the sagemaker package (version 2.213.0 when I run conda list) using conda install sagemaker without any errors on my system. If I do import sagemaker, I don't get the error. However, when I check if the huggingface submodule is included in the sagemaker package using the following code:
if 'huggingface' in dir(sagemaker):
print('The huggingface submodule is included in this version of sagemaker.')
else:
print('The huggingface submodule is not included in this version of sagemaker.')
I get the output:
The huggingface submodule is not included in this version of sagemaker.
I am using Python 3.10.13 (when I check python --version) and have created a new conda environment named "distill" with the following packages installed:
conda create --name distill python=3.10.6 -y
conda activate distill
conda install -y pytorch==1.12.1 torchvision==0.13.1 torchaudio==0.12.1 cudatoolkit=11.3 -c pytorch
pip install git+https://github.com/huggingface/[email protected] datasets sentencepiece protobuf==3.20.* tensorboardX
I am running SageMaker Distribution 1.4 on SageMaker Studio.
What could be causing this issue, and how can I resolve it to use the sagemaker.huggingface module?