I am running python 3 on a ubuntu server 18.04 LTS. I have installed python with anaconda. I have created a virtual env named llm_110623. I have tried exporting the env to a yaml file so that I can change the name and use it as a base to create a new virtual env llm_022624. When I try creating the new virtual env llm_022624 I'm getting the error messages below. I have also tried cloning the llm_110623 env, but when I do that I am able to activate the env but can not install/update packages and it doesn't show up in the list of virtual env. I'm mentioning both issues because I think they may be related. It seems like maybe conda is having trouble resolving dependencies or adding env to path. I've added all the code and errors I've gotten below along with the yaml file. Can someone please tell me what the issue is and suggest how to solve?
code:
# activating conda virtual env
conda activate llm_110623
# exporting env to yaml
conda env export > llm_022624.yml
# then updated name of env in yaml file and saved
# deactivated current virtual env
conda deactivate
# tried to create env from yaml
conda env create -f llm_022624.yml
error:
Collecting package metadata (repodata.json): \ (base)
# next tried first to activate conda
conda activate
# then tried to create env again
conda env create -f llm_022624.yml
error:
Collecting package metadata (repodata.json): - Fatal Python error: Immortal interned string died.
Thread 0x00007fb5a5ffb700 (most recent call first):
File "/home/username/anaconda3/lib/python3.7/concurrent/futures/thread.py", line 78 in _worker
File "/home/username/anaconda3/lib/python3.7/threading.py", line 870 in run
File "/home/username/anaconda3/lib/python3.7/threading.py", line 926 in _bootstrap_inner
File "/home/username/anaconda3/lib/python3.7/threading.py", line 890 in _bootstrap
Thread 0x00007fb5a67fc700 (most recent call first):
File "/home/username/anaconda3/lib/python3.7/concurrent/futures/thread.py", line 78 in _worker
File "/home/username/anaconda3/lib/python3.7/threading.py", line 870 in run
File "/home/username/anaconda3/lib/python3.7/threading.py", line 926 in _bootstrap_inner
File "/home/username/anaconda3/lib/python3.7/threading.py", line 890 in _bootstrap
Thread 0x00007fb5a6ffd700 (most recent call first):
File "/home/username/anaconda3/lib/python3.7/concurrent/futures/thread.py", line 78 in _worker
File "/home/username/anaconda3/lib/python3.7/threading.py", line 870 in run
File "/home/username/anaconda3/lib/python3.7/threading.py", line 926 in _bootstrap_inner
File "/home/username/anaconda3/lib/python3.7/threading.py", line 890 in _bootstrap
Thread 0x00007fb5a77fe700 (most recent call first):
File "/home/username/anaconda3/lib/python3.7/concurrent/futures/thread.py", line 78 in _worker
File "/home/username/anaconda3/lib/python3.7/threading.py", line 870 in run
File "/home/username/anaconda3/lib/python3.7/threading.py", line 926 in _bootstrap_inner
File "/home/username/anaconda3/lib/python3.7/threading.py", line 890 in _bootstrap
Thread 0x00007fb5a7fff700 (most recent call first):
File "/home/username/anaconda3/lib/python3.7/concurrent/futures/thread.py", line 78 in _worker
File "/home/username/anaconda3/lib/python3.7/threading.py", line 870 in run
File "/home/username/anaconda3/lib/python3.7/threading.py", line 926 in _bootstrap_inner
File "/home/username/anaconda3/lib/python3.7/threading.py", line 890 in _bootstrap
Thread 0x00007fb5c4ff9700 (most recent call first):
File "/home/username/anaconda3/lib/python3.7/concurrent/futures/thread.py", line 78 in _worker
File "/home/username/anaconda3/lib/python3.7/threading.py", line 870 in run
File "/home/username/anaconda3/lib/python3.7/threading.py", line 926 in _bootstrap_inner
File "/home/username/anaconda3/lib/python3.7/threading.py", line 890 in _bootstrap
Thread 0x00007fb5c57fa700 (most recent call first):
File "/home/username/anaconda3/lib/python3.7/concurrent/futures/thread.py", line 78 in _worker
File "/home/username/anaconda3/lib/python3.7/threading.py", line 870 in run
File "/home/username/anaconda3/lib/python3.7/threading.py", line 926 in _bootstrap_inner
File "/home/username/anaconda3/lib/python3.7/threading.py", line 890 in _bootstrap
Thread 0x00007fb5c5ffb700 (most recent call first):
File "/home/username/anaconda3/lib/python3.7/concurrent/futures/thread.py", line 78 in _worker
File "/home/username/anaconda3/lib/python3.7/threading.py", line 870 in run
File "/home/username/anaconda3/lib/python3.7/threading.py", line 926 in _bootstrap_inner
File "/home/username/anaconda3/lib/python3.7/threading.py", line 890 in _bootstrap
Thread 0x00007fb5c67fc700 (most recent call first):
File "/home/username/anaconda3/lib/python3.7/concurrent/futures/thread.py", line 78 in _worker
File "/home/username/anaconda3/lib/python3.7/threading.py", line 870 in run
File "/home/username/anaconda3/lib/python3.7/threading.py", line 926 in _bootstrap_inner
File "/home/username/anaconda3/lib/python3.7/threading.py", line 890 in _bootstrap
Thread 0x00007fb5c6ffd700 (most recent call first):
File "/home/username/anaconda3/lib/python3.7/concurrent/futures/thread.py", line 78 in _worker
File "/home/username/anaconda3/lib/python3.7/threading.py", line 870 in run
File "/home/username/anaconda3/lib/python3.7/threading.py", line 926 in _bootstrap_inner
File "/home/username/anaconda3/lib/python3.7/threading.py", line 890 in _bootstrap
Thread 0x00007fb5c77fe700 (most recent call first):
File "/home/username/anaconda3/lib/python3.7/concurrent/futures/thread.py", line 78 in _worker
File "/home/username/anaconda3/lib/python3.7/threading.py", line 870 in run
File "/home/username/anaconda3/lib/python3.7/threading.py", line 926 in _bootstrap_inner
File "/home/username/anaconda3/lib/python3.7/threading.py", line 890 in _bootstrap
Current thread 0x00007fb5c7fff700 (most recent call first):
File "/home/username/anaconda3/lib/python3.7/site-packages/conda/auxlib/logz.py", line 149 in stringify
File "/home/username/anaconda3/lib/python3.7/site-packages/conda/core/subdir_data.py", line 704 in fetch_repodata_remote_request
File "/home/username/anaconda3/lib/python3.7/site-packages/conda/core/subdir_data.py", line 379 in _load
File "/home/username/anaconda3/lib/python3.7/site-packages/conda/core/subdir_data.py", line 210 in load
File "/home/username/anaconda3/lib/python3.7/site-packages/conda/core/subdir_data.py", line 145 in query
File "/home/username/anaconda3/lib/python3.7/site-packages/conda/core/subdir_data.py", line 133 in <lambda>
File "/home/username/anaconda3/lib/python3.7/concurrent/futures/thread.py", line 57 in run
File "/home/username/anaconda3/lib/python3.7/concurrent/futures/thread.py", line 80 in _worker
File "/home/username/anaconda3/lib/python3.7/threading.py", line 870 in run
File "/home/username/anaconda3/lib/python3.7/threading.py", line 926 in _bootstrap_inner
File "/home/username/anaconda3/lib/python3.7/threading.py", line 890 in _bootstrap
Thread 0x00007fb5ecffd700 (most recent call first):
File "/home/username/anaconda3/lib/python3.7/concurrent/futures/thread.py", line 78 in _worker
File "/home/username/anaconda3/lib/python3.7/threading.py", line 870 in run
File "/home/username/anaconda3/lib/python3.7/threading.py", line 926 in _bootstrap_inner
File "/home/username/anaconda3/lib/python3.7/threading.py", line 890 in _bootstrap
Thread 0x00007fb5ed7fe700 (most recent call first):
File "/home/username/anaconda3/lib/python3.7/concurrent/futures/thread.py", line 78 in _worker
File "/home/username/anaconda3/lib/python3.7/threading.py", line 870 in run
File "/home/username/anaconda3/lib/python3.7/threading.py", line 926 in _bootstrap_inner
File "/home/username/anaconda3/lib/python3.7/threading.py", line 890 in _bootstrap
Thread 0x00007fb5edfff700 (most recent call first):
File "/home/username/anaconda3/lib/python3.7/concurrent/futures/thread.py", line 78 in _worker
File "/home/username/anaconda3/lib/python3.7/threading.py", line 870 in run
File "/home/username/anaconda3/lib/python3.7/threading.py", line 926 in _bootstrap_inner
File "/home/username/anaconda3/lib/python3.7/threading.py", line 890 in _bootstrap
Thread 0x00007fb5f8ba1700 (most recent call first):
File "/home/username/anaconda3/lib/python3.7/concurrent/futures/thread.py", line 78 in _worker
File "/home/username/anaconda3/lib/python3.7/threading.py", line 870 in run
File "/home/username/anaconda3/lib/python3.7/threading.py", line 926 in _bootstrap_inner
File "/home/username/anaconda3/lib/python3.7/threading.py", line 890 in _bootstrap
Thread 0x00007fb5fb461700 (most recent call first):
File "/home/username/anaconda3/lib/python3.7/site-packages/conda/common/io.py", line 418 in _start_spinning
File "/home/username/anaconda3/lib/python3.7/threading.py", line 870 in run
File "/home/username/anaconda3/lib/python3.7/threading.py", line 926 in _bootstrap_inner
File "/home/username/anaconda3/lib/python3.7/threading.py", line 890 in _bootstrap
Thread 0x00007fb5fe695740 (most recent call first):
File "/home/username/anaconda3/lib/python3.7/threading.py", line 296 in wait
File "/home/username/anaconda3/lib/python3.7/concurrent/futures/_base.py", line 430 in result
File "/home/username/anaconda3/lib/python3.7/concurrent/futures/_base.py", line 598 in result_iterator
File "/home/username/anaconda3/lib/python3.7/site-packages/conda/core/subdir_data.py", line 140 in query_all
File "/home/username/anaconda3/lib/python3.7/site-packages/conda/core/index.py", line 289 in get_reduced_index
File "/home/username/anaconda3/lib/python3.7/site-packages/conda/core/solve.py", line 1059 in _prepare
File "/home/username/anaconda3/lib/python3.7/site-packages/conda/core/solve.py", line 463 in _collect_all_metadata
File "/home/username/anaconda3/lib/python3.7/site-packages/conda/common/io.py", line 88 in decorated
File "/home/username/anaconda3/lib/python3.7/site-packages/conda/core/solve.py", line 300 in solve_final_state
File "/home/username/anaconda3/lib/python3.7/site-packages/conda/core/solve.py", line 196 in solve_for_diff
File "/home/username/anaconda3/lib/python3.7/site-packages/conda/core/solve.py", line 155 in solve_for_transaction
File "/home/username/anaconda3/lib/python3.7/site-packages/conda_env/installers/conda.py", line 51 in install
File "/home/username/anaconda3/lib/python3.7/site-packages/conda_env/cli/main_create.py", line 142 in execute
File "/home/username/anaconda3/lib/python3.7/site-packages/conda_env/cli/main.py", line 80 in do_call
File "/home/username/anaconda3/lib/python3.7/site-packages/conda/exceptions.py", line 1082 in __call__
File "/home/username/anaconda3/lib/python3.7/site-packages/conda/exceptions.py", line 1374 in conda_exception_handler
File "/home/username/anaconda3/lib/python3.7/site-packages/conda_env/cli/main.py", line 91 in main
File "/home/username/anaconda3/bin/conda-env", line 7 in <module>
# next tried exporting with nobuilds, but basically ran into same cycle
conda env export -f llm_022624_nobuild.yml --no-builds
# change name of env in yaml file
conda env create -f llm_022624_nobuild.yml
error message:
Collecting package metadata (repodata.json): | (llm_110623)
# I also tried creating a clone of the env
conda create --name llm_022624 --clone llm_110623
# activate cloned env
conda activate llm_022624
# list installed packages
conda list
# error
EnvironmentLocationNotFound: Not a conda environment: /home/username/anaconda3/envs/llm_022624
# it doesn't show up when I run conda env list
# I tried updating one of the packages and I get this error
conda update transformers
error:
DirectoryNotACondaEnvironmentError: The target directory exists, but it is not a conda environment.
Use 'conda create' to convert the directory to a conda environment.
target directory: /home/username/anaconda3/envs/llm_022624
# yaml file
name: llm_022624
channels:
- xformers
- pytorch
- conda-forge
- conda-forge/label/cf202003
- nvidia
- anaconda
- defaults
dependencies:
- _libgcc_mutex=0.1
- _openmp_mutex=5.1
- abseil-cpp=20211102.0
- aiohttp=3.8.5
- aiosignal=1.2.0
- appdirs=1.4.4
- arrow-cpp=11.0.0
- asttokens=2.0.5
- async-timeout=4.0.2
- attrs=23.1.0
- aws-c-common=0.6.8
- aws-c-event-stream=0.1.6
- aws-checksums=0.1.11
- aws-sdk-cpp=1.8.185
- backcall=0.2.0
- blas=1.0
- boost-cpp=1.73.0
- bottleneck=1.3.5
- brotlipy=0.7.0
- bzip2=1.0.8
- c-ares=1.19.0
- ca-certificates=2023.12.12
- certifi=2024.2.2
- cffi=1.15.1
- chardet=4.0.0
- charset-normalizer=2.0.4
- click=8.1.7
- colorama=0.4.6
- comm=0.1.2
- cryptography=41.0.2
- cuda-cudart=11.7.99
- cuda-cupti=11.7.101
- cuda-libraries=11.7.1
- cuda-nvrtc=11.7.99
- cuda-nvtx=11.7.91
- cuda-runtime=11.7.1
- cudatoolkit=10.1.243
- cudnn=7.6.5
- datasets=2.12.0
- debugpy=1.5.1
- decorator=5.1.1
- dill=0.3.6
- einops=0.2.0
- et_xmlfile=1.1.0
- executing=0.8.3
- ffmpeg=4.2.2
- filelock=3.9.0
- freetype=2.12.1
- frozenlist=1.3.3
- fsspec=2023.9.2
- gflags=2.2.2
- giflib=5.2.1
- glog=0.5.0
- gmp=6.2.1
- gmpy2=2.1.2
- gnutls=3.6.15
- grpc-cpp=1.48.2
- huggingface_hub=0.15.1
- icu=58.2
- idna=3.4
- intel-openmp=2023.1.0
- ipykernel=6.19.2
- ipython=8.12.0
- jedi=0.18.1
- jinja2=3.1.2
- joblib=1.2.0
- jpeg=9e
- jupyter_client=8.1.0
- jupyter_core=5.3.0
- krb5=1.20.1
- lame=3.100
- lcms2=2.12
- ld_impl_linux-64=2.38
- lerc=3.0
- libboost=1.73.0
- libbrotlicommon=1.0.9
- libbrotlidec=1.0.9
- libbrotlienc=1.0.9
- libcublas=11.10.3.66
- libcufft=10.7.2.124
- libcufile=1.7.0.149
- libcurand=10.3.3.53
- libcurl=8.1.1
- libcusolver=11.4.0.1
- libcusparse=11.7.4.91
- libdeflate=1.17
- libedit=3.1.20221030
- libev=4.33
- libevent=2.1.12
- libffi=3.4.4
- libgcc-ng=11.2.0
- libgfortran-ng=11.2.0
- libgfortran5=11.2.0
- libgomp=11.2.0
- libidn2=2.3.4
- libnghttp2=1.52.0
- libnpp=11.7.4.75
- libnvjpeg=11.8.0.2
- libopus=1.3.1
- libpng=1.6.39
- libprotobuf=3.20.3
- libsodium=1.0.18
- libssh2=1.10.0
- libstdcxx-ng=11.2.0
- libtasn1=4.19.0
- libthrift=0.15.0
- libtiff=4.5.0
- libunistring=0.9.10
- libuuid=1.41.5
- libvpx=1.7.0
- libwebp=1.2.4
- libwebp-base=1.2.4
- lz4-c=1.9.4
- markupsafe=2.1.1
- matplotlib-inline=0.1.6
- mkl=2023.1.0
- mkl-service=2.4.0
- mkl_fft=1.3.6
- mkl_random=1.2.2
- mpc=1.1.0
- mpfr=4.0.2
- mpmath=1.3.0
- multidict=6.0.2
- multiprocess=0.70.14
- ncurses=6.4
- nettle=3.7.3
- networkx=2.8.4
- numexpr=2.8.4
- numpy=1.25.0
- numpy-base=1.25.0
- openh264=2.1.1
- openpyxl=3.0.10
- openssl=3.0.13
- orc=1.7.4
- packaging=23.0
- pandas=2.1.1
- pandas-stubs=1.5.3.230203
- parso=0.8.3
- pexpect=4.8.0
- pickleshare=0.7.5
- pillow=9.4.0
- pip=23.2.1
- platformdirs=2.5.2
- pooch=1.4.0
- prompt-toolkit=3.0.36
- psutil=5.9.0
- ptyprocess=0.7.0
- pure_eval=0.2.2
- pyarrow=11.0.0
- pycparser=2.21
- pygments=2.15.1
- pyopenssl=23.2.0
- pysocks=1.7.1
- python=3.10.12
- python-dateutil=2.8.2
- python-tzdata=2023.3
- python-xxhash=2.0.2
- python_abi=3.10
- pytorch=2.0.0
- pytorch-cuda=11.7
- pytorch-mutex=1.0
- pytz=2023.3.post1
- pyyaml=6.0
- pyzmq=25.1.0
- re2=2022.04.01
- readline=8.2
- regex=2023.10.3
- requests=2.31.0
- responses=0.13.3
- sacremoses=0.0.43
- scikit-learn=1.0.1
- scipy=1.10.1
- setuptools=68.0.0
- six=1.16.0
- snappy=1.1.9
- sqlite=3.41.2
- stack_data=0.2.0
- sympy=1.11.1
- tbb=2021.8.0
- threadpoolctl=2.2.0
- tk=8.6.12
- tokenizers=0.13.3
- torchaudio=2.0.0
- torchtriton=2.0.0
- torchvision=0.15.0
- tornado=6.2
- tqdm=4.65.0
- traitlets=5.7.1
- transformers=4.29.2
- types-pytz=2022.4.0.0
- typing_extensions=4.7.1
- tzdata=2023c
- urllib3=1.26.16
- utf8proc=2.6.1
- wcwidth=0.2.5
- wheel=0.38.4
- x264=1!157.20191217
- xformers=0.0.19
- xxhash=0.8.0
- xz=5.4.2
- yaml=0.2.5
- yarl=1.8.1
- zeromq=4.3.4
- zlib=1.2.13
- zstd=1.5.5
- pip:
- absl-py==2.0.0
- accelerate==0.24.1
- aiostream==0.5.2
- annotated-types==0.6.0
- anyio==3.7.1
- asgiref==3.7.2
- backoff==2.2.1
- bcrypt==4.1.2
- bitsandbytes==0.42.0
- build==1.0.3
- cachetools==5.3.2
- chroma-hnswlib==0.7.3
- chromadb==0.4.22
- coloredlogs==15.0.1
- dataclasses-json==0.5.14
- deprecated==1.2.14
- distro==1.8.0
- evaluate==0.4.0
- exceptiongroup==1.1.3
- fastapi==0.109.2
- flatbuffers==23.5.26
- google-auth==2.27.0
- googleapis-common-protos==1.62.0
- greenlet==3.0.1
- grpcio==1.60.1
- h11==0.14.0
- httpcore==1.0.1
- httptools==0.6.1
- httpx==0.25.1
- humanfriendly==10.0
- importlib-metadata==6.11.0
- importlib-resources==6.1.1
- jsonpatch==1.33
- jsonpointer==2.4
- kubernetes==29.0.0
- langchain==0.0.150
- langsmith==0.0.60
- llama-index==0.8.64.post1
- marshmallow==3.20.1
- mmh3==4.1.0
- monotonic==1.6
- mypy-extensions==1.0.0
- nest-asyncio==1.5.8
- nltk==3.8.1
- oauthlib==3.2.2
- onnxruntime==1.17.0
- openai==1.2.1
- openapi-schema-pydantic==1.2.4
- opentelemetry-api==1.22.0
- opentelemetry-exporter-otlp-proto-common==1.22.0
- opentelemetry-exporter-otlp-proto-grpc==1.22.0
- opentelemetry-instrumentation==0.43b0
- opentelemetry-instrumentation-asgi==0.43b0
- opentelemetry-instrumentation-fastapi==0.43b0
- opentelemetry-proto==1.22.0
- opentelemetry-sdk==1.22.0
- opentelemetry-semantic-conventions==0.43b0
- opentelemetry-util-http==0.43b0
- overrides==7.7.0
- peft==0.3.0
- posthog==3.4.0
- protobuf==4.25.2
- pulsar-client==3.4.0
- pyasn1==0.5.1
- pyasn1-modules==0.3.0
- pydantic==1.10.13
- pydantic-core==2.10.1
- pypika==0.48.9
- pyproject-hooks==1.0.0
- python-dotenv==1.0.1
- requests-oauthlib==1.3.1
- rouge-score==0.1.2
- rsa==4.9
- sentencepiece==0.1.96
- sniffio==1.3.0
- sqlalchemy==2.0.23
- starlette==0.36.3
- tenacity==8.2.3
- tiktoken==0.5.1
- tomli==2.0.1
- trl==0.4.2.dev0
- typer==0.9.0
- typing-extensions==4.9.0
- typing-inspect==0.9.0
- uvicorn==0.27.0.post1
- uvloop==0.19.0
- watchfiles==0.21.0
- websocket-client==1.7.0
- websockets==12.0
- wrapt==1.15.0
- zipp==3.17.0
prefix: /home/username/anaconda3/envs/llm_022624