Install Python package inside a Jupyter Notebook kernel

Question

Inside a Jupyter Notebook I have managed to install a Python kernel with

!python -m ipykernel install --user --name other-env --display-name "Python (other-env)"

as informed here and it is available with other kernels on the menu Kernel → Change kernel and

!jupyter kernelspec list

will also show them

Available kernels:
  avenv         C:\Users\Full Name\AppData\Roaming\jupyter\kernels\avenv
  chatterbot    C:\Users\Full Name\AppData\Roaming\jupyter\kernels\chatterbot
  othervenv     C:\Users\Full Name\AppData\Roaming\jupyter\kernels\othervenv
  python3       C:\Users\Full Name\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\share\jupyter\kernels\python3

Then I try to install Python a package using

%pip install a_package

as given here that said

with % (instead of !) it will install a_package into the current kernel (rather than into the instance of Python that launched the notebook).

But what I have got that it installs a_package to all kernels, or %pip list will list the same installed packages in all kernels.

Is there a way to install Python package only to an active Jupyter Notebook kernel?

Answer 1

Short answer

Path of python executable in JSON file of Jupyter kernel should point to python executable in related virtual environment.

Long answer

After some tries the steps can be summarized as follow.

Python virtual environment

1. Create Python virtual environment from a shell, e.g. PowerShell, as in here, in a folder (in this case it is c:\venvs\butiran)

PS C:\> python -m venv c:\venvs\butiran

2. Change directory to the folder, e.g in this case is butiran in venvs (folder for all virtual environments)

PS C:\> cd venvs\butiran
PS C:\venvs\butiran>

3. Activate virtual environment as in here

PS C:\venvs\butiran> Scripts\activate
(butiran) PS C:\venvs\butiran>

4. Update pip as in here

(butiran) PS C:\venvs\butiran> python -m pip install --upgrade pip

5. Install ipykernel as in here

(butiran) PS C:\venvs\butiran> pip install ipykernel

6. Deactivate virtual enviroment in powershell as in here

(butiran) PS C:\venvs\butiran> deactivate
PS C:\venvs\butiran>

Jupyter kernel

1. Install ipykernel for Jupyter Notebook as in here

PS C:\venvs\butiran> python -m ipykernel install --user --name butiran --display-name "Python (butiran)"
Installed kernelspec butiran in C:\Users\Full Name\AppData\Roaming\jupyter\kernels\butiran
PS C:\venvs\butiran>

2. Show JSON file of installed kernel

PS C:\venvs\butiran> cat "C:\Users\Full Name\AppData\Roaming\jupyter\kernels\butiran\kernel.json"
{
 "argv": [
  "C:\\Users\\Full Name\\AppData\\Local\\Microsoft\\WindowsApps\\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\\python.exe",
  "-m",
  "ipykernel_launcher",
  "-f",
  "{connection_file}"
 ],
 "display_name": "Python (butiran)",
 "language": "python",
 "metadata": {
  "debugger": true
 }
}
PS C:\venvs\butiran>

3. Change path to python.exe

• from C:\\Users\\Full Name\\AppData\\Local\\Microsoft\\WindowsApps\\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\\python.exe
• to C:\\venvs\\butiran\\Scripts\\python.exe

Jupyter Notebook

1. Lanuch Jupyter Notebook from PowerShell

PS C:\> jupyter notebook

2. Open existing notebook or create a new one.

3. Change kernel from menu Kernel → Change kernel → Python (butiran).

4. Install a_package using only in active kernel

%pip install a_package

or with

import sys
!{sys.executable} -m pip install a_package

5. Show the package globally

!pip show a_package

will give

WARNING: Package(s) not found: a_package

while show the package in active kernel

%pip show a_package

will give

Name: a_package
Version: ..
Summary: ..
..
Location: c:\venvs\butiran\lib\site-packages

Answer 2

Here is what I have used to overcome this issue

import sys
!{sys.executable} -m pip install a_package

Answer 3

In ubuntu, you can try this file path and files:

conda environment name: rag

vi ~/.local/share/jupyter/kernels/rag/kernel.json

which python

/home/user/miniconda3/envs/rag/bin/python

change the python binary path to the one from which python in the active conda environment


vi ~/.local/share/jupyter/kernels/rag/kernel.json


{
 "argv": [
  "/home/user/miniconda3/envs/rag/bin/python",
  "-m",
  "ipykernel_launcher",
  "-f",
  "{connection_file}"
 ],
 "display_name": "rag",
 "language": "python",
 "metadata": {
  "debugger": true
 }
}
~