ipykernel_launcher processes are consuming memory, Not able to kill

Question

What are these zombie ipykernel_launcher process in my machine, which are hogging to much memory:

This is output of htop command, but I ps for those processes,(to kill them) I do not see them as:

ps -ef|grep ipykernel

Not sure, how to get rid of these memory hogs!

Answer 1

The reason why you're seeing all these processes in htop and not with ps is that htop is showing threads (see https://serverfault.com/questions/24198/why-does-htop-show-lots-of-apache2-processes-by-ps-aux-doesnt). Type "-H" inside htop to toggle showing threads.

Automatically stop idle kernels

Concerning Jupyter notebook processes in general: kernels are small computational engines and consume a lot of resources (mainly memory) even when they're not active. This is why one should encourage users to stop running kernels when they're not in use. The problem is that even if one closes a tab or the whole browser, the kernel keeps running, so one forgets about the kernels!

Since it's unlikely that users will shutdown their kernels, consider stopping idle kernels by configuring the parameter NotebookApp.shutdown_no_activity_timeoutInt in your Jupyter configuration file jupyter_notebook_config.py.

NotebookApp.shutdown_no_activity_timeoutInt.
Default: 0

Shut down the server after N seconds with no kernels or terminals running and no activity. This can be used together with culling idle kernels (MappingKernelManager.cull_idle_timeout) to shutdown the notebook server when it’s not in use. This is not precisely timed: it may shut down up to a minute later. 0 (the default) disables this automatic shutdown.

See also these properties:

# shutdown the server after no activity for an hour
c.ServerApp.shutdown_no_activity_timeout = 60 * 60
# shutdown kernels after no activity for 20 minutes
c.MappingKernelManager.cull_idle_timeout = 20 * 60
# check for idle kernels every two minutes
c.MappingKernelManager.cull_interval = 2 * 60

If that doesn't work, you may need to run a cron job to kill the ipykernel processes with kill after a certain amount of elapsed time (see for instance https://unix.stackexchange.com/questions/531040/list-processes-that-have-been-running-more-than-2-hours).

One-time solution

A quick solution to solve the problem is to restart the Jupyter notebook/Jupyter Hub. This will stop all kernels.