Amazon Sagemaker Run Code from jupyter notebook in background

Question

I am running code on an Amazon Sagemkaer notebook instance (in a plain jupyter notebook, nnot jupyterLab).

1. How can I run the code in the background and close the browser tab? When I close the jupyter notebook tab the program stops, I want to avoid this. I read that I should not be doing the processing in the notebook itslef, rather I should use a Sagemaker processing job. How can I run a simple cell of code like the one below on a higher i

df_new['predicted_values'] = df_original.progress_apply(lambda x: LLM_pretrained_model.predict( x['comment_body'] )

2. After 12 hours the kernal crashes saying that I need to login again. How can I avoid this? due to the size of my data the program will take at least 28 hours to run

3. Is it possible to push code to GitHub from a sagemaker jupyter notebook (not a jupyterLab notebook)?

Answer 1

You can convert your notebook to a python script and run it from the terminal.

Alternatively SageMaker offers specific components such as Training Jobs/ Processing Jobs that can run the jobs on dedicated hosted instances.

On the Github questions, yes you can push your code to GitHub using the git terminal commands in the Jupyter terminal