AWS Lambda & Papermill: Unable to import module 'lambda_function': No module named 'rpds.rpds'

Question

I'm trying to import papermill inside an AWS python lambda function. To to this, I first ran pip3 install papermill -t Desktop/python locally, zipped the python file and uploaded this as a lambda layer.

I am now getting this error:

{
  "errorMessage": "Unable to import module 'lambda_function': No module named 'rpds.rpds'",
  "errorType": "Runtime.ImportModuleError",
  "requestId": "7f217dbf-455f-4867-a5d6-8da1f3abd356",
  "stackTrace": []
}

I haven't been able to figure it out nor find any similar issues online. Below is an image of the rpds module available in the 'python' directory, if that's useful? :

I hope someone can help! Thanks.

Answer 1

assuming you have a requirements.txt file and you use python 3.10 you can try to use this script to create a lambda layer with all of yours dependencies:

#!/bin/bash 
echo "create build venv"
rm -rf temp-venv
python3.10 -m venv temp-venv
source temp-venv/bin/activate

rm -rf dist
mkdir -p dist
pip install \
    --platform manylinux2014_x86_64 \
    --target=./dist/layer/python/lib/python3.10/site-packages \
    --implementation cp \
    --python-version 3.10 \
    --only-binary=:all: --upgrade \
    --no-compile \
    -r requirements.txt


cd dist/layer
zip -r ../dep_lambda_layer.zip .
cd ..
rm -rf layer