pymongo.errors.InvalidURI when connecting to mongodb on github runner

Question

I am attempting to run a python script that connects to a mongo atlas db using a self hosted github runner on amazon linux 2. But when running the script I get the error pymongo.errors.InvalidURI: Invalid URI scheme: URI must begin with 'mongodb://' or 'mongodb+srv://'. Here's a snippet of the step in problem:

      - name: Populate VectorDB
        env:
          OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
          PINECONE_API_KEY: ${{ secrets.PINECONE_API_KEY }}
          PINECONE_INDEX_NAME: ${{ needs.creates-ephemeral-env.outputs.pinecone_index_name }}
          MONGODB_URI: ${{ needs.creates-ephemeral-env.outputs.mongodb_uri }}
          MONGODB_USERNAME: ${{ secrets.MONGODB_USERNAME }}
          MONGODB_PASSWORD: ${{ secrets.MONGODB_PASSWORD }}
          MONGODB_NAME: ${{ env.MONGODB_NAME }}
        run: |
          cd database/scripts/populate_pinecone_db
          python3 -m venv venv
          source venv/bin/activate
          pip3 install -r requirements.txt
          python3 -u script.py

here's my requirements.txt:

pinecone-client==3.1.0
pymongo==4.4.0
langchain-openai==0.0.5
sql-metadata==2.8.0

The uri is valid because I've tried connecting to the database locally as well as when I launched a separate instance on ec2 with the same image I was able to connect to the mongo database with the same uri. I also have used the secrets.MONGODB_USERNAME and secrets.MONGODB_PASSWORD in a previous steps and they worked as well. I am not sure why pymongo is behaving like this.

When I printed the uri from python, I get the following:

"***[connect_uri].mongodb.net"

Here is part of my python code:

import os

import pymongo
from pinecone import Pinecone
from sql_metadata import Parser

pinecone_api_key = os.environ.get("PINECONE_API_KEY")

pinecone_index_name = os.environ.get("PINECONE_INDEX_NAME")
mongodb_uri = os.environ.get("MONGODB_URI")
mongodb_username = os.environ.get("MONGODB_USERNAME")
mongodb_password = os.environ.get("MONGODB_PASSWORD")
mongodb_name = os.environ.get("MONGODB_NAME")

if __name__ == "__main__":
    connection_string = mongodb_uri.replace(
        "mongodb+srv://", f"mongodb+srv://{mongodb_username}:{mongodb_password}@"
    )
    data_store = pymongo.MongoClient(connection_string)[mongodb_name]
    ...

Pymongo should successfully connect to the mongo atlas database since I have configured the right network access as well as the username and password

Answer 1

This is very dumb but I figured it out, the output from needs.creates-ephemeral-env.outputs.mongodb_uri came from terraform and I set the variable as mongodb_uri=$(terraform output appservice_name_dev), which includes the double quotes as part of the string, therefore the connection uri is invalid. I had to change it to $(terraform output -raw appservice_name_dev).