Python imports fail when invoked by nf-core process

34 Views Asked by At

I created a nf-core project using nf-core create. I am trying to invoke python scripts using nf-core processes. This technique worked in my nextflow project as I was able to avoid having to write code in a process. Most of the imports are failing.

import sys --> works
import pymongo --> fails
from pymongo.mongo_client import MongoClient --> fails

From the terminal I can see the process is trying to invoke:

nf.prefetch_http.py SRR9686066 "$PWD/"SRR9686066".sra" "mongodb://xxxx:[email protected]:10255/?ssl=true&retrywrites=false&replicaSet=globaldb&maxIdleTimeMS=120000&appName=@xxxx@"

This results in error:

  *File "/home/q/nf-core-pass/bin/mongodb_logger.py", line 1, in <module>
     from google.cloud import secretmanager
  ModuleNotFoundError: No module named 'google'*

However, if I run directly:

/home/q/nf-core-xxx/bin/nf.prefetch_http.py SRR9686066 "$PWD/"SRR9686066".sra" "mongodb://xxxx:[email protected]:10255/?ssl=true&retrywrites=false&replicaSet=globaldb&maxIdleTimeMS=120000&appName=@xxxx@"

This works proving the import(s) are installed.

If I move the imports to nf.prefetch_http.py, the failure simply moves to nf.prefetch_http.py.

nf-core-xxx/bin/nf.prefetch_http.py

#!/usr/bin/env python3
import sys
import mongodb_logger
import prefetch_http


sra = sys.argv[1]
sraPath = sys.argv[2]
mongodb_connection_string = sys.argv[3]

print(f"nf.prefetch_http: sra: {sra}")
print(f"nf.prefetch_http: sraPath: {sraPath}")
print(f"nf.prefetch_http: mongodb_connection_string: {mongodb_connection_string}")

mongodb_logger.initialize(None,mongodb_connection_string)
prefetch_http.prefetch(sra, sraPath)

nf-core-xxx/bin/prefetch_http.py

from google.cloud import secretmanager
from pymongo.mongo_client import MongoClient
from pymongo.server_api import ServerApi
import pymongo
import os
import json
import response
import globals
import jsons
import mongodb_logger
import pandas as pd

from dataclasses import dataclass
from typing import List


from typing import Optional, List
from datetime import datetime

import requests
from requests.adapters import HTTPAdapter
from urllib3.util.retry import Retry
import globals

mongodb_connection_string = "NULL" 
...

Is there a configuration in nf-core required to make python work correctly?

0

There are 0 best solutions below