I'm using Snakemake and tried to order my overall pipeline structure by utilizing modules as the modularization approach suggests. However, one of my modules has a rule which depends on the ouput of the other module and since both modules have their own namespace, the outputs and inputs are not connected (atleast so it seems to me). I tried a lot of different things, but did not find a solution yet.
So, my question in clear: Is there in Snakemake a way to utilize modules, whereby the respective rules can (dynamically) rely on the output of another module's rule?
Here's a minimal code to illustrate my situation:
Snakemake main
# module1
module module1:
snakefile: "modules/module1/Snakefile"
prefix: "output/module1"
use rule * from module1 as module1_*
# module2
module module2:
snakefile: "modules/module2/Snakefile"
prefix: "output/module2"
use rule * from module2 as module2_*
# ======== Main rule ===========
rule all:
input:
rules.module1_all.input,
rules.module2_all.input
default_target: True # Makes this rule the default rule
Module 1
rule create_txt:
output:
"output/test.txt"
shell:
"touch {output}"
rule all:
input:
"output/test.txt"
Module 2
rule create_txt2:
input:
rules.module1_create_txt.output
output:
"output/test2.txt"
shell:
"touch {output}"
rule all:
input:
"output/test2.txt"
If I then run the pipeline, this is the output (I dropped the following exceptions, since I don't think this would help...)
(snakemake) Olivers-MacBook-Pro-5:test_snakemake oliverkuchler$ snakemake -nr
Building DAG of jobs...
Traceback (most recent call last):
File "/Users/oliverkuchler/opt/miniconda3/envs/snakemake/lib/python3.10/site-packages/snakemake/dag.py", line 1808, in collect_potential_dependencies
yield PotentialDependency(file, known_producers[file], True)
KeyError: 'output/module2/output/module1/output/test.txt'
So, as you can see, Snakemake accepts referencing rules from the other module. However, this leads to the application of multiple prefixes to the input-file 'output/module2/output/module1/output/test.txt'. But even if I would solve this issue, still module2 would not be able to relate to rules from module1. Are there any solutions, that I do not see? Happy about any suggestions :)