Why does Snakemake redirect output to a non-existing folder?

Question

I have a rule in Snakemake which shell section translates to something like that:

echo "Hello" > output_folder/output_file; cat input_file >> output_folder/output_file

Note that output_folder is a child folder for the one where Snakemake runs.

That worked well, but I've decided to use another folder for output, and that folder is not a child of the current one. I have to provide an absolute path. Now my pipeline fails:

/usr/bin/bash: line 1 /absolute/path/output_file: No such file or directory
/usr/bin/bash: line 1 /absolute/path/output_file: No such file or directory
...
        (one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)

Indeed, after running the pipeline with the --printshellcmds flag and repeating the same sequence of commands in bash it produces the same error.

So now I'm confused: why does it work for the child directory? I've repeated the sequence of commands for the child directory and bash produces the error (which is expected), while Snakemake successfully redirects output to the file in a non-existing folder.

Update: removing the cat ... >> part doesn't help and the behavior is the same. I've tried to replace absolute path with the relative one. Now I'm getting another error:

WorkflowError:
Error recording metadata for finished job ([Error 2] No such file or directory: '<current dir>\\.snakemake\\metadata\\Li4vLi4vLi4vZmxhc2hiYWNrYm9uZV93YXRjaC9kb2N0cmluZXMvQ2Fycm9sbC9iYWNrYXBwX3JlZmFjdG9yL1NwYW5pc2gvaW5zZXJ0X2ludG9fcGFyYWRpZ20udmVyYi50eXBlLmhpZXJhcmNoeV9zZWxlY3RfcGFyYWRpZ20udmVyYi50eXBlLmhpZXJhcmNoeXRtcC5zcWw='). Please ensure write permissions for the directory <current dir>\.snakemake')

I'm running it on Windows under MSYS2 environment.

Answer 1

I realize this is a hack that does not solve the issue, but perhaps it will help: setup a symlink to the absolute path of interest, so that the symlink is in the child folder. Not ideal, but should work.