loop over subdirectories and apply command

Question

I have many sub directories, each one of them contains .txt files and a .fastq file. I want to do something on each txt file and their corresponding file in each sub directory

#!/bin/bash
dir=$1

subdirectories=$(find $dir -type d) # find only subdirectories in dir

for subdir in $subdirectories;do
    cd $subdir
    p_files=$(find . -name '*v1.txt') # find all the txt files in that subdirectory
    
    fq_file=$(find . -name '*v1.fastq') # find all the fastq files in that subdirectory
    
    for f in $p_files;do
        for q in $fq_file;
        do
            grep -A3 -f "$f" "$q" > "${f%.txt}_ex.fastq"
        done
    done
done

I want to grep on each single txt file in each directory with its fastq counterpart and output a new file.

Answer 1

Suppose you have these files (tree is an installed program...):

$ tree .
.
├── dir_A
│   ├── file.fastq
│   └── file.txt
├── dir_B
│   ├── file.fastq
│   └── file.txt
├── dir_C
│   ├── file.fastq
│   └── file.txt
└── dir_D
    ├── file.fastq
    └── file.txt

Rather than using find a bunch of times you can do something like this with a recursive glob:

for fq in **/*.fastq; do
    ft="${fq%.fastq}.txt"
    echo "$fq => $ft"
done

In this micro example, prints:

dir_A/file.fastq => dir_A/file.txt
dir_B/file.fastq => dir_B/file.txt
dir_C/file.fastq => dir_C/file.txt
dir_D/file.fastq => dir_D/file.txt

As you go through the loop now, $fq and $ft are set to the path name (relative to where the loop started) and you should be able to do what you want.

For Bash, make sure that shopt globstar is set to on with shopt -s globstar.