Run scp in a bash script to copy multiple directory from remote

Question

I would like to write a bash script for transferring multiple directories from a Raspberry Pi to my laptop using scp command. But my code only work on one laptop (MacBook Pro 2016), but it failed on another laptop (MacBook Pro 2020), which is quite strange.

What I would like to execute is some command like:

scp -rp [email protected]:/home/pi/Documents/data/{directory1,directory2,directory3} .

This command work well when I directly execute it in terminal, on both of the laptop. But when I put it into a bash script:

#!/bin/bash
scp -rp "[email protected]:/home/pi/Documents/data/{directory1,directory2,directory3}" "."

it shows error scp: /home/pi/Documents/data/{directory1,directory2,directory3}: No such file or directory on my MacBook Pro 2020, but it works well on my MacBook Pro 2016.

I have checked the bash version on both laptop, but it seems like they don't have too much difference

MacBook Pro 2016: GNU bash, version 3.2.57(1)-release (x86_64-apple-darwin21)
MacBook Pro 2020: GNU bash, version 3.2.57(1)-release (x86_64-apple-darwin22)

Can anyone help me?

-------------------------------- Below are updates --------------------------------

Seem like I didn't explain my problem very well.

My actual situation is a litter more complicated. I am working on a bash script which takes multiple directory as parameter for doing scp from my Raspberry Pi remote. My script.sh is like:

#!/bin/bash
while getopts "hu:" option; do
   case $option in
      h) # display Help
         Help
         exit;;
      u)
         user_list+=("$OPTARG");;
     \?) # incorrect option
         echo "Error: Invalid option"
         exit;;
   esac
done

RASPBERRYPI_DATA_PATH="/home/pi/Documents/data"
RASPBERRYPI_ADDR=$1
RASPBERRYPI_USER_NAME="pi"

scp -rp ${RASPBERRYPI_USER_NAME}@${RASPBERRYPI_ADDR}:${RASPBERRYPI_DATA_PATH}/{${user_list[@]// /,}} .

I execute the script in this way:

./script.sh -u "directory1 directory2" 192.168.0.163

It shows error: scp: /home/pi/Documents/data/{directory1,directory2}: No such file or directory on my MacBook Pro 2020, but it works well on my MacBook Pro 2016.

I am a novice in bash development. I think I am having some mistake on grammar. But the script works well on my old MacBook Pro, which is quite strange.

Thank you very much for helping me.

Answer 1

This is due to the double quotes in your script. Remove them and it should work again.

Test with something like this:

cd /tmp
mkdir -p data/{directory1,directory2,directory3}
mkdir -p "data/{directory1,directory2,directory3}"
ls -alrt /tmp/data

output:

drwxr-xr-x   2 tscanlan  wheel   64 Nov 12 08:54 directory1
drwxr-xr-x   2 tscanlan  wheel   64 Nov 12 08:54 directory2
drwxr-xr-x   2 tscanlan  wheel   64 Nov 12 08:54 directory3
drwxr-xr-x   2 tscanlan  wheel   64 Nov 12 08:54 {directory1,directory2,directory3}

Why? Brace expansion in bash and zsh just work that way.

• https://www.gnu.org/software/bash/manual/html_node/Brace-Expansion.html
• https://zsh.sourceforge.io/Doc/Release/Expansion.html#Brace-Expansion

A correctly-formed brace expansion must contain unquoted opening and closing braces, and at least one unquoted comma or a valid sequence expression. Any incorrectly formed brace expansion is left unchanged.

Answer 2

I would use rsync for this. Rsync is especially useful as it does incremental backup saving you lots of time and bandwidth. You can construct the input directory list from user input and invoke rsync as under

rsync -a /path/to/dir1 /path/to/dir2 user@remoteserver:/path/to/target

What this will do is create the following directories on the target

/path/to/target/dir1
/path/to/target/dir2