a shell script to do 'cdo' operation on multiple Netcdf file embedded in multiple folders

48 views Asked by At

The problem I am trying to solve is that the loop I wrote that to read All NetCDF files nested in multiple folders is not able to process CDO operation as I expect doing for individual files. Below is a brief description of what I am looking for;

I have a directory on my Linux machine, let's name it MYDIR_SST, with a number of folders labeled based on the years (1982, 1983, 1984, ..., 2022). Whithin each sub-folder, there are other folders named according to the months (01, 02, 03, ..., 12) and several NetCDF daily data inside each of these folders as follow:

MYDIR_SST[directory]
--1982[first nested folder]   
------01[second nested folder]
------02[second nested folder]
------03[second nested folder]
------12[second nested folder]   
--1982[first nested folder]
------01[second nested folder]
------02[second nested folder]
------12[second nested folder]

~/MYDIR_SST/1982/01 19820101120000-UKMO-L4_GHRSST-SSTfnd-OSTIA-GLOB_REP-v02.0-fv02.0.nc  19820102120000-UKMO-L4_GHRSST-SSTfnd-OSTIA-GLOB_REP-v02.0-fv02.0.nc 19820103120000-UKMO-L4_GHRSST-SSTfnd-OSTIA-GLOB_REP-v02.0-fv02.0.nc  19820104120000-UKMO-L4_GHRSST-SSTfnd-OSTIA-GLOB_REP-v02.0-fv02.0.nc

~/MYDIR_SST/1983/01 19830101120000-UKMO-L4_GHRSST-SSTfnd-OSTIA-GLOB_REP-v02.0-fv02.0.nc  19830102120000-UKMO-L4_GHRSST-SSTfnd-OSTIA-GLOB_REP-v02.0-fv02.0.nc 19830103120000-UKMO-L4_GHRSST-SSTfnd-OSTIA-GLOB_REP-v02.0-fv02.0.nc  19830104120000-UKMO-L4_GHRSST-SSTfnd-OSTIA-GLOB_REP-v02.0-fv02.0.nc

I want to perform 2 operations on each file in all the folders. The operations are defined below as I am doing for Netcdf files in folder ~/MYDIR_SST/1982/01.

cdo mergetime *.nc outfile.nc;
cdo ymonmean outfile.nc L4_GHRSST-SSTfnd_1982_01.nc

I want to save FINAL OUTPUTS AS BELOW FORMAT;

L4_GHRSST-SSTfnd_YYYY_MM.nc

And save all L4_GHRSST-SSTfnd_1982_01.nc, L4_GHRSST-SSTfnd_1982_02.nc,...,L4_GHRSST-SSTfnd_2022_01.nc,..., L4_GHRSST-SSTfnd_2022_12.nc in a different folder named monmean in MYDIR_SST (~/MYDIR_SST/monmean). Can someone help me create a shell script which can perform the mentioned process?

pattern should be, L4_GHRSST-SSTfnd_1982_01.nc, L4_GHRSST-SSTfnd_1982_02.nc, etc.

#!/bin/sh
set -xv #debugging
for file in GLO_SST_L4_OBS_CP/*/*/
do
    echo "$file"
done | sed 's/-L4_.*//' | sort -u | while read -r pattern
do
    cdo mergetime "${pattern}"* "${pattern}_mergetime.nc"
done
echo 'JOB DONE'

0

There are 0 answers