Reading a file on a remote host

Question

My code involves going to a host(doing with openSSH), getting the list of files matching a pattern(doing using remote find command-OpenSSH) and then opening each file I have got and processing each(grepping etc). I am done with getting the file names and passing to a function. Now, I have to pass these filename to a function where I open each and process it. I am trying to do it using File::Remote as follows:

sub processFiles{
    my $fileList =shift;
#Iterate over the file and try to find start and stop time stamps from the file
for my $file ( @{$fileList}) {
#finding start time of file:its found in lines of file
my $HEAD;
open $HEAD, "host:head -1000 $File|" or die "Unable to check file for starting time";
 while ( <$HEAD> ) {
 #process...

But I am unable to open the file on the host an I am getting an error.

Answer 1

When performing remote actions in batch, the first principle is to reduce the number of ssh connections (ideally only one). Using shell or perl one liner, you can do very complex actions. Here is my understanding of your need:

ssh hostname 'find dirname -name \*.pl| xargs grep -l pattern /dev/null'

You can pipe further processing in the same line. If you want to process the files locally, you can transfer all the files in a single command. Here is a full example:

use Archive::Tar;

open my $file, '-|', 'ssh -C hostname "find dirname -name \*.pl | xargs grep -l pattern /dev/null | xargs tar c"'
    or die "Can't pipe: $!";

my $tar = Archive::Tar->new($file);
foreach my $file ($tar->get_files()) {
    print $file->name, ', ', $file->size, "\n";
}

Answer 2

The File::Remote module is not part of a standard Perl installation. If you want to use this module, then you need to install it.

How you install it will depend on what platform you're using and how you have installed Perl. On a Linux installation where you are using the system Perl you could try sudo yum install perl-File-Remote (on a RedHat-based system) or sudo apt-get install libfile-remote-perl (on a Debian/Ubuntu-based system). You could also try using cpan or cpanm to install it.

I'd also suggest that as this module was last updated in 2005, it's quite likely that it is unmaintained, so I would be wary of using it.