With argparse it it possible to process arguments only up to the last non-positional argument?

Question

I'm writing a tool which passes arguments to another command provided with the arguments like this:

foo --arg1 -b -c bar -v --number=42

In this example foo is my tool and --arg1 -b -c should be the arguments parsed by foo, while -v --number=42 are the arguments going to bar, which is the command called by foo.

So this is quite similar to strace where you can provide arguments to strace while still providing a command with custom arguments.

argparse provides parse_known_arguments() but it will parse all arguments it knows even those coming after bar.

Some tools use a special syntax element (e.g. --) to separate arguments with different semantics, but since I know foo will only process names arguments, I'd like to avoid this.

That can't be too hard to manually find the first argument you might think, and this is what I'm currently doing:

parser.add_argument("--verbose", "-v", action="store_true")
all_args = args or sys.argv[1:]
with suppress(StopIteration):
    split_at = next(i for i, e in enumerate(all_args) if not e.startswith("-" ))
    return parser.parse_args(all_args[:split_at]), all_args[split_at:]
raise RuntimeError("No command provided")

And this works with the example I've provided. But with argparse you can specify arguments with values which can but don't have to be provided with a =:

foo --file=data1 --file data2 bar -v --number=42

So here it would be much harder to manually identify data2 to be a value for the second --file argument, and bar to be the first positional argument.

My current approach is to manually split arguments (backwards) and see, if all 'left hand' arguments successfully parse:

def parse_args():
    class MyArgumentParser(ArgumentParser):
        def error(self, message: str) -> NoReturn:
            raise RuntimeError()

    parser = MyArgumentParser()
    parser.add_argument("--verbose", "-v", action="store_true")
    parser.add_argument("--with-value", type=str)

    all_args = sys.argv[1:]
    for split_at in (i for i, e in reversed(list(enumerate(all_args))) if not e.startswith("-")):
        with suppress(RuntimeError):
            return parser.parse_args(all_args[:split_at]), all_args[split_at:]

    parser.print_help(sys.stderr)
    print("No command provided", file=sys.stderr)
    raise SystemExit(-1)

That works for me, but next to the clumsy extra MyArgumentParser needed just to be able to manually handle parser errors I now need to manually classify mistakes, since an ArgumentError turns into something that occurs naturally.

So is there a way to tell argparse to parse only until the first positional argument and then stop even if there are arguments it knows after that one?

Answer 1

Define a parser:

In [3]: parser = argparse.ArgumentParser()
   ...: parser.add_argument('--file', action='append')
   ...: parser.add_argument('-f')
   ...: parser.add_argument('rest', nargs=argparse.PARSER);

The only thing special is this 'rest' with an undocumented nargs (same used by subparsers. It's like '+', but values, other than the first, can look like flags.

In [4]: parser.print_help()
usage: ipykernel_launcher.py [-h] [--file FILE] [-f F] rest ...

positional arguments:
  rest

options:
  -h, --help   show this help message and exit
  --file FILE
  -f F

And testing on your of inputs (I didn't account for the foo argument):

In [5]: args = parser.parse_args('--file=data1 --file data2 -f1 bar -v --number=42 --file=data3'.split())

In [6]: args
Out[6]: Namespace(file=['data1', 'data2'], f='1', rest=['bar', '-v', '--number=42', '--file=data3'])

Omit the positional, and we get an error:

In [7]: args = parser.parse_args('--file=data1 --file data2 -f1 -v --number=42 --file=data3'.split())
usage: ipykernel_launcher.py [-h] [--file FILE] [-f F] rest ...
ipykernel_launcher.py: error: the following arguments are required: rest