How can I implement an argument parser with a variable number of positional arguments and optional subcommands in Python?

Question

I am trying to implement an argument parser with the argparse module in Python. My goal is to allow a variable number of positional input arguments to the main parser and then optionally call a sub parser to perform some task. I'm interested in understanding why my implementation doesn't work as I expect it to and how I could implement something that is as close as possible to what I originally intended.

Here is a minimal example to illustrate what I am trying. I used Python 3.11 to create this example, although I originally encountered the issue using Python 3.9.

In [1]: import argparse

In [2]: parser = argparse.ArgumentParser()
   ...: parser.add_argument('positional', type=str, nargs='+')
   ...: parser.add_argument('-f', '--foo')
   ...: 
   ...: subparsers = parser.add_subparsers(dest='subcommand', required=False)
   ...: subparser1 = subparsers.add_parser('subcommand1')
   ...: subparser1.add_argument('-b', '--bar', action='store_true', )
   ...: subparser2 = subparsers.add_parser('subcommand2')
   ...: subparser2.add_argument('-b', '--baz', action='store_true')
Out[2]: _StoreTrueAction(option_strings=['-b', '--baz'], dest='baz', nargs=0, const=True, default=False, type=None, choices=None, required=False, help=None, metavar=None)

In [3]: parser.parse_args(['-f', 'a', 'b', 'c', 'd', 'subcommand1'])
Out[3]: Namespace(positional=['b', 'c', 'd'], foo='a', subcommand='subcommand1', bar=False)

In [4]: parser.parse_args(['-f', 'a', 'b', 'c', 'd', 'subcommand2'])
Out[4]: Namespace(positional=['b', 'c', 'd'], foo='a', subcommand='subcommand2', baz=False)

In [5]: parser.parse_args(['-f', 'a', 'b', 'c', 'd'])
usage: ipython [-h] [-f FOO] positional [positional ...] {subcommand1,subcommand2} ...
ipython: error: argument subcommand: invalid choice: 'd' (choose from 'subcommand1', 'subcommand2')

I expected (hoped) that, since required=False in the add_subparsers command, that I would have to option of not specifying any sub parser at all. Notably, the In [5] works, if I set nags='3' in the positional arguments.

Is this the intended behaviour? If so, what would be the intended way to achieve what I am trying to do?

Answer 1

Changing positional to -p works (sort of):

In [18]: In [1]: import argparse
    ...: 
    ...: In [2]: parser = argparse.ArgumentParser()
    ...:    ...: parser.add_argument('-p','--positional', type=str, nargs='+')
    ...:    ...: parser.add_argument('-f', '--foo')
    ...:    ...:
    ...:    ...: subparsers = parser.add_subparsers(dest='subcommand', required=
    ...: False)
    ...:    ...: subparser1 = subparsers.add_parser('subcommand1')
    ...:    ...: subparser1.add_argument('-b', '--bar', action='store_true', )
    ...:    ...: subparser2 = subparsers.add_parser('subcommand2')
    ...:    ...: subparser2.add_argument('-b', '--baz', action='store_true')
Out[18]: _StoreTrueAction(option_strings=['-b', '--baz'], dest='baz', nargs=0, const=True, default=False, type=None, choices=None, required=False, help=None, metavar=None)

In [19]: parser.parse_args(['-p','b', 'c', 'd', '-f', 'a', 'subcommand2'])
Out[19]: Namespace(positional=['b', 'c', 'd'], foo='a', subcommand='subcommand2', baz=False)

In [20]: parser.parse_args(['-p','b', 'c', 'd', '-f', 'a'])
Out[20]: Namespace(positional=['b', 'c', 'd'], foo='a', subcommand=None)

In [21]: parser.parse_args(['-p','b', 'c', 'd'])
Out[21]: Namespace(positional=['b', 'c', 'd'], foo=None, subcommand=None)

Here's where using the '-f' as separator works. It marks the end of the strings that get assigned to '-p'.

In [22]: parser.parse_args(['-p','b', 'c', 'd','subcommand2'])
Out[22]: Namespace(positional=['b', 'c', 'd', 'subcommand2'], foo=None, subcommand=None)

In [23]: parser.parse_args(['-p','b', 'c', 'd','-f','a','subcommand2'])
Out[23]: Namespace(positional=['b', 'c', 'd'], foo='a', subcommand='subcommand2', baz=False)

This may be more relevant to the github issue, but I realized that making the 'subparser' more like the true '?' optional positional won't work:

In [27]: p1=argparse.ArgumentParser()
         p1.add_argument('foo',nargs='*');
         p1.add_argument('bar',nargs='?');    
In [29]: p1.parse_args('a b d'.split())
Out[29]: Namespace(foo=['a', 'b', 'd'], bar=None)

In the combination of '*?', the '*' is greedy, grabbing all strings, and leaving none for the '?' (which is ok with none). So as long as subparsers is a positional, there's no way making it work in a non-required sense with a '*' (or even '+') positional. Another way to put it, it's hard to use a '*' positional anywhere except at the end.