First-class support for Variadic Arguments

[noahnu]

Is your feature request related to a problem? Please describe.

I'd like the ability to define variadic arguments.

Describe the solution you'd like

static args = {
    value: Args.string({
        multiple: true,
    })
}

console.log(this.args.value[0], this.args.value[1])

Describe alternatives you've considered

I've set "strict = false" and pulled the args from "const {argv} = await this.parse(..)" as it's called out in the docs: https://oclif.io/docs/args/. This means to use variadic arguments you lose the syntactic sugar (and documentation) of the args definition. You also lose validation (e.g. number / string / etc).

In other words, I want variadic arguments with strict parsing.

Additional context

Previous issues:

• Variadic arguments in oclif oclif#160. This was closed with a recommendation to disable strict parsing. This means you lose the validation.
• "multiple" flag for args oclif#293. This was closed with a link to a pull request in the oclif/parser repository. feat: allow multiple for arg parser#63. That repository has been archived.

Other tools:

• Commander supports variadic arguments: https://github.com/tj/commander.js?tab=readme-ov-file#variadic-option
• Clipanion (what Yarn Modern uses) supports variadic arguments via transparent proxying: Option.Proxy -- I've had to hack something like this together in oclif to support unknown flags so they can be forwarded without disabling strict parsing of known flags though that's a separate issue.
• In Python, the stdlib for argument parsing supports variadic args via nargs: https://docs.python.org/3/library/argparse.html

[github-actions]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 7 days.

[noahnu]

Not stale.

[mdonnalley]

@noahnu sorry about the stale bot - I'm on parental leave through the end of May but I want to take a look at this once I'm back

[Radiergummi]

Yes, please. I have a command to copy files to a destination path—similar to how cp or mv on Linux behave, if passed "a", "b", "c", and "d", it's supposed to copy "a", "b", and "c" to "d". This obviously means I have a variadic "source" argument, and a single, non-variadic "destination" argument.
The only thing I can do with oclif right now is this:

$ bin/dev.js storage cp --help
Copy an object in storage.

USAGE
  $ colibri storage cp SOURCE... DESTINATION... [--json] [-c <value>] [-i <value>] [-v]

ARGUMENTS
  SOURCE...       Source path to copy
  DESTINATION...  Destination path to copy to

…which doesn't match intended behaviour. It would be great to enable some kind of variadic mode per argument individually.

[mdonnalley]

This is a difficult - possibly impossible - problem to solve.

The example provided by @Radiergummi would have this arg definition:

static args = {
  source: Args.string({
    multiple: true,
  }),
  destination: Args.string(),
}

And then you'd expect to be able to do something like this inside the command:

const {args} = await this.parse(MyCommand)
console.log(`These are the sources: ${args.source}`)
console.log(`This is the destination: ${args.destination}`)

my-command a b c d
These are the sources: a b c
This is the destination: d

The problem is how can oclif take the input "a b c d" and determine that a, b and c are all supposed to be part of the source argument and d is supposed to be part of the destination argument?

The arguments are positional and we know that source can take multiple BUT we don't (and can't) know when to stop adding arguments to source and begin adding them to destination.

If destination were required AND there were only two arguments, then we could assume that the very last argument is part of destination and everything else is part of source. But this heuristic becomes overwhelmingly complex when you account of all the possible configurations of arguments.

Probably the only way to handle this is if we required developers to configure the number of expected inputs for a given argument. From what I can tell, this would be similar to how nargs is used in Python's argparse. But this also gets overwhelmingly complex quickly.

The simple solution here is to use flags instead of arguments. The original creator of oclif actually advocates for this in most scenarios: https://clig.dev/#arguments-and-flags and https://medium.com/@jdxcode/12-factor-cli-apps-dd3c227a0e46. Or you can figure out the parsing on your own end using strict = false and pulling the argv from this.parse

For what it's worth, the commander documentation provided in the description is for what they call options, which is analogous to what oclif calls flags. They do support variadic arguments (which they call command-arguments) but only if it's the very last argument:

The last argument of a command can be variadic, and only the last argument.

https://github.com/tj/commander.js?tab=readme-ov-file#command-arguments

Here's my last thought: as I mentioned, I think there's a possibility that we could pull this off if we require developers to configure the number of expected inputs per variadic argument. But I'm not convinced that the amount of work would be worth the added value. For that reason, I'm going to mark this as waiting for interest and help wanted. We'll consider picking up if the issue has enough interest, otherwise we'll leave this open until someone wants to submit a PR themselves.

[Radiergummi]

The simple solution here is to use flags instead of arguments. The original creator of oclif actually advocates for this in most scenarios: https://clig.dev/#arguments-and-flags and https://medium.com/@jdxcode/12-factor-cli-apps-dd3c227a0e46. Or you can figure out the parsing on your own end using strict = false and pulling the argv from this.parse

This would work, but is neither ergonomic for the user, nor really taking advantage of the shell (you'd need to escape and parse globs manually, for example).

I'm pretty sure I'm missing something, but since we're working off of node's argv, is it really that complex? We know the length of the arguments array, and can slice as required by the arguments definition. As long as there's only a single variadic argument, it should be possible to properly parse them, by shifting and popping values for singular args before assigning the rest to the variadic arg:

            src…    dst
my-cli mv  a b c d  ./e
           ^-----^   |
              |     pop
             rest


           src    dst…
my-cli cp  ./a  b c d e
            |   ^-----^
          shift    |
                  rest


           fmt   src…       dst
my-cli xz  tar  a b c d  ./e.tar.xz
            |   ^-----^      |
          shift    |        pop
                  rest

Here's a very simple but working example:

// I was too lazy for the generics, but it's obvious how they'd work anyway
function parseArgs(
  argsDef: {name: string, multiple?: boolean}[],
  argv: string[]
): Record<string, string | string[]> {
  const parsed = {};

  while (argsDef.length > 0) {
    if (!argsDef.at(0)?.multiple) {
      const { name } = argsDef.shift();

      parsed[name] = argv.shift();
    } else if (!argsDef.at(-1)?.multiple) {
      const { name } = argsDef.pop();

      parsed[name] = argv.pop();
    } else {
      const { name } = argsDef.shift();

      parsed[name] = [...argv];
    }
  }

  return parsed;
}

const mvArgs = parseArgs([
  { name: "src", "multiple": true },
  { name: "dst" },
], ["a", "b", "c", "d", "./e"]); // { src: ["a", "b", "c", "d"], dst: "./e" }

const cpArgs = parseArgs([
  { name: "src" },
  { name: "dst", "multiple": true },
], ["./a", "b", "c", "d", "e"]); // { src: "./a", dst: ["b", "c", "d", "e"] }

const xzArgs = parseArgs(
  [
    { name: "fmt" },
    { name: "src", "multiple": true },
    { name: "dst" },
  ],
  ["tar", "a", "b", "c", "d", "./e.tar.xz"]
); // { fmt: "tar", src: ["a", "b", "c", "d"], dst: "./e.tar.xz" }

This is missing the error handling for multiple args with a multiple option, but that's trivial. Is there anything else not covered in this implementation?

[mdonnalley]

@Radiergummi thanks for putting so much thought into this 🏆

As long as there's only a single variadic argument

I hadn't considered that possibility but yeah that would certainly make it easy

I'm technically on parental leave until 6/2 so I won't implement anything before then. But you're welcome to submit a PR for it if you're up to the task and I'll review when I get back in the office

[Radiergummi]

@mdonnalley by all means, get away from GitHub and enjoy the time with your kid(s) then! :)

I'll see what I can come up with and open a PR in the mean time.

[noahnu]

clipanion also only supports a single variadic argument. I think that's a reasonable constraint to add.