MRT User Configuration - Overriding YAML File by Command Line Positional Arguments

[declmal]

There's a common case that when a user invokes a MRT process, he/she will type in the command line as follows:

python main2.py --yaml-path /path/to/test.yaml

where the yaml file test.yaml is configured beforehand.

when he/she needs to revise some hyper-parameters, one feasible way is to reopen test.yaml and revise the corresponding parameter, which could be tedious.

Another possible way is overriding by optional command line arguments, such as

python main2.py --yaml-path /path/to/test.yaml --softmax-lambd 0.5

where the value of softmax_lambd will be overriden as 0.5.

However, as shown in yacs doc:

yaml configuration is updated by the interface cfg.merge_from_list, and it is not recommended for configuration along with a command line argument.

So, what could be the best way to implement this command line overriden feature, or is this feature necessary?

Though, one possible solution is to use the common package located in python/mrt/common.

[declmal]

It seems there are not many opensource choices to override user command in the .yaml file.
One possible way is to use yq, which is a compiled go binary that should be installed beforehand.

suppose the original yaml configuration is written in /path/to/test.yaml as:

AAA:
  BB:
    C: true

The program yq only supports offline overriding, which means that after the user execute:

yq -i e '.AAA.BB.C |= false' /path/to/test.yaml

/path/to/test.yaml will be overriden as:

AAA:
  BB:
    C: false

But this feature is beyond our scope of development.

[declmal]

I did not find any other choices that satisfies the need, so I suppose using argparse would be a feasible choice.