Trade-off between parametrization conciseness and function name expressiveness

[chillenzer]

Should we point out that parametrisation is not always the way to go? One of the big advantages of an extensive test suite is its documenting function that gets a bit spoiled if you have extensive parametrisation in your tests. The additional comments in the parametrisation in "pytest features/Fix the function" are a good example. They are something that would have been part of the function name otherwise. Thus, if, e.g., the "Test non-integers" line failed, pytest would only give you the raw parameter values (and not the comment explaining what the test was supposed to test). One could even have two identical tests that are still supposed to test for two fundamentally different behaviors that only coincidentally happen to be identical. It would be more expressive to indicate that these behaviors are to be distinguished by having two functions. While it would, of course, be possible to split up the parametrised test if this degeneracy ever got lifted, it is not certain that at that point someone will be around to remember that and even know that there are separate behaviors there.

[edbennett]

I would separate having differing groups of arguments to conceptually the same test from having two conceptually different tests that happen to map to the same code.

I would also point out that you can pass ids to parametrize to control what names pytest spits out when it runs parametrized tests.

(That doesn't mean I don't think this is worth mentioning at all however.)

[chillenzer]

Good point! Didn't know that. Then, we should, I believe, use that as the default and add some ids to the parameters.

[edbennett]

I'll confess that neither did I, until I read your message, and channeled Raymond Hettinger and thought "there must be a better way!"