Unit testing frameworks such an xUnit or GoogleTest make it easy to run a suite of tests, and with IDE integration you are only a couple of clicks away from having a full debugging session running on the particular test which is failing for whatever reason. I find this to be incredibly helpful.
However, often in my projects I find myself wishing I could a different type of testing, not Unit testing, but some sort of system testing, I'm not sure what to call it. Essentially I'd like to test a function given all possible arguments, or a range of arguments, and have something that resembles a Unit test be automatically created and run, ideally with the same kind of IDE integration so I can easily debug my function with a particular set of arguments.
Obviously there are limitations to this type of testing, even a trivial function with a single 64 bit "int" argument would be impractical to exhaustively test all possible values. On the other hand these days running millions or even billions of similar tests on a relatively complex function is well within the realm of possibility of even low grade PC's.
I could get what I want by programmatically generating Unit tests, then compiling the resulting file as my testSuite, but I have to assume tooling for this kind of testing already exists. I'm hoping for an answer that points me to whatever solutions people are commonly using for this kind of testing.
I'm mostly interested in tools for c, c++, and maybe C#. But other language examples would be interesting to reference as well.
You brought up a good point about the time cost of actually testing all inputs. I think what you're looking for is "Fuzz Testing" / "Fuzzing" or some variation of it. There are a lot of different types of fuzzing that you can further look into: Black-box, grey-box, white-box, mutation-based, generation-based, coverage-based
I'm not too too familiar with specific fuzzing technologies. I've heard of AFL, and PeachFuzzer, but I'm sure you can find many more by doing further searching.
Another type of testing-related thing you might want to look more into is "Symbolic execution", which uses techniques from constraint solving to do things like try to get an idea of what ranges of inputs will take a program down different paths of execution. Or you might also want to look deeper into "Concolic testing", which combines symbolic and concrete execution.