Is it possible to have VUNIT run a test suite based on top level generics?

Question

I am new to Vunit - but not testing. I currently have a setup where I have testbenches that use a lot of OSVVM where setup is provided via generics to the testbench, and I use either TCL or Pytest to run the test through a full test suite.

Having started looking into Vunit properly, I can see it might do what I'm after or at least allow me to move some of the test suit for some tests into the VHDL, but what I can't find is an example where part of the test case differences involve testing with different bus widths or sync/async clocks, for example.

Is there a way to do this via Vunit? if there is, is there an example I can look at? Or would you just expect to instantiate them all in the same Vunit testbench and run them in sequence?

My infrastructure is very OSVVM heavy, so I am not looking to change any of that - but I am looking for a runner to regress the testcases in GitLab. TCL doesn't cut it (mainly because ActiveHDL has a bug where it doesn't catch runtime exceptions properly in TCL).

Answer 1

VUnit has a configuration concept (not to be confused with VHDL configurations) that allow you to do that, among other things. You can read about it here.

You can also look at the following example.

I'm not sure if OSVVM heavy also implies using different VHDL configurations for different tests? If so, you should read this

Update based on comments:

Let's say we have a UUT with two generics, a width that can be 32/64/128 bits and a boolean use_fifo. Here's a dummy (but executable) representation of that UUT

entity uut is
  generic (
    width : positive;
    use_fifo : boolean);
end entity;

architecture a of uut is
begin
  do_stuff: process
  begin
    report "Width = " & to_string(width) & ", use_fifo = " & to_string(use_fifo);
    wait;
  end process;
end architecture;

Assume that we have a testbench and want to run that with all combinations of the generics. To do that we expose the UUT generics as generics to the VUnit testbench

library vunit_lib;
context vunit_lib.vunit_context;

entity tb_uut is
  generic (
    runner_cfg : string;
    width : positive;
    use_fifo : boolean);
end entity;

architecture tb of tb_uut is
begin
  main : process
  begin
    test_runner_setup(runner, runner_cfg);

    -- Do some testing

    test_runner_cleanup(runner);
  end process;

  uut_inst: entity work.uut
    generic map (
      width => width,
      use_fifo => use_fifo);
end architecture;

Now we can create VUnit configurations for the testbench that represent all generic combinations. This is done in the run script.

from pathlib import Path
from itertools import product
from vunit import VUnit


prj = VUnit.from_argv()
root = Path(__file__).parent
lib = prj.add_library("lib")
lib.add_source_files(root / "*.vhd")

tb_uut = lib.test_bench("tb_uut")
for width, use_fifo in product([32, 64, 128], [False, True]):
    tb_uut.add_config(
        name=f"width={width}.use_fifo={use_fifo}",
        generics=dict(width=width, use_fifo=use_fifo),
    )

prj.main()

If we list the tests we will see all six combinations:

$ python run.py --list
lib.tb_uut.width=32.use_fifo=False
lib.tb_uut.width=32.use_fifo=True
lib.tb_uut.width=64.use_fifo=False
lib.tb_uut.width=64.use_fifo=True
lib.tb_uut.width=128.use_fifo=False
lib.tb_uut.width=128.use_fifo=True
Listed 6 tests