diff --git a/bin/verilator b/bin/verilator index 353fa95ea..c601a58ed 100755 --- a/bin/verilator +++ b/bin/verilator @@ -521,7 +521,9 @@ changing debug verbosity. Enabled automatically when --debug specified. =item --debugi- Rarely needed - for developer use. Set internal debugging level globally -or on the specified source file to the specified level. +to the specified debug level (1-10) or set the specified source file to the +specified level. Higher levels produce more detailed messages (plain +C<--debug> is equivalent to C<--debugi 4>). =item +define+I+I @@ -531,9 +533,10 @@ standard across Verilog tools while -D is an alias for GCC compatibility. =item --dump-tree Rarely needed. Enable writing .tree debug files with dumping level 3, -which dumps the standard critical stages. --dump-tree is enabled -automatically with --debug, so "--debug --no-dump-tree" may be useful if -the dump files are large and not desired. +which dumps the standard critical stages. For details on the format see +the Verilator Internals manual. --dump-tree is enabled automatically with +--debug, so "--debug --no-dump-tree" may be useful if the dump files are +large and not desired. =item --dump-treei @@ -3395,12 +3398,27 @@ follows: cp -p t/t_EXAMPLE.pl t/t_BUG.pl cp -p t/t_EXAMPLE.v t/t_BUG.v +There are many hits on how to write a good test in the driver.pl +documentation which can be seen by running: + + cd $VERILATOR_ROOT # Need the original distribution kit + test_regress/driver.pl --help + Edit t/t_BUG.pl to suit your example; you can do anything you want in the Verilog code there; just make sure it retains the single clk input and no outputs. Now, the following should fail: + cd $VERILATOR_ROOT # Need the original distribution kit cd test_regress - t/t_BUG.pl + t/t_BUG.pl # Run on Verilator + t/t_BUG.pl --debug # Run on Verilator, passing --debug to Verilator + t/t_BUG.pl --vcs # Run on a commercial simulator + t/t_BUG.pl --nc|--iv|--ghdl # Likewise on other simulators + +The test driver accepts a number of options, many of which mirror the main +Verilator option. For example the previous test could have been run with +debugging enabled. The full set of test options can be seen by running +driver.pl --help as shown above. Finally, report the bug using the bug tracker at L. The bug will become publicly diff --git a/internals.pod b/internals.pod index 5132acddd..65a93e082 100644 --- a/internals.pod +++ b/internals.pod @@ -336,10 +336,10 @@ AST. =head1 TESTING -To write a test see notes in the forum and in the verilator.txt manual. +To write a test see the BUGS section of the Verilator primary manual, and +the documentation in: -Note you can run the regression tests in parallel; see the -test_regress/driver.pl script -j flag. + test_regress/t/driver.pl --help =head1 DEBUGGING diff --git a/test_regress/driver.pl b/test_regress/driver.pl index 18575750c..0920be132 100755 --- a/test_regress/driver.pl +++ b/test_regress/driver.pl @@ -1469,27 +1469,226 @@ The driver reports the number of tests which pass, fail, skipped (some resource required by the test is not available, such as SystemC), or are unsupported (buggy or require a feature change before will pass.) -=head1 ARGUMENTS +There are hundreds of tests, and for faster completion you may want to run +the regression tests with CCACHE enabled and in parallel on a machine with +many cores. See the -j option. + +=head1 TEST CONFIGURATION + +The Perl script (e.g. C) controls how the test +will run by driver.pl. In general it includes a call to the C +subroutine to compile the test with Verilator (or an alternative +simulator), followed by a call to the C subroutine to run the +test. Compile-only tests omit the call to C. + +If those complete, the script calls C to increment the count of +successful tests and then returns 1 as its result. + +Both C and C take an optional argument hash table to +control their behavior. For example: + + compile ( + verilator_flags2 => ["--lint-only"], + fails => 1, + ); + +indicates that when compiling this test, the C<--lint-only> flag should be +passed and that the test is expected to fail. + +The full list of arguments can be found by looking at the C +source code. Some commonly used arguments are: + +=over 4 + +=item all_run_flags + +A list of flags to be passed when running the simulator (Verilated model or +one of the other simulators). + +=item check_finished + +Set to 1 to indicate successful completion of the test is indicated by the +string C<*-* All Finished *-*> being printed on standard output. This is +the normal way for successful tests to finish. + +=item expect + +A quoted list of strings or regular expression to be matched in the +output. See for more detail on how this argument +should be used. + +=item fails + +Set to 1 to indicate this step (C or C) is expected to +fail. Tests that are expected to fail generally have _bad in their +filename. + +=item make_main + +Set to 0 to disable the automatic creation of a C++ test wrapper (for +example when a hand-written test wrapper is provided using C<--exe>). + +=item make_top_shell + +Set to 0 to disable the automatic creation of a top level shell to run the +executable (for example when a hand-written test wrapper is provided using +C<--exe>). + +=item nc_flags + +=item nc_flags2 + +=item nc_run_flags + +The equivalent of C, C and C, but only +for use with the Cadence NC simulator. + +=item iv_flags + +=item iv_flags2 + +=item iv_run_flags + +The equivalent of C, C and C, but only +for use with the Icarus Verilog simulator. + +=item v_flags + +A list of standard Verilog simulator flags to be passed to the simulator +compiler (Verilator or one of the other simulators). This list is create +by the driver and rarely changed, use v_flags2 instead. + +=item v_flags2 + +A list of standard Verilog simulator flags to be passed to the simulator +compiler (Verilator or one of the other simulators). Unlike v_flags, these +options may be overridden in some simulation files. + +Similar sets of flags exist for atsim, GHDL, Cadence NC and Synopsys VCS. + +=item vcs_flags + +=item vcs_flags2 + +=item vcs_run_flags + +The equivalent of C, C and C, but only +for use with the Synopsys VCS simulator. + +=item verilator_flags + +=item verilator_flags2 + +The equivalent of C and C, but only for use with +Verilator. If a flag is a standard flag (+incdir for example) v_flags2 +should be used instead. + +=back + +=head2 HINTS ON WRITING TESTS + +There is generally no need for the test to create its own main program or +top level shell as the driver creates one automatically, however some tests +require their own C++ or SystemC test harness. This is commonly given the +same name as the test, but with .cpp as suffix +(C). This can be specified as follows: + + compile ( + make_top_shell => 0, + make_main => 0, + verilator_flags2 => ["--exe $Self->{t_dir}/$Self->{name}.cpp"], ); + +Tests should be self-checking, rather than producing lots of output. If a +test succeeds it should print C<*-* All Finished *-*> to standard output +and terminate (in Verilog C<$finish>), if not it should just stop (in +Verilog C<$stop>) as that signals an error. + +If termination should be triggered from the C++ wrapper, the following code +can be used: + + vl_fatal (__FILE__, __LINE__, "dut", ""); + exit (1); + +This can be particularly useful if checking that the Verilator model has +not unexpectedly terminated. + + if (Verilated::gotFinish ()) { + vl_fatal (__FILE__, __LINE__, "dut", ""); + exit (1); + } + +Where it might be useful for a test to produce output, it should qualify +this with C. For example in Verilog: + + `ifdef TEST_VERBOSE + $write ("Conditional generate if MASK [%1d] = %d\n", g, MASK[g]); + `endif + +Or in a hand-written C++ wrapper: + + #ifdef TEST_VERBOSE + cout << "Read a = " << a << endl; + #endif + +The C argument should not generally be used to decide if a test has +succeeded. However, in the case of tests that are designed to fail at +compile time, it is the only option. For example: + + compile ( + v_flags2 => ["--lint-only"], + fails=>1, + expect=> + q{%Error: t/t_gen_cond_bitrange_bad.v:\d+: Selection index out of range: 2:2 outside 1:0 + %Error: t/t_gen_cond_bitrange_bad.v:\d+: Selection index out of range: 2:2 outside 1:0 + %Error: t/t_gen_cond_bitrange_bad.v:\d+: Selection index out of range: 2:2 outside 1:0 + %Error: t/t_gen_cond_bitrange_bad.v:\d+: Selection index out of range: 2:2 outside 1:0 + %Error: Exiting due to .*}, + ); + +The strings to match should be made general - for example the line numbers +in the Verilog should not be critical and the total number of errors should +not matter. This makes it easier to extend or modify the test in future. + +=head1 DRIVER ARGUMENTS =over 4 =item --atsim -Run using ATSIM. +Run using ATSIM simulator. =item --benchmark [] Show execution times of each step. If an optional number is given, specifies the number of simulation cycles (for tests that support it). +=item --debug + +Same as C: Use the debug version of Verilator which +enables additional assertions, debugging messages, and structure dump +files. + +=item --debugi(-) + +Same as C: Set Verilator internal debugging level +globally to the specified debug level (1-10) or set the specified source +file to the specified level. Higher levels produce more detailed messages +(plain C<--debug> is equivalent to C<--debugi 4>). + +=item --dump-tree + +Same as C: Enable Verilator writing .tree debug +files with dumping level 3, which dumps the standard critical stages. For +details on the format see the Verilator Internals manual. + =item --gdb -Run Verilator under the debugger. +Same as C: Run Verilator under the debugger. =item --gdbbt -Run Verilator under the debugger, only to print backtrace information. -Requires --debug. +Same as C: Run Verilator under the debugger, only to +print backtrace information. Requires --debug. =item --gdbsim @@ -1497,7 +1696,7 @@ Run Verilator generated executable under the debugger. =item --ghdl -Run using GHDL. +Run using GHDL simulator. =item --golden @@ -1507,20 +1706,24 @@ Update golden files, equivalent to setting HARNESS_UPDATE_GOLDEN=1. Displays this message and program version and exits. +=item --iverilog + +Run using Icarus Verilog simulator. + =item --j # Run number of parallel tests, or 0 to determine the count based on the number of cores installed. Requires Perl's Parallel::Forker package. +=item --nc + +Run using Cadence NC-Verilog simulator. + =item --optimize Randomly turn on/off different optimizations. With specific flags, use those optimization settings -=item --nc - -Run using NC-Verilog. - =item --site Run site specific tests also. @@ -1529,17 +1732,22 @@ Run site specific tests also. Stop on the first error. +=item --trace + +Set the simulator specific flags to request waveform tracing. + =item --vcs -Run using VCS. +Run using Synopsys VCS simulator. =item --verbose -Enable test verbose messages. +Compile and run the test in verbose mode. This means C will +be defined for the test (Verilog and any C++/SystemC wrapper). =item --vlt -Run using Verilator. +Run using Verilator. Defaults set unless another simulator is requested. =back