diff --git a/docs/source/debug.md b/docs/source/debug.md index e8ae091c..a2ea6aaf 100644 --- a/docs/source/debug.md +++ b/docs/source/debug.md @@ -1,7 +1,8 @@ ### [Go Back](./index.md#table-of-contents) # Debugging and Unit Testing -This page of the documentation explains the debugging and unit testing of OpenRAM. +This page of the documentation explains the debugging and unit testing of +OpenRAM. @@ -16,11 +17,14 @@ This page of the documentation explains the debugging and unit testing of OpenRA ## Unit Tests -OpenRAM has the set of thorough regression tests implemented with the Python unit test framework: -* Unit tests allow users to add features without worrying about breaking functionality. +OpenRAM has the set of thorough regression tests implemented with the Python +unit test framework: +* Unit tests allow users to add features without worrying about breaking + functionality. * Unit tests guide users when porting to new technologies. * Every sub-module has its own regression test. -* There are regression tests for memory functionality, library cell verification, timing verification, and technology verification. +* There are regression tests for memory functionality, library cell + verification, timing verification, and technology verification. @@ -33,7 +37,8 @@ OpenRAM has the set of thorough regression tests implemented with the Python uni * `05-19_*_test.py` checks DRC and LVS of module cells (moving upward in hierarchy with numbers) * `20_*_test.py` check DRC and LVS of full SRAM layouts with various configurations. * `21_*_test.py` checks timing of full SRAMs and compares (with tolerance) to precomputed result. - > **Note**: These tests may fail using different simulators due to the tolerance level. + > **Note**: These tests may fail using different simulators due to the + > tolerance level. * `22_*_test.py` checks functional simulation of full SRAMs with various configurations. * `23-25_*_test.py` checks lib, lef, and verilog outputs using diff. * `30_openram_test.py` checks command-line interface and whether output files are created. @@ -41,16 +46,20 @@ OpenRAM has the set of thorough regression tests implemented with the Python uni ## Running Unit Tests - -Regression testing performs a number of tests for all modules in OpenRAM. -From the unit test directory ($OPENRAM\_HOME/tests), -use the following command to run all regression tests: +Regression testing performs a number of tests for all modules in OpenRAM. From +the unit test directory (`$OPENRAM_HOME/tests`), use the following command to run +all regression tests: ``` cd OpenRAM/compiler/tests make -j 3 ``` -The -j can run with 3 threads. By default, this will run in all technologies. Note that if you have not run openram, the conda environment will not be installed. You can install it by running OpenRAM/install_conda.sh (see [Basic Setup](basic_setup.md#anaconda) for more details). + +The `-j` can run with 3 threads. By default, this will run in all technologies. +> **Note**: If you have not run openram before running unit tests, the conda +> environment will not be installed. You can install it by running +> `OpenRAM/install_conda.sh` (see [Basic Setup](basic_setup.md#anaconda) for +> more details). To run a specific test in all technologies: ``` @@ -63,8 +72,8 @@ cd OpenRAM/compiler/tests TECHS=scn4m_subm make 05_bitcell_array_test ``` -To increase the verbosity of the test, add one (or more) -v options and -pass it as an argument to OpenRAM: +To increase the verbosity of the test, add one (or more) `-v` options and pass +it as an argument to OpenRAM: ``` ARGS="-v" make 05_bitcell_array_test ``` @@ -73,95 +82,52 @@ Unit test results are put in a directory: ``` OpenRAM/compiler/tests/results// ``` -If the test fails, there will be a tmp directory with intermediate results. -If the test passes, this directory will be deleted to save space. -You can view the .out file to see what the output of a test is in either case. -* Tests can be run in the `$OPENRAM_HOME/tests` directory -* Command line arguments - * `-v` for verbose - * `-t` freepdk45 for tech - * `-d` to preserve /tmp results (done automatically if test fails) -* Individual tests - * `01_library_drc_test.py` -* All tests - * `regress.py` +If the test fails, there will be a `tmp` directory with intermediate results. If +the test passes, this directory will be deleted to save space. You can view the +`.out` file to see what the output of a test is in either case. + +To preserve results on successful tests (done automatically if test fails): +``` +KEEP=1 make 05_bitcell_array_test +``` ## Successful Unit Tests ```console -user@host:/openram/compiler/tests$ ./regress.py - ______________________________________________________________________________ -|==============================================================================| -|========= Running Test for: =========| -|========= scn4m_subm =========| -|========= ./regress.py =========| -|========= /tmp/openram_mrg_13245_temp/ =========| -|==============================================================================| -runTest (00_code_format_check_test.code_format_test) ... ok -runTest (01_library_drc_test.library_drc_test) ... ok -runTest (02_library_lvs_test.library_lvs_test) ... ok -runTest (03_contact_test.contact_test) ... ok -runTest (03_path_test.path_test) ... ok +user@host:/openram/compiler/tests$ make +scn4m_subm/12_tri_gate_array_test ... PASS! +scn4m_subm/19_pmulti_bank_test ... PASS! +freepdk45/21_ngspice_delay_global_test ... PASS! +scn4m_subm/23_lib_sram_linear_regression_test ... PASS! . . . ``` ```console -user@host:/openram/compiler/tests$ ./03_ptx_1finger_nmos_test.py - ______________________________________________________________________________ -|==============================================================================| -|========= Running Test for: =========| -|========= scn4m_subm =========| -|========= ./03_ptx_1finger_nmos_test.py =========| -|========= /tmp/openram_mrg_13750_temp/ =========| -|==============================================================================| -. ----------------------------------------------------------------------- -Ran 1 test in 0.596s - -OK +user@host:/openram/compiler/tests$ make 01_library_test +scn4m_subm/01_library_test ... PASS! +freepdk45/01_library_test ... PASS! ``` ## Debugging Unsuccessful Unit Tests (or sram\_compiler.py) -* You will get an ERROR during unit test and see a stack trace -* Examine the temporary output files in the temp directory (/tmp/mydir) +* You will get a FAIL during unit test +* You can see the output and stack trace in + `$OPENRAM_HOME/tests/results//.out` +* Examine the temporary output files in the temp directory + (`$OPENRAM_HOME/tests/results///`) ```console - _____________________________________________________________________________ -|==============================================================================| -|========= Running Test for: =========| -|========= scn4m_subm =========| -|========= ./04_pinv_10x_test.py =========| -|========= /tmp/mydir =========| -|==============================================================================| -ERROR: file magic.py: line 174: DRC Errors pinv_0 2 -F -====================================================================== -FAIL: runTest (__main__.pinv_test) ----------------------------------------------------------------------- -Traceback (most recent call last): - File "./04_pinv_10x_test.py", line 22, in runTest - self.local_check(tx) - File "/Users/mrg/openram/compiler/tests/testutils.py", line 45, in local_check - self.fail("DRC failed: {}".format(a.name)) -AssertionError: DRC failed: pinv_0 - ----------------------------------------------------------------------- -Ran 1 test in 0.609s - -FAILED (failures=1) +user@host:/openram/compiler/tests$ make 01_library_test +scn4m_subm/01_library_test ... FAIL! ``` ### It didn't finish... where are my files? * OpenRAM puts all temporary files in a temporary directory named: - * `/tmp/openram___temp` - * This allows multiple processes/users to simultaneously run - * This allows /tmp to be mapped to a RAM disk for faster performance + * `$OPENRAM_HOME/tests/results///` + * This allows multiple unit tests to simultaneously run * After a successful run, the directory and contents are deleted - * To preserve the contents, you can run with the `-d` option for debugging -* `OPENRAM_TMP` will override the temporary directory location for debug - * `export OPENRAM_TMP="/home/myname/debugdir"` + * To preserve the contents, you can run with the `KEEP` option for debugging @@ -176,4 +142,3 @@ FAILED (failures=1) -