OpenRAM/docs/source/debug.md

Go Back

Debugging and Unit Testing

This page of the documentation explains the debugging and unit testing of OpenRAM.

Table of Contents

1. Unit Tests
2. Unit Test Organization
3. Running Unit Tests
4. Successful Unit Tests
5. Debugging Unsuccessful Unit Tests
6. Temporary Output Files

Unit Tests

OpenRAM has the set of thorough regression tests implemented with the Python unit test framework:

• Unit tests allow users to add features without worrying about breaking functionality.
• Unit tests guide users when porting to new technologies.
• Every sub-module has its own regression test.
• There are regression tests for memory functionality, library cell verification, timing verification, and technology verification.

Unit Test Organization

• 00_code_format_test.py does basic lint checking.
• 01_library_drc_test.py checks DRC of all library cells for the technology.
• 02_library_lvs_test.py checks LVS of all library cells for the technology.
• 03_*_test.py checks DRC and LVS of wires and transistors classes.
• 04_*_test.py checks DRC and LVS of parameterized cells.
• 05-19_*_test.py checks DRC and LVS of module cells (moving upward in hierarchy with numbers)
• 20_*_test.py check DRC and LVS of full SRAM layouts with various configurations.
• 21_*_test.py checks timing of full SRAMs and compares (with tolerance) to precomputed result.

  Note: These tests may fail using different simulators due to the tolerance level.

• 22_*_test.py checks functional simulation of full SRAMs with various configurations.
• 23-25_*_test.py checks lib, lef, and verilog outputs using diff.
• 30_openram_test.py checks command-line interface and whether output files are created.

Running Unit Tests

Regression testing performs a number of tests for all modules in OpenRAM. From the unit test directory ($OPENRAM_HOME/tests), use the following command to run all regression tests:

cd OpenRAM/compiler/tests
make -j 3

The -j can run with 3 threads. By default, this will run in all technologies.

Note: If you have not run openram before running unit tests, the conda environment will not be installed. You can install it by running OpenRAM/install_conda.sh (see Basic Setup for more details).

To run a specific test in all technologies:

cd OpenRAM/compiler/tests
make 05_bitcell_array_test

To run a specific technology:

cd OpenRAM/compiler/tests
TECHS=scn4m_subm make 05_bitcell_array_test

To increase the verbosity of the test, add one (or more) -v options and pass it as an argument to OpenRAM:

ARGS="-v" make 05_bitcell_array_test

Unit test results are put in a directory:

OpenRAM/compiler/tests/results/<technology>/<test>

If the test fails, there will be a tmp directory with intermediate results. If the test passes, this directory will be deleted to save space. You can view the .out file to see what the output of a test is in either case.

To preserve results on successful tests (done automatically if test fails):

KEEP=1 make 05_bitcell_array_test

Successful Unit Tests

user@host:/openram/compiler/tests$ make
scn4m_subm/12_tri_gate_array_test ... PASS!
scn4m_subm/19_pmulti_bank_test ... PASS!
freepdk45/21_ngspice_delay_global_test ... PASS!
scn4m_subm/23_lib_sram_linear_regression_test ... PASS!
.
.
.

user@host:/openram/compiler/tests$ make 01_library_test
scn4m_subm/01_library_test ... PASS!
freepdk45/01_library_test ... PASS!

Debugging Unsuccessful Unit Tests (or sram_compiler.py)

• You will get a FAIL during unit test
• You can see the output and stack trace in $OPENRAM_HOME/tests/results/<tech>/<test>.out
• Examine the temporary output files in the temp directory ($OPENRAM_HOME/tests/results/<tech>/<test>/)

user@host:/openram/compiler/tests$ make 01_library_test
scn4m_subm/01_library_test ... FAIL!

It didn't finish... where are my files?

• OpenRAM puts all temporary files in a temporary directory named:
  • $OPENRAM_HOME/tests/results/<tech>/<test>/
  • This allows multiple unit tests to simultaneously run
  • After a successful run, the directory and contents are deleted
  • To preserve the contents, you can run with the KEEP option for debugging

Temporary Output Files

• DRC standard output (*.drc.out), errors (*.drc.err), and results (*.drc.results)
• LVS standard output (*.lvs.out), errors (*.lvs.out), and results (*.lvs.results)
• GDS (and Magic) files for intermediate modules (temp.gds, temp.mag)
• SPICE netlist for intermediate module results (temp.sp)
• Extracted layout netlist for intermediate module results (extracted.sp)
• Magic only: Run scripts for DRC (run_drc.sh) and LVS (run_lvs.sh)
• Calibre only: Runset file for DRC (drc_runset) and LVS (lvs_runset)