luke
/
OpenSTA
mirror of https://github.com/The-OpenROAD-Project/OpenSTA.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

Merge pull request #387 from The-OpenROAD-Project-staging/sta_update_doc 

add MT info to tag doc
This commit is contained in:
Matt Liberty 2026-07-09 04:24:46 +00:00 committed by GitHub
parent a536a9e526 3770ca007f
commit 97d84f0ed7
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 189 additions and 15 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

204
doc/TagsAndScenes.md
View File

@ -2,8 +2,9 @@
This document describes how `Tag`/`ClkInfo`/`TagGroup` distinguish and store multiple This document describes how `Tag`/`ClkInfo`/`TagGroup` distinguish and store multiple
timing paths per vertex, how arrival and required times propagate through the graph, timing paths per vertex, how arrival and required times propagate through the graph,
how delay calculation is indexed, and how multiple clocks and multiple scenes how delay calculation is indexed, how multiple clocks and multiple scenes
(MCMM) are carried through all of it. (MCMM) are carried through all of it, and how the shared data structures are
made safe under multithreaded propagation (§7).
It is written as a handoff reference: each section starts with a plain-language It is written as a handoff reference: each section starts with a plain-language
explanation before diving into the code. Section 0 is a glossary of terms used explanation before diving into the code. Section 0 is a glossary of terms used
@ -650,7 +651,179 @@ paths filtered by (scene, min/max, rf).
--- ---
## 7. Quick reference: what makes two paths "different" ## 7. Threading and concurrency
*Plain language: OpenSTA runs single-threaded by default. When more threads are
enabled, worker threads exist only inside a handful of well-delimited parallel
phases (delay calc, arrival search, required search, path-end collection, clock
skew). Everything those phases need — graph topology, levelization, constraints,
liberty data, scenes — is built beforehand on the main thread and treated as
read-only inside the phase. Shared structures that must grow during a phase
(the tag/ClkInfo/TagGroup tables, BFS queues, invalidation sets) take a mutex
on the write path; the hot read paths are lock-free.*
### 7.1 Thread pool
- Default thread count is **1** (`Sta::defaultThreadCount`, `search/Sta.cc:307`).
The Tcl command `set_thread_count` (`util/Util.i:99`) calls
`Sta::setThreadCount`, which stores `StaState::thread_count_` and creates (or
resizes) a `DispatchQueue` thread pool (`Sta::setThreadCount1`,
`search/Sta.cc:320`; pool in `util/DispatchQueue.cc`,
`include/sta/DispatchQueue.hh`). No pool exists until thread count > 1.
- `DispatchQueue::dispatch(fn)` hands a task to a worker; each task receives its
thread index (used to pick per-thread scratch state).
`DispatchQueue::finishTasks()` blocks until all dispatched tasks complete
(`DynamicLatch`, an atomic counter with C++20 `atomic::wait`). Every parallel
phase ends with `finishTasks()` — worker threads never outlive a phase.
- All Tcl commands, netlist edits, SDC changes, liberty/SPEF reading, and
reporting run on the main thread. Worker threads run only in the phases below.
### 7.2 The parallel phases
| Phase | Entry point | Parallelized over |
|---|---|---|
| Delay calculation | `GraphDelayCalc::findDelays` → `iter_->visitParallel` (`dcalc/GraphDelayCalc.cc:356`) | driver vertices, level by level |
| Arrival search | `Search::findArrivals1` / `findClkArrivals` → `arrival_iter_->visitParallel` (`search/Search.cc:657,1047`) | vertices, level by level |
| Required search | `Search::findRequireds` → `required_iter_->visitParallel` (`search/Search.cc:3152`) | vertices, reverse level order |
| Path end collection | `PathGroups::makeGroupPathEnds` (`search/PathGroup.cc:1008`) | endpoint vertices |
| Clock skew | `ClkSkews::findClkSkew` (`search/ClkSkew.cc:198`) | register clock vertices; per-thread partial skew maps merged after `finishTasks` |
The generated-clock source-path search is deliberately **not** parallel
(`Genclks::findSrcArrivals`, `search/Genclks.cc:788-795`, comment: "Parallel
visit is slightly slower (at last check)").
### 7.3 The level-barrier invariant
`BfsIterator::visitParallel` (`search/Bfs.cc:160`) processes one level at a
time: the level's vertex list is split into one contiguous chunk per thread,
the chunks are dispatched, and `finishTasks()` is called **before moving to the
next level**. (If a level has fewer vertices than threads it is visited
serially on the main thread.) Two consequences:
1. **Each vertex is visited by exactly one thread.** Per-vertex and per-edge
result storage is therefore written without locks: `Vertex::paths_`
(written via `Search::setVertexArrivals` → `Vertex::makePaths`,
`graph/Graph.cc:1198`), per-vertex slew slots (`Graph::setSlew`,
`graph/Graph.cc:677`), and per-edge arc delay slots. The arrays themselves
are allocated on the main thread before the phase (`Graph::initSlews` /
`Graph::initArcDelays`, `graph/Graph.cc:866-914`) — the parallel phase only
fills slots.
2. **All fanin (fanout, for requireds) is final before a vertex is visited.**
When `ArrivalVisitor` reads the from-vertex `Path` arrays in
`visitFaninPaths`, those vertices are at lower levels, completed before the
previous level barrier. Latch D→Q edges, which violate level order, are not
followed in-pass: the latch output is parked in `postponed_arrivals_` (under
`postponed_arrivals_lock_`, `search/Search.cc:1428-1437`) and visited on the
next pass of the loop in `Search::findAllArrivals`.
Workers do enqueue fanout vertices into the BFS queue during a visit; the queue
is protected by `BfsIterator::queue_lock_` (`search/Bfs.cc:271`) with a
double-checked "already in queue" flag kept as an atomic per-vertex bitmask
(`Vertex::bfs_in_queue_`, `std::atomic<uint8_t>`, `include/sta/Graph.hh:325`).
### 7.4 Per-thread visitor state
`visitParallel` makes one `visitor->copy()` per thread (`search/Bfs.cc:169-172`);
each copy owns the mutable scratch state a visit needs:
- `ArrivalVisitor` copies own their `TagGroupBldr` scratch builders
(`tag_bldr_`, `tag_bldr_no_crpr_`, `search/Search.cc:1083-1096`) and a
private `tag_cache_` (`PathVisitor` copy constructor passes
`make_tag_cache=true`, `search/Search.cc:1962-1973`) — see §7.5.
- `FindVertexDelays::copy` clones the `ArcDelayCalc` "because it needs separate
state for each thread" (`dcalc/GraphDelayCalc.cc:321-327`).
- `MakeEndpointPathEnds` copies its `PathEndVisitor` per thread
(`search/PathGroup.cc:979-997`); results funnel into shared `PathGroup`s
under a lock (§7.6).
- `ClkSkews` gives each thread its own `ClkSkewMap` (`partial_skews`,
`search/ClkSkew.cc:195-203`) and reduces them on the main thread after the
barrier.
### 7.5 Interned global tables: locked create, lock-free read
The three interning tables of §2 grow concurrently during arrival/required
search. All use the same discipline — mutex on create, no lock on read:
- **Tags** — `Search::findTag` takes `tag_lock_` (`search/Search.cc:2894`) to
probe/insert `tag_set_`. Before touching the lock it probes the visitor's
per-thread `tag_cache_` (`search/Search.cc:2888-2892`); cache hits skip the
global lock entirely.
- **ClkInfos** — `Search::findClkInfo` takes `clk_info_lock_`
(`search/Search.cc:2984`).
- **TagGroups** — `Search::findTagGroup` takes `tag_group_lock_`
(`search/Search.cc:2646`). `TagGroup::ref_count_` is `std::atomic<int>`
(`search/TagGroup.hh:82`) because `setVertexArrivals` increments/decrements
it from concurrent visits.
Index-to-object lookup (`Search::tag(TagIndex)`, used on every `Path::tag`
call) reads the `tags_` array with **no lock**. That is safe because the array
pointers are atomic (`std::atomic<Tag**> tags_`, `std::atomic<TagGroup**>
tag_groups_`, `include/sta/Search.hh:632,641`) and growth is copy-then-publish:
under the table lock, a double-size array is allocated, existing entries are
copied, and only then is the atomic pointer swung
(`search/Search.cc:2919-2930,2659-2670` — "make the new array and copy the
contents into it before updating tags_ so that other threads can use
Search::tag(TagIndex) without returning gubbish"). The retired arrays are
parked in `tags_prev_`/`tag_groups_prev_` and freed only after the parallel
pass by `Search::deleteTagsPrev` (`search/Search.cc:671-681`), so a reader
holding the old pointer never sees freed memory. New tags are stored into
`tags_[index]` *before* being inserted into `tag_set_`
(`search/Search.cc:2906-2909`), so any tag visible in the set is indexable.
### 7.6 Other shared structures written during parallel phases
Each has a dedicated mutex (`LockGuard` = `std::scoped_lock<std::mutex>`,
`include/sta/Mutex.hh:32`):
| Structure | Lock | Who writes concurrently |
|---|---|---|
| BFS level queues | `BfsIterator::queue_lock_` (`search/Bfs.cc:271`) | visits enqueueing fanout/fanin |
| `invalid_arrivals_` / `invalid_requireds_` | `invalid_arrivals_lock_` (`search/Search.cc:851,930` — "Lock for StaDelayCalcObserver called by delay calc threads") | delay-calc threads invalidating search results via observer |
| `invalid_tns_` | `tns_lock_` (`search/Search.cc:3721`) | endpoint slack invalidation from visits |
| Worst-slack queue per path index | `WorstSlack::lock_` (`search/WorstSlack.cc:265` — "Locking is required because ArrivalVisitor is called by multiple threads") | arrival visits updating WNS |
| `postponed_arrivals_` (latch outputs), `postponed_clk_endpoints_` | own locks (`search/Search.cc:1437,1024`) | arrival visits |
| `filtered_arrivals_` | `filtered_arrivals_lock_` (`search/Search.cc:2704`) | `setVertexArrivals` when a filter tag lands |
| `PathGroup::path_ends_` + pruning threshold | `PathGroup::lock_` (`search/PathGroup.cc:119,180`) | per-endpoint path-end insertion in `makeGroupPathEnds` |
| `invalid_check_edges_` / `invalid_latch_edges_` | `invalid_edge_lock_` (`dcalc/GraphDelayCalc.cc:762-782`) | dcalc visits queueing check/latch edges (processed serially after the parallel pass, `GraphDelayCalc.cc:359-367`) |
| Multi-driver net registry | `multi_drvr_lock_` (`dcalc/GraphDelayCalc.cc:814` — taken only when a vertex has multiple drivers: "Avoid locking for single driver nets") | lazy `MultiDrvrNet` creation during dcalc |
| Cycle accounting table | `Sdc::cycle_acctings_lock_` (`sdc/Sdc.cc:2411-2417` — "Determine cycle accounting on demand") | `CycleAccting` interning from required-time math in parallel required search |
| CUDD BDD manager (`Sim`) | `bdd_lock_` (`search/Sim.cc:91,121`) | BDD evaluation — the shared CUDD manager is serialized |
| Parasitic maps | `ConcreteParasitics::lock_` (`parasitics/ConcreteParasitics.cc:915-1187`) | on-demand reduced-parasitic creation and lookups from dcalc threads |
| Lazy voltage waveforms | `LibertyCell::waveform_lock_` + atomic `have_voltage_waveforms_` double-check (`liberty/Liberty.cc:1861-1868`) | first dcalc thread to need CCS waveforms for a cell |
| Debug print buffers | `Debug::buffer_lock_` (`include/sta/Debug.hh:60-66`) | `debugPrint` from any thread |
### 7.7 Pre-populated, read-only during parallel phases
The following are built or updated only on the main thread (between commands or
in a serial preamble) and are read without locks inside parallel phases:
- **Graph topology** — vertices/edges are made/deleted only during graph
building and netlist edits (`Graph::makeEdge`, `graph/Graph.cc:707`); never
from worker threads.
- **Levelization** — `Levelize` runs serially before search (no locks anywhere
in `search/Levelize.cc`); levels are what the barrier discipline of §7.3 is
built on.
- **Sdc / exceptions / clocks** — constraint data is read-only during search
(exception `-thru` lookup in `mutateTag`, derates, etc.); the one lazily
built piece, cycle accounting, is locked (§7.6).
- **Liberty data** — read-only during dcalc/search except the lazy voltage
waveforms noted above.
- **Scenes and Modes** — `scenes_` / `modes_` are created by
`define_scene`-time code on the main thread; parallel phases only read them.
- **Sim (constant propagation) values** — computed when constraints change;
visits read per-pin values, and the shared BDD manager used for function
evaluation is lock-serialized (§7.6).
Network edits and constraint changes therefore never race with search: they run
on the main thread and merely insert into the `invalid_*` sets, which the next
(possibly parallel) pass consumes.
For debugging: `set_thread_count 1` makes all of the above run serially and
deterministically ordered (see §10.7).
---
## 8. Quick reference: what makes two paths "different"
A vertex holds separate arrival/required slots for paths that differ in any of: A vertex holds separate arrival/required slots for paths that differ in any of:
@ -672,7 +845,7 @@ slot and the worst one wins.
--- ---
## 8. Worked example (plain language) ## 9. Worked example (plain language)
Design: input port `in1` with `set_input_delay -clock clk1 2.0`, flop `r1` Design: input port `in1` with `set_input_delay -clock clk1 2.0`, flop `r1`
clocked by propagated `clk1`, two scenes `fast` (mode M, FF libs) and `slow` clocked by propagated `clk1`, two scenes `fast` (mode M, FF libs) and `slow`
@ -698,7 +871,7 @@ reduces across scenes at the end.
--- ---
## 9. Debugging tags: commands and techniques ## 10. Debugging tags: commands and techniques
*Plain language: everything below answers "what paths does this pin actually *Plain language: everything below answers "what paths does this pin actually
hold, and why?" The commands live in the Tcl layer; most are intentionally hold, and why?" The commands live in the Tcl layer; most are intentionally
@ -707,7 +880,7 @@ OpenSTA shell. Arrivals must exist first — run `report_checks` (or anything
that triggers `findArrivals`) or a `find_timing` before inspecting, otherwise that triggers `findArrivals`) or a `find_timing` before inspecting, otherwise
vertices report "no arrivals".* vertices report "no arrivals".*
### 9.1 Report all tags + arrivals/requireds on a pin: `report_tag_arrivals` ### 10.1 Report all tags + arrivals/requireds on a pin: `report_tag_arrivals`
The workhorse. Internal debug proc (`search/Search.tcl:808`, marked "Internal The workhorse. Internal debug proc (`search/Search.tcl:808`, marked "Internal
debugging command", call as `sta::report_tag_arrivals`): debugging command", call as `sta::report_tag_arrivals`):
@ -725,7 +898,7 @@ Vertex r1/Q (fall)
Group 17 Group 17
r max 1.234 / 4.567 23 scene1 clk1 r clk_src ck1buf/Y crpr_pin r1/CLK max ... r max 1.234 / 4.567 23 scene1 clk1 r clk_src ck1buf/Y crpr_pin r1/CLK max ...
^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^
rf mm arrival required tag_index scene <rest = Tag::to_string, see 9.4> rf mm arrival required tag_index scene <rest = Tag::to_string, see 10.4>
``` ```
Format string: `" {rf} {min_max} {arrival} / {required} {tag}"`. So this one Format string: `" {rf} {min_max} {arrival} / {required} {tag}"`. So this one
@ -733,7 +906,7 @@ command answers both "what tags does this pin have" and "what arrival/required
is stored per tag". `Group N` is the vertex's TagGroup number in the shared is stored per tag". `Group N` is the vertex's TagGroup number in the shared
group table. group table.
### 9.2 Report the worst (or every) path through a pin: `report_path` ### 10.2 Report the worst (or every) path through a pin: `report_path`
User-visible command with two hidden flags `-all` and `-tags` User-visible command with two hidden flags `-all` and `-tags`
(`search/Search.tcl:614`, comment: "Note that -all and -tags are intentionally (`search/Search.tcl:614`, comment: "Note that -all and -tags are intentionally
@ -754,7 +927,7 @@ report_path [-min|-max] [-all] [-tags] [-format ...] pin r|f
So `report_path -max -all -tags r1/D f` = every fall/max path class into So `report_path -max -all -tags r1/D f` = every fall/max path class into
`r1/D`, each labeled with its tag. `r1/D`, each labeled with its tag.
### 9.3 Arrival/required/slack per clock edge: `report_arrival` / `report_required` / `report_slack` ### 10.3 Arrival/required/slack per clock edge: `report_arrival` / `report_required` / `report_slack`
User-level, tag-aware but summarized per launching clock edge User-level, tag-aware but summarized per launching clock edge
(`search/Search.tcl:741-802` → `Sta::reportArrivalWrtClks` etc., (`search/Search.tcl:741-802` → `Sta::reportArrivalWrtClks` etc.,
@ -771,7 +944,7 @@ clock edge (the tag's `clkEdge()`), optionally restricted to one scene. Use
these when you want "what are the arrivals wrt each clock" without raw tag these when you want "what are the arrivals wrt each clock" without raw tag
dumps. dumps.
### 9.4 Reading a tag dump: anatomy of `Tag::to_string` ### 10.4 Reading a tag dump: anatomy of `Tag::to_string`
(`search/Tag.cc:86-174`; ClkInfo part `search/ClkInfo.cc:145`.) A line like (`search/Tag.cc:86-174`; ClkInfo part `search/ClkInfo.cc:145`.) A line like
@ -797,7 +970,7 @@ decodes as:
`ClkInfo::to_string` (printed by `report_clk_infos`) additionally shows `ClkInfo::to_string` (printed by `report_clk_infos`) additionally shows
`insert <delay>` (source insertion) and `uncertain <min>:<max>`. `insert <delay>` (source insertion) and `uncertain <min>:<max>`.
### 9.5 Global table dumps and statistics ### 10.5 Global table dumps and statistics
All in `sta::` namespace (`search/Search.i:273-330`): All in `sta::` namespace (`search/Search.i:273-330`):
@ -815,7 +988,7 @@ All in `sta::` namespace (`search/Search.i:273-330`):
Watching `tag_count`/`path_count` before and after a constraint change is the Watching `tag_count`/`path_count` before and after a constraint change is the
quick way to measure tag-explosion cost of exceptions. quick way to measure tag-explosion cost of exceptions.
### 9.6 Scriptable access from Tcl (vertex/path objects) ### 10.6 Scriptable access from Tcl (vertex/path objects)
For ad-hoc debugging scripts (`graph/Graph.i`, `search/Search.i:1212-1285`): For ad-hoc debugging scripts (`graph/Graph.i`, `search/Search.i:1212-1285`):
@ -841,7 +1014,7 @@ foreach vertex [$pin vertices] { ;# bidirect pins have 2 vertices
`find_timing_paths`) expose `path`, `slack`, `margin`, `data_required_time`, `find_timing_paths`) expose `path`, `slack`, `margin`, `data_required_time`,
etc. etc.
### 9.7 `set_debug` trace categories ### 10.7 `set_debug` trace categories
`sta::set_debug <category> <level>` (kept out of the global namespace on `sta::set_debug <category> <level>` (kept out of the global namespace on
purpose — `tcl/Util.tcl:288`). Use `set_thread_count 1` first; parallel BFS purpose — `tcl/Util.tcl:288`). Use `set_thread_count 1` first; parallel BFS
@ -868,7 +1041,7 @@ report_checks -through [get_pins u1/Y] ;# or find_timing
# point where mutateTag kills the tag (false path complete, etc.) # point where mutateTag kills the tag (false path complete, etc.)
``` ```
### 9.8 C++-level hooks (for gdb / temporary instrumentation) ### 10.8 C++-level hooks (for gdb / temporary instrumentation)
- `Search::reportArrivals(vertex, true, digits)` — callable from gdb on any - `Search::reportArrivals(vertex, true, digits)` — callable from gdb on any
vertex. vertex.
@ -885,7 +1058,7 @@ report_checks -through [get_pins u1/Y] ;# or find_timing
--- ---
## 10. Key file index ## 11. Key file index
| Area | Files | | Area | Files |
|---|---| |---|---|
@ -899,3 +1072,4 @@ report_checks -through [get_pins u1/Y] ;# or find_timing
| Scene / Mode | `include/sta/Scene.hh`, `search/Scene.cc`, `include/sta/Mode.hh`, `search/Mode.cc` | | Scene / Mode | `include/sta/Scene.hh`, `search/Scene.cc`, `include/sta/Mode.hh`, `search/Mode.cc` |
| Delay calc | `dcalc/GraphDelayCalc.cc`, `graph/Graph.cc` | | Delay calc | `dcalc/GraphDelayCalc.cc`, `graph/Graph.cc` |
| BFS iterators | `search/Bfs.cc` | | BFS iterators | `search/Bfs.cc` |
| Thread pool / locking primitives | `include/sta/DispatchQueue.hh`, `util/DispatchQueue.cc`, `include/sta/Mutex.hh` |