The OrderGraph used during V3Order step deliberately omits some variable
accesses from the dependency graph. E.g.: a read of a variable that is
in the reading block's own hybrid sensitivity list emits no edge, nor
does a read ignored due to a force/release, nor an access to a variable
marked 'ignoreSchedWrite' and friends. For serial mode that is fine, the
logic runs one block at a time. In parallel mode two such blocks can run
concurrently, and if one writes what the other reads, that is a data
race at runtime.
These accesses cannot be recovered from the graph edges. They are now
collected from the AST while the OrderGraph is built, and held by the
OrderLogicVertex performing them.
FixDataHazards is reworked around these access lists stored in
OrderLogicVertex, so it is now aware of all variable accesses the logic
makes, including those not encoded by the dependency graph edges. The
previous heuristic of fixing data hazards by merging same-rank MTasks is
removed. Additional edges are inserted instead to prescribe a fixed
ordering of conflicting MTasks. To insert edges without unduly
increasing the critical path, or introducing cycles, new edges are
added such that they preserve topological ordering, and they are
inserted between vertices sorted by critical path length. See algorithm
details in the code.
Also add a data hazard checker under '--debug-partition', reporting every
unordered accessor pair left in the final MTask graph.
This fixes the race demonstrated by t_sched_hybrid_hazard (#7913),
which is no longer expected to fail.
Under ThreadSanitizer over the vltmt tests: 17 failing before, 3 after,
with no regressions. The 3 remaining are different defects.
This is a large refactor of the MTask graph and coarsening algorithm, in
prep for fixing the bug described in #7913, it can also improve the
resulting multi-threaded schedule.
Two major changes:
OrderMTaskGraph now maintains the critical paths of the MTasks through
mutation. There are 2 ways to mutate the graph, which are done via
methods on the graph itself: adding an edge (used during construction,
and will be used later during fixing data hazards), or merging an MTask
into another (used during contraction). All critical path measures are
automatically updated and propagated on any mutation, so no external
algorithm needs to maintain them explicitly.
The merge candidate scoreboard used during contraction is simplified to
remove deferral of updated scores. This simplifies the code and results
in a greedily more optimal schedule. (The previous tranched rescore was
an optimization to work around the previous std::set based scoreboard,
however since the algorithm now uses an efficient PairingHeap,
verilation time is not impacted by the more accurate scoring, while
yielding better results).
Combining these two into a single patch as the code is highly
interdependent and any one change without the other would be just a
noisy transit point with unclear performance implications. Together it
should be a clear improvement.
Also added a stronger validation step run with '--debug-partition',
which checks all invariants throughout the algorithms.
As the removed comment suggests, this is dubious and RTLMeter suggests
it makes no measurable difference on average (some designs improve, some
regress small amounts, on average ~1.0x)
Prep for further fixes that will remove the pre ranking.
Prep for fixing test added in #7913.
This is a large scale no functional change refactor, however, MT output
is perturbed as tied scores will be broken differently due to ordering
changes (still deterministic).
Split multi-threaded scheduling out of the monolithic
V3OrderParallel.cpp, into relatively independent parts, simplify the
data structures, and drop redundant or unused code.
New translation units:
- V3OrderMTaskGraph.h/.cpp: OrderMTaskGraph, the graph of LogicMTask
vertices and MTaskEdge edges. LogicMTask and MTaskEdge no longer
depend on the coarsening algorithm's merge candidate types;
per-algorithm auxiliary data is attached externally via the vertex and
edge user pointers.
- V3OrderMTaskFixHazards.cpp: data hazard fixup, was FixDataHazards.
- V3OrderMTaskContraction.cpp: graph coarsening, was Partitioner
together with PropagateCp and the merge candidate types.
- V3OrderParallel.cpp: now just the partitioning driver and ExecMTask
graph construction.
Data structure changes:
- Delete V3Scoreboard.h/.cpp. The generic template had a single user, now
a file-local MergeCandidateScoreboard in V3OrderMTaskContraction.cpp.
- Merge candidates are now MergeCandidate/SiblingMC/EdgeMC, distinguished
by a bit in the candidate id rather than by a vtable, and allocated by
the scoreboard, which owns their lifetime. This removes the multiple
inheritance previously used by MTaskEdge.
Move `hashGraphDebug` which prints the hash of a graph's shape for debugging
to generic `V3Graph::hashGraphDebug`.
Removed (can be added back later):
- Unnecesasry self tests that force special data stucture requirements.
- Per stage --stats output under --debug. (Final figures still reported.)
- Various debug dumps
AstMTaskBody is somewhat redundant and is problematic for #6280. We used
to wrap all MTasks in a CFunc before emit anyway. Now we create that
CFunc when we create the ExecMTask in V3OrderParallel, and subsequently
use the CFunc to represent the contents of the MTask. Final output and
optimizations are the same, but internals are simplified to move
towards #6280.
No functional change.
Removed the VlTriggerVec type, and refactored to use an unpacked array
of 64-bit words instead. This means the trigger vector and its
operations are now the same as for any other unpacked array. The few
special functions required for operating on a trigger vector are now
generated in V3SchedTrigger as regular AstCFunc if needed.
No functional change intended, performance should be the same.
The AstIf nodes conditional on events being triggered used to be created
in V3Clock. Now it is in V3Sched*, in order to avoid having to pass
AstActive in CFunc or MTask bodies. No functional change intended, some
improved optimization due to simplifying timing triggers that were
previously missed, also fixes what seems like a bug in the original
timing commit code.
The goal here is to use as single ordering heuristic (which can be
improved later) within MTasks as we do for serial code ordering. The
heuristic itself is factored out into the new OrderMoveGraphSerializer.
This also yields slightly nicer ordering than the previously use
GraphStream, so we end up with fewer trigger (domain) conditionals in
the MTasks, this can be worth a few percent speedup.
This has the somewhat nice side-effect of reusing OrderMoveGraphVertex
for both serial and parallel mode, so MTaskMoveGraphVertex can be
removed.
Serial mode yields identical output.
Instead of carrying around MTask affinity from scheduling, compute it in
V3VariableOrder (where it is used), by tracing through the code. This
simplifies some code and has the benefit of handling variables
introduced after scheduling. It's worth a few % speed at run-time, and
the new implementation of V3VariableOrder is slightly more efficient,
though the speed/space is still dominated by the TSP sort.
There is no strong need to re-map LogicMTask IDs and it just adds extra
processing. Instead we just allocate a separate set of ExecMTask IDs as
they are created, which can also be used as the unique profiling ID as
well. The only effect on the output of this is the change in mtask IDs
emitted, which was fairly arbitrary to begin with.
V3Partition used to contain 2 conceptually separate set of algorithms
- The MTask partitioning/coarsening algorithm used by V3Order. This has
been moved to V3OrderParallel.cpp
- The lowering of AstExecGraph into per thread functions by packing
tasks into threads and creating additional code
(V3Partition::finalize). This has been moved to the new
V3ExecGraph.cpp
This patch is just code movement/rename with minimal fixes required to
do so.
Continuing the idea of decoupling the implementations of the various algorithms.
The main points:
-Move the former "processDomain" stuff, dealing with assigning combinational logic into the relevant sensitivity domains into V3OrderProcessDomains.cpp
-Move the parallel code construction in V3OrderParallel.cpp (Could combine this with some parts of V3Partition - those not called from V3Partition::finalize - but that's not for this patch).
-Move the serial code construction into V3OrderSerial.cpp
-Factored the very small common code between the parallel and serial code construction (processMoveOneLogic) into V3OrderCFuncEmitter.cpp
Geza Lore
Wilson Snyder
Geza Lore
Bartłomiej Chmiel