This is a large refactor of the MTask graph and coarsening algorithm, in
prep for fixing the bug described in #7913, it can also improve the
resulting multi-threaded schedule.
Two major changes:
OrderMTaskGraph now maintains the critical paths of the MTasks through
mutation. There are 2 ways to mutate the graph, which are done via
methods on the graph itself: adding an edge (used during construction,
and will be used later during fixing data hazards), or merging an MTask
into another (used during contraction). All critical path measures are
automatically updated and propagated on any mutation, so no external
algorithm needs to maintain them explicitly.
The merge candidate scoreboard used during contraction is simplified to
remove deferral of updated scores. This simplifies the code and results
in a greedily more optimal schedule. (The previous tranched rescore was
an optimization to work around the previous std::set based scoreboard,
however since the algorithm now uses an efficient PairingHeap,
verilation time is not impacted by the more accurate scoring, while
yielding better results).
Combining these two into a single patch as the code is highly
interdependent and any one change without the other would be just a
noisy transit point with unclear performance implications. Together it
should be a clear improvement.
Also added a stronger validation step run with '--debug-partition',
which checks all invariants throughout the algorithms.
Prep for fixing test added in #7913.
This is a large scale no functional change refactor, however, MT output
is perturbed as tied scores will be broken differently due to ordering
changes (still deterministic).
Split multi-threaded scheduling out of the monolithic
V3OrderParallel.cpp, into relatively independent parts, simplify the
data structures, and drop redundant or unused code.
New translation units:
- V3OrderMTaskGraph.h/.cpp: OrderMTaskGraph, the graph of LogicMTask
vertices and MTaskEdge edges. LogicMTask and MTaskEdge no longer
depend on the coarsening algorithm's merge candidate types;
per-algorithm auxiliary data is attached externally via the vertex and
edge user pointers.
- V3OrderMTaskFixHazards.cpp: data hazard fixup, was FixDataHazards.
- V3OrderMTaskContraction.cpp: graph coarsening, was Partitioner
together with PropagateCp and the merge candidate types.
- V3OrderParallel.cpp: now just the partitioning driver and ExecMTask
graph construction.
Data structure changes:
- Delete V3Scoreboard.h/.cpp. The generic template had a single user, now
a file-local MergeCandidateScoreboard in V3OrderMTaskContraction.cpp.
- Merge candidates are now MergeCandidate/SiblingMC/EdgeMC, distinguished
by a bit in the candidate id rather than by a vtable, and allocated by
the scoreboard, which owns their lifetime. This removes the multiple
inheritance previously used by MTaskEdge.
Move `hashGraphDebug` which prints the hash of a graph's shape for debugging
to generic `V3Graph::hashGraphDebug`.
Removed (can be added back later):
- Unnecesasry self tests that force special data stucture requirements.
- Per stage --stats output under --debug. (Final figures still reported.)
- Various debug dumps
- Rename `--dump-treei` option to `--dumpi-tree`, which itself is now a
special case of `--dumpi-<tag>` where tag can be a magic word, or a
filename
- Control dumping via static `dump*()` functions, analogous to `debug()`
- Make dumping independent of the value of `debug()` (so dumping always
works even without the debug flag)
- Add separate `--dumpi-graph` for dumping V3Graphs, which is again a
special case of `--dumpi-<tag>`
- Alias `--dump-<tag>` to `--dumpi-<tag> 3` as before
Instead of deleting then re-allocating MTaskEdge instances when merging
two MTasks, just redirect the edged of the donor MTask to the recipient
MTask. This is both faster as it avoids an allocation and a deletion,
together with one update of the sibling maps, and also makes the
algorithm more stable due to MergeCandidate IDs being stable and
allocated up front for all MTaskEdges, before any SiblingMCs are
allocated.
Perturbations in output are expected as the IDs used to break ties
between merge candidates with equal costs are not updated when
redirecting an edge (on purpose). The relinking of only one end of the
graph edges also perturbs the order in which they are enumerated, which
does change candidate opportunities when the number of edges is larger
than PART_SIBLING_EDGE_LIMIT. Confirmed output is identical when
IDs are updated and edges are updated to appear in their original order.
This is a simple debugging aid to allow constraining the graph layout
via GraphViz rank directives. Note this is not related in any way to the
vertex 'rank' attribute used by some of the graph algorithms.
No functional change.
Geza Lore
Wilson Snyder
Geza Lore
Mariusz Glebocki
Anthony Donlon
Kamil Rakoczy