mirror of
https://github.com/verilator/verilator.git
synced 2026-10-06 10:03:44 +02:00
Optimize bit-scan loops into $mostsetbitp1 / $countones (#7822)
Recognize the common single-bit scan loop idioms in V3Unroll (before it
unrolls) and lower them to bit-reduction primitives, replacing a literal
W-iteration loop with one intrinsic-backed expression:
target=0; for (i=0;i<W;i++) if (vec[i]) target = i + 1; -> $mostsetbitp1(vec)
target=0; for (i=0;i<W;i++) if (vec[i]) target = target + 1; -> $countones(vec)
The leading-one form lowers to a new AstMostSetBitP1 node, emitted as
VL_MOSTSETBITP1_{I,Q,W}; those runtime helpers now use __builtin_clz where
available (same pattern as VL_REDXOR's __builtin_parity), with the existing
bit scan as fallback. The count-ones form reuses AstCountOnes ($countones,
popcount); as the DFG requires a 32-bit countones result it is built at 32
bits and narrowed to the accumulator width with a select.
Matching is structural to stay sound: the index must start at 0, increment
by exactly 1, and scan all W==width(vec) bits via a single 1-bit select of a
distinct vector, with the target pre-zeroed and no else branch. The loop
bound is accepted as a strict ascending 'idx < W' written either way and
signed or unsigned (Gt/GtS/Lt/LtS). Gated by -fbit-scan-loops (on at -O).
Adds t_bit_scan_loops (I/Q/W, count-ones and unsigned-index positives;
step-2, start-1, idx*2+1, vec[idx+1], target=idx and W!=width negatives, all
self-checked and asserted via --stats not to lower) plus t_bit_scan_loops_off
for the disable flag.
Motivated by a transformer inference design whose 80-bit leading-one detector
ran every cycle (~37% of runtime); the lowering is worth ~39% there.
This commit is contained in:
@@ -1464,6 +1464,20 @@ V3Number& V3Number::opCLog2(const V3Number& lhs) {
|
||||
setZero();
|
||||
return *this;
|
||||
}
|
||||
V3Number& V3Number::opMostSetBitP1(const V3Number& lhs) {
|
||||
// Most-significant set bit plus one (bit-width / find-last-set); 0 if value is zero
|
||||
NUM_ASSERT_OP_ARGS1(lhs);
|
||||
NUM_ASSERT_LOGIC_ARGS1(lhs);
|
||||
if (lhs.isFourState()) return setAllBitsX();
|
||||
for (int bit = lhs.width() - 1; bit >= 0; bit--) {
|
||||
if (lhs.bitIs1(bit)) {
|
||||
setLong(bit + 1);
|
||||
return *this;
|
||||
}
|
||||
}
|
||||
setZero();
|
||||
return *this;
|
||||
}
|
||||
|
||||
V3Number& V3Number::opLogNot(const V3Number& lhs) {
|
||||
NUM_ASSERT_OP_ARGS1(lhs);
|
||||
|
||||
Reference in New Issue
Block a user