Optimize bit-scan loops into $mostsetbitp1 / $countones (#7822)

Recognize the common single-bit scan loop idioms in V3Unroll (before it
unrolls) and lower them to bit-reduction primitives, replacing a literal
W-iteration loop with one intrinsic-backed expression:

  target=0; for (i=0;i<W;i++) if (vec[i]) target = i + 1;      -> $mostsetbitp1(vec)
  target=0; for (i=0;i<W;i++) if (vec[i]) target = target + 1; -> $countones(vec)

The leading-one form lowers to a new AstMostSetBitP1 node, emitted as
VL_MOSTSETBITP1_{I,Q,W}; those runtime helpers now use __builtin_clz where
available (same pattern as VL_REDXOR's __builtin_parity), with the existing
bit scan as fallback.  The count-ones form reuses AstCountOnes ($countones,
popcount); as the DFG requires a 32-bit countones result it is built at 32
bits and narrowed to the accumulator width with a select.

Matching is structural to stay sound: the index must start at 0, increment
by exactly 1, and scan all W==width(vec) bits via a single 1-bit select of a
distinct vector, with the target pre-zeroed and no else branch.  The loop
bound is accepted as a strict ascending 'idx < W' written either way and
signed or unsigned (Gt/GtS/Lt/LtS).  Gated by -fbit-scan-loops (on at -O).

Adds t_bit_scan_loops (I/Q/W, count-ones and unsigned-index positives;
step-2, start-1, idx*2+1, vec[idx+1], target=idx and W!=width negatives, all
self-checked and asserted via --stats not to lower) plus t_bit_scan_loops_off
for the disable flag.

Motivated by a transformer inference design whose 80-bit leading-one detector
ran every cycle (~37% of runtime); the lowering is worth ~39% there.
This commit is contained in:
Thomas Santerre
2026-06-24 10:43:05 +01:00
committed by GitHub
parent 7752625f49
commit bd6b9161dc
13 changed files with 472 additions and 7 deletions
+14
View File
@@ -1464,6 +1464,20 @@ V3Number& V3Number::opCLog2(const V3Number& lhs) {
setZero();
return *this;
}
V3Number& V3Number::opMostSetBitP1(const V3Number& lhs) {
// Most-significant set bit plus one (bit-width / find-last-set); 0 if value is zero
NUM_ASSERT_OP_ARGS1(lhs);
NUM_ASSERT_LOGIC_ARGS1(lhs);
if (lhs.isFourState()) return setAllBitsX();
for (int bit = lhs.width() - 1; bit >= 0; bit--) {
if (lhs.bitIs1(bit)) {
setLong(bit + 1);
return *this;
}
}
setZero();
return *this;
}
V3Number& V3Number::opLogNot(const V3Number& lhs) {
NUM_ASSERT_OP_ARGS1(lhs);