VlWide4AB prototype for 4-state data

Based on top of #7642. And as discussed in #7193, experiments.

This patch contains 2 parts.

---

The important part:

In the runtime, it introduces `VlWide4AB`, which is is intended to be the
representation of 4-state signals with 32-bit A/B words interleaved
(each 2x32-bit word is an `EData[2] array`, as I'm a bit paranoid about
strict aliasing issues, but could be made a struct with care).
`VlWide4AB` contains `.abits()` and `.bbits()` methods, which yields a
proxy class that through implict constructors can be passed directly to
the existing runtime functions that expect a `WDataInP`/`WDataOutP`.
Whether the `WData*` handle is for a `VlWide` (contiguous), or
`VlWide4AB` is encoded in the spare LSB of the pointer held in the
`WData*` handle. Indexing, arithmetic on the handle considers the
stride, so as long as the runtime only accesses wide state through
`WData*` handles, the routines work for both the 2-state and
4-state-interleaved formats (and the same could be done for
4-state-non-interleaved, or other exotic encodings if necessary.

Example to clarify, though I would recommend a `make test-diff` with
this patch:

```c++
  // This works just like before
  VlWide<10> a2, b2, c2;
  VL_MUL_W(10, a2, b2, c2);

  // This also works
  VlWide4AB<10> a4, b4, c4;
  VL_MUL_W(10, a4.abits(), b4.abits(), c4.abits());

  // But this would also work no problem - note mixed strides/parts:
  VL_MUL_W(10, a2, b4.abits(), c4.bbits());
```

That is, the runtime needs minimal changes. Places where a wide is
accessed without a `WData*` handle or where `WData*::datap()` is called
to get to the raw pointer will need to be fixed. (I will post a separate
patch to add a type-safe template based `scanf` implementation which will
also just work afterwards).

Regarding performance, the encoding is fairly efficient, so impact
should be small, but also most things should be V3Expanded anyway.
If there are performance issues with some, we can always add explicit
variants for those, but I would expect most time to be in the generated
code after V3Expand.

To add runtime functions that take VlWide4AB directly, one would cerate
a `WData4*` handle analogous to `WData`, then can pass the `VlWide4AB`
dierctly without `.abits()`, `.bbits()`. Tracing for example will need
this.

---

The unimportant part:

I hacked up V3Emit and friends to emit `VlWide4AB` for all wide data,
currently always using the `bbits()`, just to test this out. It passes
~94% of the test suite. You can largely ignore this demonstration hack.

---

Continuing the discussion from
https://github.com/verilator/verilator/pull/7193#issuecomment-4444858411
the runtime with the above architecture will work to do 2-state
arithmetic with minimal changes. Verilator internally will do no 4-state
arithmetic after V3Forustate (which needs to run early), which coverts
all 4-state arithmetic into 2-state over abits/bbits like it
currently does.

I came to the conclusion agreeing with Wilson that we will need an
explicit `AstABits`/`AstBBits` LValue type to represent the reference
over aggregate data types (e.g. unpacked arrays of queues of classes
that have associative arrays of 4-state values as members). With the
above runtime, the Emit rule for `AstABits` is simply to print
`.abits()`, and similarly for `AstBBits`, simply print `.bbits()`.

However, we should special case `AstNodeVarRef`, to contain an enum
value, to represent the reference directly when dealing with simple
variable references. (These are the ones that we can optimize
aggressively throughout the compiler). That is, add this to
`AstNodeVarRef`:

```c++
  enum class VarPart : uint8_t {
     WHOLE, // References whole variable, e.g. for `x = y`
     ABITS, // References A bits, e.g `x = p.abits() | q.bbits()`
     BBITS, // References B bits
  };

  class AstNodeVarRef {
        ...
        VarPart m_part
        ...

        VarPart part() const { return m_part; }
        // Encoded reference if needed for map keys - can be other than
        // uintptr_t, but we have a few spare bottom bits in the ptr.
        uintptr_t varpart() const {
          return reinterpret_cast<uintptr_t>(m_varp) | m_part;
        }
  };
```

`AstVarRef(AstVar, VarPart::ABITS)` is then semantically equivalent to
`AstABits(AstVarRef(AstVar, VarPart::WHOLE))`.

(We could special case other LValue nodes if it helps with the
internals, and we will probably want to do e.g. `AstWordSel`, but on the
first try it's probably enoughto have the generic `Ast{A,B}Bits` +
VarRef special.)

Rules to encode in V3Broken:

- After V3FourState, only LValue expressions can have a 4-state type
- `AstABits`/`AstBBits` (and `AstNodeVarRef` with non WHOLE flag) can only
  be applied to a 4-state packed type.
- `AstABits`/`AstBBits (and `AstNodeVarRef` with non WHOLE flag) has a
  2-state type matching it's operand (same width)

include/verilated_types.h

@@ -123,6 +123,23 @@ struct VlIsVlWide : public std::false_type {};
 template <std::size_t N_Words>
 struct VlIsVlWide<VlWide<N_Words>> : public std::true_type {};
 
+// Proxy class to get a WDataInP or WDataOutP from a VlWide4AB for 'A' or 'B' bits.
+// This only exists transiently and is immediately converted to a WDataInP or WDataOutP
+// via an implicit constructor.
+enum class VlWide4ABPart { A_BITS, B_BITS };
+template <VlWide4ABPart Part>
+class VlWide4ABToWData final {
+    template <std::size_t N_Words>
+    friend struct VlWide4AB;
+    friend class WDataOutP;
+    friend class WDataInP;
+
+    EData* const m_datap;
+
+    explicit VlWide4ABToWData(EData* datap) VL_PURE : m_datap{datap} {}
+    explicit VlWide4ABToWData(const EData* datap) VL_PURE : m_datap{const_cast<EData*>(datap)} {}
+};
+
 // Opaque handles to backing store of a VlWide, These are used to pass the
 // backing store to runtime functions without having to template all the
 // runtime functions on the VlWide template patameters. They created via
@@ -135,10 +152,17 @@ struct VlIsVlWide<VlWide<N_Words>> : public std::true_type {};
 
 // Read-Write VlWide handle
 class WDataOutP final {
-    EData* m_datap;  // The backing store - non const as the handle is assignable
+    char* m_datap;  // The backing store - non const as the handle is assignable
+
+    EData* basep() const VL_PURE {
+        return reinterpret_cast<EData*>(reinterpret_cast<uintptr_t>(m_datap) & ~0x1ULL);
+    }
+    size_t scale() const VL_PURE {  //
+        return reinterpret_cast<uintptr_t>(m_datap) & 0x1ULL;
+    }
 
     // Use WDataOutP::external() to create a handle from a raw pointer
-    explicit WDataOutP(EData* datap) VL_PURE : m_datap{datap} {}
+    explicit WDataOutP(EData* datap) VL_PURE : m_datap{reinterpret_cast<char*>(datap)} {}
 
 public:
     // FACTORY METHODS
@@ -149,26 +173,44 @@ public:
     // Implicit conversion from 'VlWide'
     template <std::size_t N_Words>
     // cppcheck-suppress noExplicitConstructor
-    /* implicit */ WDataOutP(VlWide<N_Words>& vlWide) VL_PURE : m_datap{vlWide.data()} {}
+    /* implicit */ WDataOutP(VlWide<N_Words>& vlWide) VL_PURE
-    WDataOutP(const WDataOutP& other) VL_PURE = default;
+        : m_datap{reinterpret_cast<char*>(vlWide.data())} {}
-    WDataOutP(WDataOutP&& other) VL_PURE = default;
+    // Implicit conversion from 'VlWide4ABToWData'
-    WDataOutP& operator=(const WDataOutP& other) VL_PURE = default;
+    template <VlWide4ABPart Part>
+    // cppcheck-suppress noExplicitConstructor
+    /* implicit */ WDataOutP(const VlWide4ABToWData<Part>& tmp) VL_PURE
+        : m_datap{reinterpret_cast<char*>(reinterpret_cast<uintptr_t>(tmp.m_datap) | 1)} {}
+    WDataOutP(const WDataOutP& other) = default;
+    WDataOutP(WDataOutP&& other) = default;
+    WDataOutP& operator=(const WDataOutP& other) = default;
 
     // METHODS
-    EData* datap() const VL_PURE { return m_datap; }
+    EData* datap() const VL_PURE { return reinterpret_cast<EData*>(m_datap); }
-    operator bool() const VL_PURE { return m_datap; }
+    operator bool() const VL_PURE { return basep(); }
-    EData& operator[](size_t index) const VL_PURE { return m_datap[index]; }
+    EData& operator[](std::size_t index) const VL_PURE { return basep()[index << scale()]; }
-    WDataOutP operator+(size_t index) const VL_PURE { return WDataOutP(m_datap + index); }
+    WDataOutP operator+(std::size_t index) const VL_PURE {
-    WDataOutP operator+(int index) const VL_PURE { return WDataOutP(m_datap + index); }
+        return WDataOutP(reinterpret_cast<EData*>(m_datap + (index << scale()) * sizeof(EData)));
+    }
+    WDataOutP operator+(int index) const VL_PURE {
+        return WDataOutP(reinterpret_cast<EData*>(m_datap + (index << scale()) * sizeof(EData)));
+    }
 };
 static_assert(sizeof(WDataOutP) == sizeof(EData*), "WDataOutP should be a single pointer");
 
 // Read-Only VlWide handle
 class WDataInP final {
-    const EData* m_datap;  // The backing store - non const as the handle is assignable
+    const char* m_datap;  // The backing store - non const as the handle is assignable
+
+    const EData* basep() const VL_PURE {
+        return reinterpret_cast<const EData*>(reinterpret_cast<uintptr_t>(m_datap) & ~0x1ULL);
+    }
+    size_t scale() const VL_PURE {  //
+        return reinterpret_cast<uintptr_t>(m_datap) & 0x1ULL;
+    }
 
     // Use WDataInP::external() to create a handle from a raw pointer
-    explicit WDataInP(const EData* datap) VL_PURE : m_datap{datap} {}
+    explicit WDataInP(const EData* datap) VL_PURE : m_datap{reinterpret_cast<const char*>(datap)} {
+    }
 
 public:
     // FACTORY METHODS
@@ -179,26 +221,40 @@ public:
     // Implicit conversion from 'VlWide'
     template <std::size_t N_Words>
     // cppcheck-suppress noExplicitConstructor
-    /* implicit */ WDataInP(VlWide<N_Words>& vlWide) VL_PURE : m_datap{vlWide.data()} {}
+    /* implicit */ WDataInP(VlWide<N_Words>& vlWide) VL_PURE
+        : m_datap{reinterpret_cast<const char*>(vlWide.data())} {}
     // Implicit conversion from 'const VlWide'
     template <std::size_t N_Words>
     // cppcheck-suppress noExplicitConstructor
-    /* implicit */ WDataInP(const VlWide<N_Words>& vlWide) VL_PURE : m_datap{vlWide.data()} {}
+    /* implicit */ WDataInP(const VlWide<N_Words>& vlWide) VL_PURE
-    // Implicit conversion from 'WDataOutP'
+        : m_datap{reinterpret_cast<const char*>(vlWide.data())} {}
+    // Implicit conversion from WDataOutP
     // cppcheck-suppress noExplicitConstructor
-    /* implicit */ WDataInP(const WDataOutP& owp) VL_PURE : m_datap{owp.datap()} {}
+    /* implicit */ WDataInP(const WDataOutP& owp) VL_PURE
-    // Initialize with 'nullptr'
+        : m_datap{reinterpret_cast<const char*>(owp.datap())} {}
-    explicit WDataInP(std::nullptr_t) VL_PURE : m_datap{nullptr} {}
+    explicit WDataInP(std::nullptr_t)  // Initialize to nullptr
-    WDataInP(const WDataInP& other) VL_PURE = default;
+        : m_datap{nullptr} {}
-    WDataInP(WDataInP&& other) VL_PURE = default;
+    // Implicit conversion from VlWide4ABToWData
-    WDataInP& operator=(const WDataInP& other) VL_PURE = default;
+    // cppcheck-suppress noExplicitConstructor
+    template <VlWide4ABPart Part>
+    /* implicit */ WDataInP(const VlWide4ABToWData<Part>& tmp) VL_PURE
+        : m_datap{reinterpret_cast<const char*>(reinterpret_cast<uintptr_t>(tmp.m_datap) | 1)} {}
+    WDataInP(const WDataInP& other) = default;
+    WDataInP(WDataInP&& other) = default;
+    WDataInP& operator=(const WDataInP& other) = default;
 
     // METHODS
-    const EData* datap() const VL_PURE { return m_datap; }
+    const EData* datap() const VL_PURE { return reinterpret_cast<const EData*>(m_datap); }
-    operator bool() const VL_PURE { return m_datap; }
+    operator bool() const VL_PURE { return basep(); }
-    const EData& operator[](size_t index) const VL_PURE { return m_datap[index]; }
+    const EData& operator[](std::size_t index) const VL_PURE { return basep()[index << scale()]; }
-    WDataInP operator+(size_t index) const VL_PURE { return WDataInP(m_datap + index); }
+    WDataInP operator+(std::size_t index) const VL_PURE {
-    WDataInP operator+(int index) const VL_PURE { return WDataInP(m_datap + index); }
+        return WDataInP(
+            reinterpret_cast<const EData*>(m_datap + (index << scale()) * sizeof(EData)));
+    }
+    WDataInP operator+(int index) const VL_PURE {
+        return WDataInP(
+            reinterpret_cast<const EData*>(m_datap + (index << scale()) * sizeof(EData)));
+    }
 };
 static_assert(sizeof(WDataInP) == sizeof(EData*), "WDataInP should be a single pointer");
 
@@ -209,6 +265,51 @@ bool VlWide<N_Words>::operator<(const VlWide<N_Words>& rhs) const VL_PURE {
     return _vl_cmp_w(N_Words, *this, rhs) < 0;
 }
 
+// 4-state VlWide with 'A' and 'B' bits interleaved
+template <std::size_t N_Words>
+struct VlWide4AB final {
+    static constexpr size_t Words = N_Words;
+    using EData4AB = EData[2];
+
+    // MEMBERS
+    // This should be the only data member, otherwise generated static initializers need updating
+    EData4AB m_storage[N_Words];  // Contents of the packed array
+
+    // CONSTRUCTORS
+    // Default constructors and destructor are used. Note however that C++20 requires that
+    // aggregate types do not have a user declared constructor, not even an explicitly defaulted
+    // one.
+
+    // OPERATOR METHODS
+    // Default copy assignment operators are used.
+    bool operator==(const VlWide<N_Words>& that) const VL_PURE {
+        for (size_t i = 0; i < N_Words; ++i) {
+            if (m_storage[i][0] != that.m_storage[i][0]) return false;
+            if (m_storage[i][1] != that.m_storage[i][1]) return false;
+        }
+        return true;
+    }
+    bool operator!=(const VlWide<N_Words>& that) const VL_PURE { return !(*this == that); }
+    EData4AB& operator[](size_t index) { return m_storage[index]; }
+    const EData4AB& operator[](size_t index) const { return m_storage[index]; }
+
+    // METHODS
+    size_t size() const { return N_Words; }
+
+    VlWide4ABToWData<VlWide4ABPart::A_BITS> aBits() {
+        return VlWide4ABToWData<VlWide4ABPart::A_BITS>(&m_storage[0][0]);
+    }
+    VlWide4ABToWData<VlWide4ABPart::B_BITS> bBits() {
+        return VlWide4ABToWData<VlWide4ABPart::B_BITS>(&m_storage[0][1]);
+    }
+    VlWide4ABToWData<VlWide4ABPart::A_BITS> aBits() const {
+        return VlWide4ABToWData<VlWide4ABPart::A_BITS>(&m_storage[0][0]);
+    }
+    VlWide4ABToWData<VlWide4ABPart::B_BITS> bBits() const {
+        return VlWide4ABToWData<VlWide4ABPart::B_BITS>(&m_storage[0][1]);
+    }
+};
+
 //===================================================================
 // String formatters (required by below containers)
 
@@ -233,24 +334,24 @@ inline std::string VL_TO_STRING(const VlWide<N_Words>& obj) {
 #define VL_SIG16(name, msb, lsb) SData name  ///< Declare signal, 9-16 bits
 #define VL_SIG64(name, msb, lsb) QData name  ///< Declare signal, 33-64 bits
 #define VL_SIG(name, msb, lsb) IData name  ///< Declare signal, 17-32 bits
-#define VL_SIGW(name, msb, lsb, words) VlWide<words> name  ///< Declare signal, 65+ bits
+#define VL_SIGW(name, msb, lsb, words) VlWide4AB<words> name  ///< Declare signal, 65+ bits
 #endif
 
 #define VL_IN8(name, msb, lsb) CData name  ///< Declare input signal, 1-8 bits
 #define VL_IN16(name, msb, lsb) SData name  ///< Declare input signal, 9-16 bits
 #define VL_IN64(name, msb, lsb) QData name  ///< Declare input signal, 33-64 bits
 #define VL_IN(name, msb, lsb) IData name  ///< Declare input signal, 17-32 bits
-#define VL_INW(name, msb, lsb, words) VlWide<words> name  ///< Declare input signal, 65+ bits
+#define VL_INW(name, msb, lsb, words) VlWide4AB<words> name  ///< Declare input signal, 65+ bits
 #define VL_INOUT8(name, msb, lsb) CData name  ///< Declare bidir signal, 1-8 bits
 #define VL_INOUT16(name, msb, lsb) SData name  ///< Declare bidir signal, 9-16 bits
 #define VL_INOUT64(name, msb, lsb) QData name  ///< Declare bidir signal, 33-64 bits
 #define VL_INOUT(name, msb, lsb) IData name  ///< Declare bidir signal, 17-32 bits
-#define VL_INOUTW(name, msb, lsb, words) VlWide<words> name  ///< Declare bidir signal, 65+ bits
+#define VL_INOUTW(name, msb, lsb, words) VlWide4AB<words> name  ///< Declare bidir signal, 65+ bits
 #define VL_OUT8(name, msb, lsb) CData name  ///< Declare output signal, 1-8 bits
 #define VL_OUT16(name, msb, lsb) SData name  ///< Declare output signal, 9-16 bits
 #define VL_OUT64(name, msb, lsb) QData name  ///< Declare output signal, 33-64 bits
 #define VL_OUT(name, msb, lsb) IData name  ///< Declare output signal, 17-32 bits
-#define VL_OUTW(name, msb, lsb, words) VlWide<words> name  ///< Declare output signal, 65+ bits
+#define VL_OUTW(name, msb, lsb, words) VlWide4AB<words> name  ///< Declare output signal, 65+ bits
 
 //===================================================================
 // Functions needed here

src/V3AstNodeDType.h

@@ -563,7 +563,7 @@ public:
         if (width <= VL_IDATASIZE) return "IData";
         if (width <= VL_QUADSIZE) return "QData";
 
-        return "VlWide<" + std::to_string(VL_WORDS_I(width)) + ">";
+        return "VlWide4AB<" + std::to_string(VL_WORDS_I(width)) + ">";
     }
 };
 class AstClassRefDType final : public AstNodeDType {

src/V3AstNodes.cpp

@@ -1205,14 +1205,14 @@ AstNodeDType::CTypeRecursed AstNodeDType::cTypeRecurse(bool compound, bool packe
         } else if (dtypep->isQuad()) {
             info.m_type = "QData" + bitvec;
         } else if (dtypep->isWide()) {
-            info.m_type = "VlWide<" + cvtToStr(dtypep->widthWords()) + ">" + bitvec;
+            info.m_type = "VlWide4AB<" + cvtToStr(dtypep->widthWords()) + ">" + bitvec;
         }
         // CData, SData, IData, QData or VlWide are packed type.
         const bool packedType = VString::startsWith(info.m_type, "CData")
                                 || VString::startsWith(info.m_type, "SData")
                                 || VString::startsWith(info.m_type, "IData")
                                 || VString::startsWith(info.m_type, "QData")
-                                || VString::startsWith(info.m_type, "VlWide");
+                                || VString::startsWith(info.m_type, "VlWide4AB");
         UASSERT_OBJ(!packed || packedType, this, "Unsupported type for packed struct or union");
     } else {
         v3fatalSrc("Unknown data type in var type emitter: " << dtypep->prettyName());

src/V3EmitCFunc.cpp

@@ -121,6 +121,7 @@ void EmitCFunc::emitOpName(AstNode* nodep, const string& format, AstNode* lhsp,
                                         m_wideTempRefp->selfPointerProtect(m_useSelfForThis));
                     }
                     out += m_wideTempRefp->varp()->nameProtect();
+                    out += ".bBits()";
                     m_wideTempRefp = nullptr;
                     needComma = true;
                 } else if (usesQueue) {
@@ -623,6 +624,7 @@ string EmitCFunc::emitVarResetRecurse(const AstVar* varp, bool constructing,
                                                  : "VL_SCOPED_RAND_RESET_W(");
                 out += cvtToStr(dtypep->widthMin());
                 out += ", " + varNameProtected + suffix;
+                out += ".bBits()";
                 if (!zeroit) {
                     emitVarResetScopeHash();
                     const uint64_t salt = VString::hashMurmur(varp->prettyName());

src/V3EmitCFunc.h

@@ -121,6 +121,8 @@ class EmitCFunc VL_NOT_FINAL : public EmitCConstInit {
     std::unordered_map<AstJumpBlock*, size_t> m_labelNumbers;  // Label numbers for AstJumpBlocks
     bool m_createdScopeHash = false;  // Already created a scope hash
 
+    bool m_noBits = false;  // Don't emit .bBits() for wide variables
+
 protected:
     VL_DEFINE_DEBUG_FUNCTIONS;
 
@@ -1726,6 +1728,7 @@ public:
             emitDereference(nodep, nodep->selfPointerProtect(m_useSelfForThis));
         }
         putns(nodep, nodep->varp()->nameProtect());
+        if (nodep->isWide() && !m_noBits) puts(".bBits()");
     }
     void visit(AstAddrOfCFunc* nodep) override {
         // Note: Can be thought to handle more, but this is all that is needed right now
@@ -1749,12 +1752,8 @@ public:
         puts(VSelfPointerText::replaceThis(m_useSelfForThis, "this"));
         puts("}");
     }
-    void visit(AstNodeSel* nodep) override {
+    void visit(AstNodeSel* nodep) override { visit(static_cast<AstNodeBiop*>(nodep)); }
-        if (!VN_IS(nodep, ArraySel) && !VN_IS(nodep, WordSel)) {
+    void visit(AstArraySel* nodep) override {
-            visit(static_cast<AstNodeBiop*>(nodep));
-            return;
-        }
-        // ArraySel or WordSel
         iterateAndNextConstNull(nodep->fromp());
         // Special case constant index for readability
         if (AstConst* const idxp = VN_CAST(nodep->bitp(), Const)) {
@@ -1765,6 +1764,21 @@ public:
         iterateAndNextConstNull(nodep->bitp());
         puts("]");
     }
+    void visit(AstWordSel* nodep) override {
+        VL_RESTORER(m_noBits);
+        m_noBits = true;
+        iterateAndNextConstNull(nodep->fromp());
+        // Special case constant index for readability
+        if (AstConst* const idxp = VN_CAST(nodep->bitp(), Const)) {
+            puts("[" + std::to_string(idxp->toUInt()) + "U]");
+            puts("[1]");  // B bits
+            return;
+        }
+        putbs("[");
+        iterateAndNextConstNull(nodep->bitp());
+        puts("]");
+        puts("[1]");  // B bits
+    }
 
     //
     void visit(AstConsAssoc* nodep) override {

src/V3Number.cpp

@@ -966,7 +966,7 @@ string V3Number::emitC() const VL_MT_STABLE {
         // Note the double {{ initializer. The first { starts the initializer of the VlWide,
         // and the second starts the initializer of m_storage within the VlWide.
         // Alternative is to have constructor with std::initializer_list
-        result = "VlWide<" + std::to_string(words()) + ">{{";
+        result = "VlWide4AB<" + std::to_string(words()) + ">{{";
         if (words() > 4) result += '\n';
         for (int n = 0; n < words(); ++n) {
             if (n) result += ((n % 4) ? ", " : ",\n");