luke
/
verilator
mirror of https://github.com/verilator/verilator.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
verilator/src/V3DfgPeephole.cpp

2561 lines
106 KiB
C++
Raw Normal View History

Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
// -*- mode: C++; c-file-style: "cc-mode" -*-
//*************************************************************************
// DESCRIPTION: Verilator: Peephole optimizations over DfgGraph
//
// Code available from: https://verilator.org
//
//*************************************************************************
//
Add SPDX copyright identifiers, and get 'reuse' clean. No functional change.
2026-01-27 02:24:34 +01:00
// This program is free software; you can redistribute it and/or modify it
// under the terms of either the GNU Lesser General Public License Version 3
// or the Perl Artistic License Version 2.0.
// SPDX-FileCopyrightText: 2003-2026 Wilson Snyder
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
// SPDX-License-Identifier: LGPL-3.0-only OR Artistic-2.0
//
//*************************************************************************
//
Optimize DfgPeephole until fixed point (#7309)
2026-03-26 07:56:36 +01:00
// A pattern-matching based optimizer for DfgGraph. This is in some aspects
// similar to V3Const, but more powerful in that it does not care about
// ordering combinational statement. This is also less broadly applicable
// than V3Const, as it does not apply to procedural statements with
// sequential execution semantics.
//
// Each pattern can look at a certain number of source vertices to see
// if a simplified form can be introduced. Some patterns also look at the
// immediate sinks of some vertices. Ideally the algorithm should run to
// fixed point (until nothing else changes). To do this efficiently, two
// lists of vertices are maintained:
// - the 'work' list contains vertices to be considered on the current
// iteration
// - the 'iter' list contains vertices whose whole neighborhood could be
// considered on the next iteration
// The 'work' list ensures simple cascading pattern applications are
// handled in a single pass. The 'iter' list ensures the algorithm runs
// to fixed point if no pattern looks deeper across the graph than the
// neighborhood considered on the next iteration.
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
//
//*************************************************************************
Fix header order botched by clang-format in recent commit.
2023-10-18 12:37:46 +02:00
#include "V3PchAstNoMT.h" // VL_MT_DISABLED_CODE_UNIT
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
#include "V3Dfg.h"
Avoid creating redundant vertices in V3DfgPeephole (#4944) Add a new data-structure V3DfgCache, which can be used to retrieve existing vertices with some given inputs vertices. Use this in V3DfgPeephole to eliminate the creation of redundant vertices. Overall this is performance neutral, but is in prep for some future work.
2024-03-06 19:01:52 +01:00
#include "V3DfgCache.h"
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
#include "V3DfgPasses.h"
Refactor DFG context objects (#6232) - Move All DFG context objects to V3DfgContext.h - Add separate object for ast2dfg and dfg2ast passes - Factor out commonalities No functional change
2025-07-26 21:37:01 +02:00
#include "V3DfgPeepholePatterns.h"
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
#include "V3Stats.h"
Internals: Remove unused vertices immediately in DfgPeephole
2026-03-23 16:06:07 +01:00
#include <algorithm>
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
#include <cctype>
Optimize concatenations that produce unused bits in DFG (#6971) Concatenations that are only used by Sel expressions that do not consume some bits on the edges can be narrowed to not compute the unused bits. E.g.: `{a[4:0], b[4:0]}[5:4]` -> `{a[0], b[4]}[1:0]` This is a superset or the PUSH_SEL_THROUGH_CONCAT DFG pattern, which is removed.
2026-01-31 18:28:14 +01:00
#include <vector>
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
VL_DEFINE_DEBUG_FUNCTIONS;
Refactor DFG context objects (#6232) - Move All DFG context objects to V3DfgContext.h - Add separate object for ast2dfg and dfg2ast passes - Factor out commonalities No functional change
2025-07-26 21:37:01 +02:00
V3DfgPeepholeContext::V3DfgPeepholeContext(V3DfgContext& ctx, const std::string& label)
: V3DfgSubContext{ctx, label, "Peephole"} {
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
const auto checkEnabled = [this](VDfgPeepholePattern id) {
Refactor DFG context objects (#6232) - Move All DFG context objects to V3DfgContext.h - Add separate object for ast2dfg and dfg2ast passes - Factor out commonalities No functional change
2025-07-26 21:37:01 +02:00
std::string str{id.ascii()};
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
std::transform(str.begin(), str.end(), str.begin(), [](unsigned char c) { //
return c == '_' ? '-' : std::tolower(c);
});
m_enabled[id] = v3Global.opt.fDfgPeepholeEnabled(str);
};
#define OPTIMIZATION_CHECK_ENABLED(id, name) checkEnabled(VDfgPeepholePattern::id);
FOR_EACH_DFG_PEEPHOLE_OPTIMIZATION(OPTIMIZATION_CHECK_ENABLED)
#undef OPTIMIZATION_CHECK_ENABLED
}
V3DfgPeepholeContext::~V3DfgPeepholeContext() {
const auto emitStat = [this](VDfgPeepholePattern id) {
Refactor DFG context objects (#6232) - Move All DFG context objects to V3DfgContext.h - Add separate object for ast2dfg and dfg2ast passes - Factor out commonalities No functional change
2025-07-26 21:37:01 +02:00
std::string str{id.ascii()};
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
std::transform(str.begin(), str.end(), str.begin(), [](unsigned char c) { //
return c == '_' ? ' ' : std::tolower(c);
});
Refactor DFG context objects (#6232) - Move All DFG context objects to V3DfgContext.h - Add separate object for ast2dfg and dfg2ast passes - Factor out commonalities No functional change
2025-07-26 21:37:01 +02:00
addStat(str, m_count[id]);
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
};
#define OPTIMIZATION_EMIT_STATS(id, name) emitStat(VDfgPeepholePattern::id);
FOR_EACH_DFG_PEEPHOLE_OPTIMIZATION(OPTIMIZATION_EMIT_STATS)
#undef OPTIMIZATION_EMIT_STATS
}
DFG: Add more peephole patterns
2022-09-30 17:19:53 +02:00
// clang-format off
template <typename T_Reduction>
struct ReductionToBitwiseImpl {};
template <> struct ReductionToBitwiseImpl<DfgRedAnd> { using type = DfgAnd; };
template <> struct ReductionToBitwiseImpl<DfgRedOr> { using type = DfgOr; };
template <> struct ReductionToBitwiseImpl<DfgRedXor> { using type = DfgXor; };
Fix spelling
2023-11-11 05:25:53 +01:00
template <typename T_Reduction>
using ReductionToBitwise = typename ReductionToBitwiseImpl<T_Reduction>::type;
DFG: Add more peephole patterns
2022-09-30 17:19:53 +02:00
template <typename T_Bitwise>
struct BitwiseToReductionImpl {};
template <> struct BitwiseToReductionImpl<DfgAnd> { using type = DfgRedAnd; };
template <> struct BitwiseToReductionImpl<DfgOr> { using type = DfgRedOr; };
template <> struct BitwiseToReductionImpl<DfgXor> { using type = DfgRedXor; };
Fix spelling
2023-11-11 05:25:53 +01:00
template <typename T_Reduction>
using BitwiseToReduction = typename BitwiseToReductionImpl<T_Reduction>::type;
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
namespace {
template<typename Vertex> void foldOp(V3Number& out, const V3Number& src);
template <> void foldOp<DfgExtend> (V3Number& out, const V3Number& src) { out.opAssign(src); }
template <> void foldOp<DfgExtendS> (V3Number& out, const V3Number& src) { out.opExtendS(src, src.width()); }
template <> void foldOp<DfgLogNot> (V3Number& out, const V3Number& src) { out.opLogNot(src); }
template <> void foldOp<DfgNegate> (V3Number& out, const V3Number& src) { out.opNegate(src); }
template <> void foldOp<DfgNot> (V3Number& out, const V3Number& src) { out.opNot(src); }
template <> void foldOp<DfgRedAnd> (V3Number& out, const V3Number& src) { out.opRedAnd(src); }
template <> void foldOp<DfgRedOr> (V3Number& out, const V3Number& src) { out.opRedOr(src); }
template <> void foldOp<DfgRedXor> (V3Number& out, const V3Number& src) { out.opRedXor(src); }
template<typename Vertex> void foldOp(V3Number& out, const V3Number& lhs, const V3Number& rhs);
template <> void foldOp<DfgAdd> (V3Number& out, const V3Number& lhs, const V3Number& rhs) { out.opAdd(lhs, rhs); }
template <> void foldOp<DfgAnd> (V3Number& out, const V3Number& lhs, const V3Number& rhs) { out.opAnd(lhs, rhs); }
template <> void foldOp<DfgConcat> (V3Number& out, const V3Number& lhs, const V3Number& rhs) { out.opConcat(lhs, rhs); }
template <> void foldOp<DfgDiv> (V3Number& out, const V3Number& lhs, const V3Number& rhs) { out.opDiv(lhs, rhs); }
template <> void foldOp<DfgDivS> (V3Number& out, const V3Number& lhs, const V3Number& rhs) { out.opDivS(lhs, rhs); }
template <> void foldOp<DfgEq> (V3Number& out, const V3Number& lhs, const V3Number& rhs) { out.opEq(lhs, rhs); }
template <> void foldOp<DfgGt> (V3Number& out, const V3Number& lhs, const V3Number& rhs) { out.opGt(lhs, rhs); }
template <> void foldOp<DfgGtS> (V3Number& out, const V3Number& lhs, const V3Number& rhs) { out.opGtS(lhs, rhs); }
template <> void foldOp<DfgGte> (V3Number& out, const V3Number& lhs, const V3Number& rhs) { out.opGte(lhs, rhs); }
template <> void foldOp<DfgGteS> (V3Number& out, const V3Number& lhs, const V3Number& rhs) { out.opGteS(lhs, rhs); }
template <> void foldOp<DfgLogAnd> (V3Number& out, const V3Number& lhs, const V3Number& rhs) { out.opLogAnd(lhs, rhs); }
template <> void foldOp<DfgLogEq> (V3Number& out, const V3Number& lhs, const V3Number& rhs) { out.opLogEq(lhs, rhs); }
template <> void foldOp<DfgLogIf> (V3Number& out, const V3Number& lhs, const V3Number& rhs) { out.opLogIf(lhs, rhs); }
template <> void foldOp<DfgLogOr> (V3Number& out, const V3Number& lhs, const V3Number& rhs) { out.opLogOr(lhs, rhs); }
template <> void foldOp<DfgLt> (V3Number& out, const V3Number& lhs, const V3Number& rhs) { out.opLt(lhs, rhs); }
template <> void foldOp<DfgLtS> (V3Number& out, const V3Number& lhs, const V3Number& rhs) { out.opLtS(lhs, rhs); }
template <> void foldOp<DfgLte> (V3Number& out, const V3Number& lhs, const V3Number& rhs) { out.opLte(lhs, rhs); }
Fix folding of LteS in DfgPeephole (#6004) Fixes #6000
2025-05-14 14:33:20 +02:00
template <> void foldOp<DfgLteS> (V3Number& out, const V3Number& lhs, const V3Number& rhs) { out.opLteS(lhs, rhs); }
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
template <> void foldOp<DfgModDiv> (V3Number& out, const V3Number& lhs, const V3Number& rhs) { out.opModDiv(lhs, rhs); }
template <> void foldOp<DfgModDivS> (V3Number& out, const V3Number& lhs, const V3Number& rhs) { out.opModDivS(lhs, rhs); }
template <> void foldOp<DfgMul> (V3Number& out, const V3Number& lhs, const V3Number& rhs) { out.opMul(lhs, rhs); }
template <> void foldOp<DfgMulS> (V3Number& out, const V3Number& lhs, const V3Number& rhs) { out.opMulS(lhs, rhs); }
template <> void foldOp<DfgNeq> (V3Number& out, const V3Number& lhs, const V3Number& rhs) { out.opNeq(lhs, rhs); }
template <> void foldOp<DfgOr> (V3Number& out, const V3Number& lhs, const V3Number& rhs) { out.opOr(lhs, rhs); }
template <> void foldOp<DfgPow> (V3Number& out, const V3Number& lhs, const V3Number& rhs) { out.opPow(lhs, rhs); }
template <> void foldOp<DfgPowSS> (V3Number& out, const V3Number& lhs, const V3Number& rhs) { out.opPowSS(lhs, rhs); }
template <> void foldOp<DfgPowSU> (V3Number& out, const V3Number& lhs, const V3Number& rhs) { out.opPowSU(lhs, rhs); }
template <> void foldOp<DfgPowUS> (V3Number& out, const V3Number& lhs, const V3Number& rhs) { out.opPowUS(lhs, rhs); }
template <> void foldOp<DfgReplicate> (V3Number& out, const V3Number& lhs, const V3Number& rhs) { out.opRepl(lhs, rhs); }
template <> void foldOp<DfgShiftL> (V3Number& out, const V3Number& lhs, const V3Number& rhs) { out.opShiftL(lhs, rhs); }
template <> void foldOp<DfgShiftR> (V3Number& out, const V3Number& lhs, const V3Number& rhs) { out.opShiftR(lhs, rhs); }
template <> void foldOp<DfgShiftRS> (V3Number& out, const V3Number& lhs, const V3Number& rhs) { out.opShiftRS(lhs, rhs, lhs.width()); }
template <> void foldOp<DfgSub> (V3Number& out, const V3Number& lhs, const V3Number& rhs) { out.opSub(lhs, rhs); }
template <> void foldOp<DfgXor> (V3Number& out, const V3Number& lhs, const V3Number& rhs) { out.opXor(lhs, rhs); }
}
DFG: Add more peephole patterns
2022-09-30 17:19:53 +02:00
// clang-format on
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
class V3DfgPeephole final : public DfgVisitor {
Internals: Use a custom worklist in DfgPeephole
2026-03-23 15:41:36 +01:00
// TYPES
struct VertexInfo final {
Optimize DfgPeephole until fixed point (#7309)
2026-03-26 07:56:36 +01:00
size_t m_workListIndex = 0; // Position of this vertx m_workList (0 means not in list)
size_t m_iterListIndex = 0; // Position of this vertx m_iterList (0 means not in list)
size_t m_generation = 0; // Generation number of this vertex - for uniqueness check
size_t m_id = 0; // Unique vertex ID (0 means unassigned) - for sorting
Internals: Use a custom worklist in DfgPeephole
2026-03-23 15:41:36 +01:00
};
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
// STATE
DfgGraph& m_dfg; // The DfgGraph being visited
V3DfgPeepholeContext& m_ctx; // The config structure
Improve Dfg type system (#6390) Added a mini type system for Dfg using DfgDataType to replace Dfg's use of AstNodeDType. This is much more restricted and represents only the types Dfg can handle in a canonical form. This will be needed when adding more support for unpacked arrays and maybe unpacked structs one day. Also added an internal type checker for DfgGraphs which encodes all the assumptions the code makes about type relationships in the graph. Run this in a few places with --debug-check. Fix resulting fallout.
2025-09-07 21:38:50 +02:00
const DfgDataType& m_bitDType = DfgDataType::packed(1); // Common, so grab it up front
Optimize DfgPeephole until fixed point (#7309)
2026-03-26 07:56:36 +01:00
std::vector<DfgVertex*> m_workList; // List of vertices processed in current interation
std::vector<DfgVertex*> m_iterList; // Vertices to start from on next iteration
Internals: Use a custom worklist in DfgPeephole
2026-03-23 15:41:36 +01:00
DfgUserMap<VertexInfo> m_vInfo = m_dfg.makeUserMap<VertexInfo>(); // Map to VertexInfo
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
V3DfgCache m_cache{m_dfg}; // Vertex cache to avoid creating redundant vertices
DfgVertex* m_vtxp = nullptr; // Currently considered vertex
Internals: Remove unused vertices immediately in DfgPeephole
2026-03-23 16:06:07 +01:00
size_t m_currentGeneration = 0; // Current generation number
Optimize commutative vertex operands in Dfg for better combining
2026-03-24 07:53:19 +01:00
size_t m_lastId = 0; // Last unique vertex ID assigned
Internals: Add DfgWorklist commonly used in algorithms (#6361)
2025-09-03 14:55:33 +02:00
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
// STATIC STATE
static V3DebugBisect s_debugBisect; // Debug aid
Internals: Add utility to perform bisection search for debugging (#7294)
2026-03-21 11:13:27 +01:00
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
#define APPLYING(id) if (checkApplying(VDfgPeepholePattern::id))
// METHODS
bool checkApplying(VDfgPeepholePattern id) {
Internals: Add utility to perform bisection search for debugging (#7294)
2026-03-21 11:13:27 +01:00
if (VL_UNLIKELY(!m_ctx.m_enabled[id] || s_debugBisect.isStopped())) return false;
Internals: UINFO now includes newline itself. No functional change.
2025-05-23 02:29:32 +02:00
UINFO(9, "Applying DFG pattern " << id.ascii());
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
++m_ctx.m_count[id];
return true;
}
Internals: Remove unused vertices immediately in DfgPeephole
2026-03-23 16:06:07 +01:00
void incrementGeneration() {
++m_currentGeneration;
// TODO: could sweep on overflow
}
Internals: Use a custom worklist in DfgPeephole
2026-03-23 15:41:36 +01:00
void addToWorkList(DfgVertex* vtxp) {
VertexInfo& vInfo = m_vInfo[*vtxp];
// If already on work list, ignore
if (vInfo.m_workListIndex) return;
// Add to work list
Optimize DfgPeephole until fixed point (#7309)
2026-03-26 07:56:36 +01:00
vInfo.m_workListIndex = m_workList.size();
m_workList.push_back(vtxp);
Internals: Use a custom worklist in DfgPeephole
2026-03-23 15:41:36 +01:00
}
DfgPeephole: Use a work list driven algorithm for speed Replace the 'run to fixed point' algorithm with a work list driven approach. Instead of marking the graph as changed, we explicitly add vertices to the work list, to be visited, when a vertex is changed. This improves both memory locality (as the work list is processed in last in first out order), and removed unnecessary visitations when only a few nodes changes.
2022-11-04 16:50:20 +01:00
Internals: Remove unused vertices immediately in DfgPeephole
2026-03-23 16:06:07 +01:00
void removeFromWorkList(DfgVertex* vtxp) {
VertexInfo& vInfo = m_vInfo[*vtxp];
Optimize DfgPeephole until fixed point (#7309)
2026-03-26 07:56:36 +01:00
// m_workList[0] is always nullptr, fine to assign same here
m_workList[vInfo.m_workListIndex] = nullptr;
Internals: Remove unused vertices immediately in DfgPeephole
2026-03-23 16:06:07 +01:00
vInfo.m_workListIndex = 0;
}
Optimize DfgPeephole until fixed point (#7309)
2026-03-26 07:56:36 +01:00
void addToIterList(DfgVertex* vtxp) {
VertexInfo& vInfo = m_vInfo[*vtxp];
// If already on iter list, ignore
if (vInfo.m_iterListIndex) return;
// Add to iter list
vInfo.m_iterListIndex = m_iterList.size();
m_iterList.push_back(vtxp);
DfgPeephole: Use a work list driven algorithm for speed Replace the 'run to fixed point' algorithm with a work list driven approach. Instead of marking the graph as changed, we explicitly add vertices to the work list, to be visited, when a vertex is changed. This improves both memory locality (as the work list is processed in last in first out order), and removed unnecessary visitations when only a few nodes changes.
2022-11-04 16:50:20 +01:00
}
Optimize DfgPeephole until fixed point (#7309)
2026-03-26 07:56:36 +01:00
void removeFromIterList(DfgVertex* vtxp) {
VertexInfo& vInfo = m_vInfo[*vtxp];
// m_iterList[0] is always nullptr, fine to assign same here
m_iterList[vInfo.m_iterListIndex] = nullptr;
vInfo.m_iterListIndex = 0;
DfgPeephole: Use a work list driven algorithm for speed Replace the 'run to fixed point' algorithm with a work list driven approach. Instead of marking the graph as changed, we explicitly add vertices to the work list, to be visited, when a vertex is changed. This improves both memory locality (as the work list is processed in last in first out order), and removed unnecessary visitations when only a few nodes changes.
2022-11-04 16:50:20 +01:00
}
void deleteVertex(DfgVertex* vtxp) {
Internals: Use a custom worklist in DfgPeephole
2026-03-23 15:41:36 +01:00
UASSERT_OBJ(!m_vInfo[vtxp].m_workListIndex, vtxp, "Deleted Vertex is in work list");
Optimize duplicate vertices in DfgPeephole (#7305)
2026-03-21 23:45:02 +01:00
UASSERT_OBJ(!vtxp->hasSinks(), vtxp, "Should not delete used vertex");
Internals: Remove unused vertices immediately in DfgPeephole
2026-03-23 16:06:07 +01:00
// Invalidate cache entry
m_cache.invalidate(vtxp);
Optimize DfgPeephole until fixed point (#7309)
2026-03-26 07:56:36 +01:00
// It might be in the iter list, remove it
removeFromIterList(vtxp);
Internals: Remove unused vertices immediately in DfgPeephole
2026-03-23 16:06:07 +01:00
// Gather source vertices - they might be duplicates, make unique using generation number
incrementGeneration();
std::vector<DfgVertex*> srcps;
srcps.reserve(vtxp->nInputs());
vtxp->foreachSource([&](DfgVertex& src) {
// If it's a variable, add to work list to see if it became redundant
if (src.is<DfgVertexVar>()) {
addToWorkList(&src);
return false;
}
// Gather unique sources
VertexInfo& vInfo = m_vInfo[src];
if (vInfo.m_generation != m_currentGeneration) srcps.push_back(&src);
vInfo.m_generation = m_currentGeneration;
return false;
});
Optimize: Remove redundant variables during Dfg Peephole pass (#7076) Many rules in the Dfg Peephole pass check if a node has more than one sinks. Redundant variables that will ultimately be removed can prevent these from matching. Remove such variables during the Peeophole pass itself to enable more matches.
2026-02-14 13:33:20 +01:00
// This pass only removes variables that are either not driven in this graph,
// or are not observable outside the graph. If there is also no external write
// to the variable and no references in other graph then delete the Ast var too.
const DfgVertexVar* const varp = vtxp->cast<DfgVertexVar>();
Internals: Remove unused vertices immediately in DfgPeephole
2026-03-23 16:06:07 +01:00
if (varp && !varp->isVolatile() && !varp->hasDfgRefs()) {
AstNode* const nodep = varp->nodep();
VL_DO_DANGLING(vtxp->unlinkDelete(m_dfg), vtxp);
VL_DO_DANGLING(nodep->unlinkFrBack()->deleteTree(), nodep);
} else {
VL_DO_DANGLING(vtxp->unlinkDelete(m_dfg), vtxp);
}
// Partition sources into used/unused after removing the vertex
const auto mid
= std::stable_partition(srcps.begin(), srcps.end(), [](DfgVertex* srcp) { //
return srcp->hasSinks();
});
Optimize DfgPeephole until fixed point (#7309)
2026-03-26 07:56:36 +01:00
// Add used sources to the iter list - their sinks have changed
for (auto it = srcps.begin(); it != mid; ++it) addToIterList(*it);
Internals: Remove unused vertices immediately in DfgPeephole
2026-03-23 16:06:07 +01:00
// Recursively delete unused sources
for (auto it = mid; it != srcps.end(); ++it) {
// Remove from work list
removeFromWorkList(*it);
// Delete vertex
deleteVertex(*it);
}
DfgPeephole: Use a work list driven algorithm for speed Replace the 'run to fixed point' algorithm with a work list driven approach. Instead of marking the graph as changed, we explicitly add vertices to the work list, to be visited, when a vertex is changed. This improves both memory locality (as the work list is processed in last in first out order), and removed unnecessary visitations when only a few nodes changes.
2022-11-04 16:50:20 +01:00
}
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
// Replace 'm_vtxp' with the given vertex
void replace(DfgVertex* resp) {
// Should not be in the work list
Internals: Use a custom worklist in DfgPeephole
2026-03-23 15:41:36 +01:00
UASSERT_OBJ(!m_vInfo[m_vtxp].m_workListIndex, m_vtxp, "Replaced Vertex is in work list");
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
// Debug bisect check
Internals: Add utility to perform bisection search for debugging (#7294)
2026-03-21 11:13:27 +01:00
const auto debugCallback = [&]() -> void {
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
UINFO(0, "Problematic DfgPeephole replacement: " << m_vtxp << " -> " << resp);
m_dfg.sourceCone({m_vtxp, resp});
const auto cone = m_dfg.sourceCone({m_vtxp, resp});
Internals: Add utility to perform bisection search for debugging (#7294)
2026-03-21 11:13:27 +01:00
m_dfg.dumpDotFilePrefixed("peephole-broken", [&](const DfgVertex& v) { //
return cone->count(&v);
});
};
if (VL_UNLIKELY(s_debugBisect.stop(debugCallback))) return;
Optimize DfgPeephole until fixed point (#7309)
2026-03-26 07:56:36 +01:00
// Add sources of the original vertex to the iter list - their sinks are changing
m_vtxp->foreachSource([&](DfgVertex& src) {
addToIterList(&src);
return false;
});
// Remove sinks of the original vertex from the cache - their inputs are changing
Internals: Remove unused vertices immediately in DfgPeephole
2026-03-23 16:06:07 +01:00
m_vtxp->foreachSink([&](DfgVertex& dst) {
m_cache.invalidate(&dst);
Internals: Improve DFG implementation details (#6355) Large scale refactoring to simplify some of the more obtuse internals of DFG. Remove multiple redundant internal APIs, simplify representation of variables, fix potential unsoundness in circular decomposition. No functional change intended.
2025-09-02 17:50:40 +02:00
return false;
});
Optimize duplicate vertices in DfgPeephole (#7305)
2026-03-21 23:45:02 +01:00
// Replace vertex with the replacement
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
m_vtxp->replaceWith(resp);
// Re-cache all sinks of the replacement
Internals: Remove unused vertices immediately in DfgPeephole
2026-03-23 16:06:07 +01:00
resp->foreachSink([&](DfgVertex& dst) {
m_cache.cache(&dst);
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
return false;
});
Internals: Remove unused vertices immediately in DfgPeephole
2026-03-23 16:06:07 +01:00
// Original vertex is now unused, so delete it
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
deleteVertex(m_vtxp);
Internals: Remove unused vertices immediately in DfgPeephole
2026-03-23 16:06:07 +01:00
Optimize DfgPeephole until fixed point (#7309)
2026-03-26 07:56:36 +01:00
// Add new vertex to iter list to consider neighborhood
addToIterList(resp);
// Add new vertex and sinks to work list as likely to match
Internals: Remove unused vertices immediately in DfgPeephole
2026-03-23 16:06:07 +01:00
addToWorkList(resp);
Optimize DfgPeephole until fixed point (#7309)
2026-03-26 07:56:36 +01:00
resp->foreachSink([&](DfgVertex& dst) {
addToWorkList(&dst);
return false;
});
DfgPeephole: Use a work list driven algorithm for speed Replace the 'run to fixed point' algorithm with a work list driven approach. Instead of marking the graph as changed, we explicitly add vertices to the work list, to be visited, when a vertex is changed. This improves both memory locality (as the work list is processed in last in first out order), and removed unnecessary visitations when only a few nodes changes.
2022-11-04 16:50:20 +01:00
}
Make DfgConst hold V3Number directly Remove intermediary AstConst. No functional change intended.
2022-10-07 16:44:14 +02:00
// Create a 32-bit DfgConst vertex
DfgConst* makeI32(FileLine* flp, uint32_t val) { return new DfgConst{m_dfg, flp, 32, val}; }
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
Make DfgConst hold V3Number directly Remove intermediary AstConst. No functional change intended.
2022-10-07 16:44:14 +02:00
// Create a DfgConst vertex with the given width and value zero
Internals: Improve DFG implementation details (#6355) Large scale refactoring to simplify some of the more obtuse internals of DFG. Remove multiple redundant internal APIs, simplify representation of variables, fix potential unsoundness in circular decomposition. No functional change intended.
2025-09-02 17:50:40 +02:00
DfgConst* makeZero(FileLine* flp, uint32_t width) {
return new DfgConst{m_dfg, flp, width, 0};
}
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
DfgPeephole: Use a work list driven algorithm for speed Replace the 'run to fixed point' algorithm with a work list driven approach. Instead of marking the graph as changed, we explicitly add vertices to the work list, to be visited, when a vertex is changed. This improves both memory locality (as the work list is processed in last in first out order), and removed unnecessary visitations when only a few nodes changes.
2022-11-04 16:50:20 +01:00
// Create a new vertex of the given type
Avoid creating redundant vertices in V3DfgPeephole (#4944) Add a new data-structure V3DfgCache, which can be used to retrieve existing vertices with some given inputs vertices. Use this in V3DfgPeephole to eliminate the creation of redundant vertices. Overall this is performance neutral, but is in prep for some future work.
2024-03-06 19:01:52 +01:00
template <typename Vertex, typename... Operands>
Improve Dfg type system (#6390) Added a mini type system for Dfg using DfgDataType to replace Dfg's use of AstNodeDType. This is much more restricted and represents only the types Dfg can handle in a canonical form. This will be needed when adding more support for unpacked arrays and maybe unpacked structs one day. Also added an internal type checker for DfgGraphs which encodes all the assumptions the code makes about type relationships in the graph. Run this in a few places with --debug-check. Fix resulting fallout.
2025-09-07 21:38:50 +02:00
Vertex* make(FileLine* flp, const DfgDataType& dtype, Operands... operands) {
Avoid creating redundant vertices in V3DfgPeephole (#4944) Add a new data-structure V3DfgCache, which can be used to retrieve existing vertices with some given inputs vertices. Use this in V3DfgPeephole to eliminate the creation of redundant vertices. Overall this is performance neutral, but is in prep for some future work.
2024-03-06 19:01:52 +01:00
// Find or create an equivalent vertex
Improve Dfg type system (#6390) Added a mini type system for Dfg using DfgDataType to replace Dfg's use of AstNodeDType. This is much more restricted and represents only the types Dfg can handle in a canonical form. This will be needed when adding more support for unpacked arrays and maybe unpacked structs one day. Also added an internal type checker for DfgGraphs which encodes all the assumptions the code makes about type relationships in the graph. Run this in a few places with --debug-check. Fix resulting fallout.
2025-09-07 21:38:50 +02:00
Vertex* const vtxp = m_cache.getOrCreate<Vertex, Operands...>(flp, dtype, operands...);
Internals: Type check vertices made in DfgPeephole
2026-03-21 11:15:18 +01:00
// Sanity check
UASSERT_OBJ(vtxp->dtype() == dtype, vtxp, "Vertex dtype mismatch");
if (VL_UNLIKELY(v3Global.opt.debugCheck())) vtxp->typeCheck(m_dfg);
Optimize commutative vertex operands in Dfg for better combining
2026-03-24 07:53:19 +01:00
// Assign vertex ID
VertexInfo& vInfo = m_vInfo[vtxp];
if (!vInfo.m_id) vInfo.m_id = ++m_lastId;
Internals: Manage DfgVertex user data via a typed map (#6356) Add DfgUserMap as a handle around the one pointer worth of algorithm specific 'user' storage in each DfgVertex. This reduces verbosity, improves type safety and correctness. Also enables us to remove one pointer from DfgVertex to reduce memory use. No functional change.
2025-09-02 23:21:24 +02:00
// Add to work list
addToWorkList(vtxp);
DfgPeephole: Use a work list driven algorithm for speed Replace the 'run to fixed point' algorithm with a work list driven approach. Instead of marking the graph as changed, we explicitly add vertices to the work list, to be visited, when a vertex is changed. This improves both memory locality (as the work list is processed in last in first out order), and removed unnecessary visitations when only a few nodes changes.
2022-11-04 16:50:20 +01:00
// Return new node
return vtxp;
}
Avoid creating redundant vertices in V3DfgPeephole (#4944) Add a new data-structure V3DfgCache, which can be used to retrieve existing vertices with some given inputs vertices. Use this in V3DfgPeephole to eliminate the creation of redundant vertices. Overall this is performance neutral, but is in prep for some future work.
2024-03-06 19:01:52 +01:00
// Same as above, but 'flp' and 'dtypep' are taken from the given example vertex
template <typename Vertex, typename... Operands>
Internals: Cleanup cppcheck warnings (#6317) `make cppcheck-4` is now clean.
2025-08-20 19:21:24 +02:00
Vertex* make(const DfgVertex* examplep, Operands... operands) {
Improve Dfg type system (#6390) Added a mini type system for Dfg using DfgDataType to replace Dfg's use of AstNodeDType. This is much more restricted and represents only the types Dfg can handle in a canonical form. This will be needed when adding more support for unpacked arrays and maybe unpacked structs one day. Also added an internal type checker for DfgGraphs which encodes all the assumptions the code makes about type relationships in the graph. Run this in a few places with --debug-check. Fix resulting fallout.
2025-09-07 21:38:50 +02:00
return make<Vertex>(examplep->fileline(), examplep->dtype(), operands...);
Avoid creating redundant vertices in V3DfgPeephole (#4944) Add a new data-structure V3DfgCache, which can be used to retrieve existing vertices with some given inputs vertices. Use this in V3DfgPeephole to eliminate the creation of redundant vertices. Overall this is performance neutral, but is in prep for some future work.
2024-03-06 19:01:52 +01:00
}
Add DfgPeephole patterns (#6149)
2025-07-07 17:25:29 +02:00
// Check two vertex are the same, or the same constant value
static bool isSame(const DfgVertex* ap, const DfgVertex* bp) {
if (ap == bp) return true;
Optimize more patterns in DfgPeephole (#7332)
2026-03-27 08:47:50 +01:00
if (ap->dtype() != bp->dtype()) return false;
Add DfgPeephole patterns (#6149)
2025-07-07 17:25:29 +02:00
const DfgConst* const aConstp = ap->cast<DfgConst>();
if (!aConstp) return false;
const DfgConst* const bConstp = bp->cast<DfgConst>();
if (!bConstp) return false;
return aConstp->num().isCaseEq(bConstp->num());
}
Internals: Improve DFG implementation details (#6355) Large scale refactoring to simplify some of the more obtuse internals of DFG. Remove multiple redundant internal APIs, simplify representation of variables, fix potential unsoundness in circular decomposition. No functional change intended.
2025-09-02 17:50:40 +02:00
static bool isZero(const DfgVertex* vtxp) {
if (const DfgConst* const constp = vtxp->cast<DfgConst>()) return constp->isZero();
return false;
}
static bool isOnes(const DfgVertex* vtxp) {
if (const DfgConst* const constp = vtxp->cast<DfgConst>()) return constp->isOnes();
return false;
}
Optimize more patterns in DfgPeephole (#7332)
2026-03-27 08:47:50 +01:00
static bool isEqOne(const DfgVertex* vtxp) {
if (const DfgConst* const constp = vtxp->cast<DfgConst>()) return constp->num().isEqOne();
return false;
}
static bool areAdjacent(uint32_t& lsb, const DfgSel* lSelp, const DfgSel* rSelp) {
if (!isSame(lSelp->srcp(), rSelp->srcp())) return false;
if (lSelp->lsb() + lSelp->width() == rSelp->lsb()) {
lsb = lSelp->lsb();
return true;
}
if (lSelp->lsb() == rSelp->lsb() + rSelp->width()) {
lsb = rSelp->lsb();
return true;
}
return false;
}
Avoid creating redundant vertices in V3DfgPeephole (#4944) Add a new data-structure V3DfgCache, which can be used to retrieve existing vertices with some given inputs vertices. Use this in V3DfgPeephole to eliminate the creation of redundant vertices. Overall this is performance neutral, but is in prep for some future work.
2024-03-06 19:01:52 +01:00
// Note: If any of the following transformers return true, then the vertex was replaced and the
// caller must not do any further changes, so the caller must check the return value, otherwise
// there will be hard to debug issues.
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
// Constant fold unary vertex, return true if folded
template <typename Vertex>
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
VL_ATTR_WARN_UNUSED_RESULT bool foldUnary(Vertex* const vtxp) {
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
static_assert(std::is_base_of<DfgVertexUnary, Vertex>::value, "Must invoke on unary");
static_assert(std::is_final<Vertex>::value, "Must invoke on final class");
if (DfgConst* const srcp = vtxp->srcp()->template cast<DfgConst>()) {
APPLYING(FOLD_UNARY) {
DfgConst* const resultp = makeZero(vtxp->fileline(), vtxp->width());
foldOp<Vertex>(resultp->num(), srcp->num());
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(resultp);
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
return true;
}
}
return false;
}
// Constant fold binary vertex, return true if folded
template <typename Vertex>
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
VL_ATTR_WARN_UNUSED_RESULT bool foldBinary(Vertex* const vtxp) {
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
static_assert(std::is_base_of<DfgVertexBinary, Vertex>::value, "Must invoke on binary");
static_assert(std::is_final<Vertex>::value, "Must invoke on final class");
Internals: Improve DFG implementation details (#6355) Large scale refactoring to simplify some of the more obtuse internals of DFG. Remove multiple redundant internal APIs, simplify representation of variables, fix potential unsoundness in circular decomposition. No functional change intended.
2025-09-02 17:50:40 +02:00
if (DfgConst* const lhsp = vtxp->inputp(0)->template cast<DfgConst>()) {
if (DfgConst* const rhsp = vtxp->inputp(1)->template cast<DfgConst>()) {
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
APPLYING(FOLD_BINARY) {
DfgConst* const resultp = makeZero(vtxp->fileline(), vtxp->width());
foldOp<Vertex>(resultp->num(), lhsp->num(), rhsp->num());
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(resultp);
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
return true;
}
}
}
return false;
}
DfgPeephole: Rework folding of associative operations Allow constant folding through adjacent nodes of all associative operations, for example '((a & 2) & 3)' or '(3 & (2 & a))' can now be folded into '(a & 2)' and '(2 & a)' respectively. Also improve speed of making associative expression trees right leaning by using rotation of the existing vertices whenever instead of allocation of new nodes.
2022-10-05 11:36:28 +02:00
// Transformations that apply to all associative binary vertices.
// Returns true if vtxp was replaced.
template <typename Vertex>
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
VL_ATTR_WARN_UNUSED_RESULT bool associativeBinary(Vertex* const vtxp) {
DfgPeephole: Rework folding of associative operations Allow constant folding through adjacent nodes of all associative operations, for example '((a & 2) & 3)' or '(3 & (2 & a))' can now be folded into '(a & 2)' and '(2 & a)' respectively. Also improve speed of making associative expression trees right leaning by using rotation of the existing vertices whenever instead of allocation of new nodes.
2022-10-05 11:36:28 +02:00
static_assert(std::is_base_of<DfgVertexBinary, Vertex>::value, "Must invoke on binary");
static_assert(std::is_final<Vertex>::value, "Must invoke on final class");
DfgVertex* const lhsp = vtxp->lhsp();
DfgVertex* const rhsp = vtxp->rhsp();
FileLine* const flp = vtxp->fileline();
DfgConst* const lConstp = lhsp->cast<DfgConst>();
DfgConst* const rConstp = rhsp->cast<DfgConst>();
if (lConstp && rConstp) {
APPLYING(FOLD_ASSOC_BINARY) {
DfgConst* const resultp = makeZero(flp, vtxp->width());
foldOp<Vertex>(resultp->num(), lConstp->num(), rConstp->num());
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(resultp);
DfgPeephole: Rework folding of associative operations Allow constant folding through adjacent nodes of all associative operations, for example '((a & 2) & 3)' or '(3 & (2 & a))' can now be folded into '(a & 2)' and '(2 & a)' respectively. Also improve speed of making associative expression trees right leaning by using rotation of the existing vertices whenever instead of allocation of new nodes.
2022-10-05 11:36:28 +02:00
return true;
}
}
if (lConstp) {
if (Vertex* const rVtxp = rhsp->cast<Vertex>()) {
if (DfgConst* const rlConstp = rVtxp->lhsp()->template cast<DfgConst>()) {
APPLYING(FOLD_ASSOC_BINARY_LHS_OF_RHS) {
// Fold constants
const uint32_t width = std::is_same<DfgConcat, Vertex>::value
? lConstp->width() + rlConstp->width()
: vtxp->width();
DfgConst* const constp = makeZero(flp, width);
foldOp<Vertex>(constp->num(), lConstp->num(), rlConstp->num());
// Replace vertex
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(make<Vertex>(vtxp, constp, rVtxp->rhsp()));
Avoid creating redundant vertices in V3DfgPeephole (#4944) Add a new data-structure V3DfgCache, which can be used to retrieve existing vertices with some given inputs vertices. Use this in V3DfgPeephole to eliminate the creation of redundant vertices. Overall this is performance neutral, but is in prep for some future work.
2024-03-06 19:01:52 +01:00
return true;
DfgPeephole: Rework folding of associative operations Allow constant folding through adjacent nodes of all associative operations, for example '((a & 2) & 3)' or '(3 & (2 & a))' can now be folded into '(a & 2)' and '(2 & a)' respectively. Also improve speed of making associative expression trees right leaning by using rotation of the existing vertices whenever instead of allocation of new nodes.
2022-10-05 11:36:28 +02:00
}
}
}
}
if (rConstp) {
if (Vertex* const lVtxp = lhsp->cast<Vertex>()) {
if (DfgConst* const lrConstp = lVtxp->rhsp()->template cast<DfgConst>()) {
APPLYING(FOLD_ASSOC_BINARY_RHS_OF_LHS) {
// Fold constants
const uint32_t width = std::is_same<DfgConcat, Vertex>::value
? lrConstp->width() + rConstp->width()
: vtxp->width();
DfgConst* const constp = makeZero(flp, width);
foldOp<Vertex>(constp->num(), lrConstp->num(), rConstp->num());
// Replace vertex
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(make<Vertex>(vtxp, lVtxp->lhsp(), constp));
Avoid creating redundant vertices in V3DfgPeephole (#4944) Add a new data-structure V3DfgCache, which can be used to retrieve existing vertices with some given inputs vertices. Use this in V3DfgPeephole to eliminate the creation of redundant vertices. Overall this is performance neutral, but is in prep for some future work.
2024-03-06 19:01:52 +01:00
return true;
DfgPeephole: Rework folding of associative operations Allow constant folding through adjacent nodes of all associative operations, for example '((a & 2) & 3)' or '(3 & (2 & a))' can now be folded into '(a & 2)' and '(2 & a)' respectively. Also improve speed of making associative expression trees right leaning by using rotation of the existing vertices whenever instead of allocation of new nodes.
2022-10-05 11:36:28 +02:00
}
}
}
}
// Make associative trees right leaning to reduce pattern variations, and for better CSE
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
if (Vertex* const alhsp = vtxp->lhsp()->template cast<Vertex>()) {
if (!alhsp->hasMultipleSinks()) {
Optimize more patterns in DfgPeephole (#7332)
2026-03-27 08:47:50 +01:00
DfgVertex* const ap = alhsp->lhsp();
DfgVertex* const bp = alhsp->rhsp();
DfgVertex* const cp = vtxp->rhsp();
// Only do this if the rhs is not th same as the operands of the LHS, otherwise
// SWAP_SIDE_IN_COMMUTATIVE_BINARY can get in a loop with this pattern.
if (ap != cp && bp != cp) {
APPLYING(RIGHT_LEANING_ASSOC) {
// Rotate the expression tree rooted at 'vtxp' to the right,
// producing a right-leaning tree
// Concatenation dtypes need adjusting, other assoc vertices preserve types
const DfgDataType& childDType
= std::is_same<Vertex, DfgConcat>::value
? DfgDataType::packed(bp->width() + cp->width())
: vtxp->dtype();
Vertex* const bcp = make<Vertex>(vtxp->fileline(), childDType, bp, cp);
replace(make<Vertex>(alhsp->fileline(), vtxp->dtype(), ap, bcp));
return true;
}
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
}
DfgPeephole: Rework folding of associative operations Allow constant folding through adjacent nodes of all associative operations, for example '((a & 2) & 3)' or '(3 & (2 & a))' can now be folded into '(a & 2)' and '(2 & a)' respectively. Also improve speed of making associative expression trees right leaning by using rotation of the existing vertices whenever instead of allocation of new nodes.
2022-10-05 11:36:28 +02:00
}
}
Optimize reuse of existing associative terms in DfgPeephole Enable V3DfgCache to look up vertices without creating one. Reuse existing terms in associative expression trees if they already exist somewhere in the graph.
2026-03-17 09:20:00 +01:00
Optimize commutative vertex operands in Dfg for better combining
2026-03-24 07:53:19 +01:00
// Attempt to reuse associative binary expressions if hey already exist, e.g.:
// '(a OP (b OP c))' -> '(a OP b) OP c', iff '(a OP b)' already exists, or
// '(a OP c) OP b' iff '(a OP c)' already exists and the vertex is commutative.
// Only do this is 'b OP c' has a single use and can subsequently be removed,
// otherwise there is no improvement.
if (!rhsp->hasMultipleSinks()) {
if (Vertex* rVtxp = rhsp->template cast<Vertex>()) {
DfgVertex* const rlVtxp = rVtxp->lhsp();
DfgVertex* const rrVtxp = rVtxp->rhsp();
const DfgDataType& dtype
= std::is_same<Vertex, DfgConcat>::value
? DfgDataType::packed(lhsp->width() + rlVtxp->width())
: vtxp->dtype();
if (Vertex* const existingp = m_cache.get<Vertex>(dtype, lhsp, rlVtxp)) {
UASSERT_OBJ(existingp->hasSinks(), vtxp, "Existing vertex should be used");
if (existingp != rhsp) {
APPLYING(REUSE_ASSOC_BINARY_LHS_WITH_LHS_OF_RHS) {
replace(make<Vertex>(vtxp, existingp, rrVtxp));
return true;
}
}
}
// Concat is not commutative
if VL_CONSTEXPR_CXX17 (!std::is_same<Vertex, DfgConcat>::value) {
if (Vertex* const existingp = m_cache.get<Vertex>(dtype, lhsp, rrVtxp)) {
UASSERT_OBJ(existingp->hasSinks(), vtxp, "Existing vertex should be used");
if (existingp != rhsp) {
APPLYING(REUSE_ASSOC_BINARY_LHS_WITH_RHS_OF_RHS) {
replace(make<Vertex>(vtxp, existingp, rlVtxp));
return true;
}
}
Optimize reuse of existing associative terms in DfgPeephole Enable V3DfgCache to look up vertices without creating one. Reuse existing terms in associative expression trees if they already exist somewhere in the graph.
2026-03-17 09:20:00 +01:00
}
}
}
}
DfgPeephole: Rework folding of associative operations Allow constant folding through adjacent nodes of all associative operations, for example '((a & 2) & 3)' or '(3 & (2 & a))' can now be folded into '(a & 2)' and '(2 & a)' respectively. Also improve speed of making associative expression trees right leaning by using rotation of the existing vertices whenever instead of allocation of new nodes.
2022-10-05 11:36:28 +02:00
Optimize reuse of existing associative terms in DfgPeephole Enable V3DfgCache to look up vertices without creating one. Reuse existing terms in associative expression trees if they already exist somewhere in the graph.
2026-03-17 09:20:00 +01:00
return false;
DfgPeephole: Rework folding of associative operations Allow constant folding through adjacent nodes of all associative operations, for example '((a & 2) & 3)' or '(3 & (2 & a))' can now be folded into '(a & 2)' and '(2 & a)' respectively. Also improve speed of making associative expression trees right leaning by using rotation of the existing vertices whenever instead of allocation of new nodes.
2022-10-05 11:36:28 +02:00
}
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
// Transformations that apply to all commutative binary vertices
Avoid creating redundant vertices in V3DfgPeephole (#4944) Add a new data-structure V3DfgCache, which can be used to retrieve existing vertices with some given inputs vertices. Use this in V3DfgPeephole to eliminate the creation of redundant vertices. Overall this is performance neutral, but is in prep for some future work.
2024-03-06 19:01:52 +01:00
template <typename Vertex>
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
VL_ATTR_WARN_UNUSED_RESULT bool commutativeBinary(Vertex* const vtxp) {
Avoid creating redundant vertices in V3DfgPeephole (#4944) Add a new data-structure V3DfgCache, which can be used to retrieve existing vertices with some given inputs vertices. Use this in V3DfgPeephole to eliminate the creation of redundant vertices. Overall this is performance neutral, but is in prep for some future work.
2024-03-06 19:01:52 +01:00
static_assert(std::is_base_of<DfgVertexBinary, Vertex>::value, "Must invoke on binary");
static_assert(std::is_final<Vertex>::value, "Must invoke on final class");
DfgVertex* const lhsp = vtxp->lhsp();
DfgVertex* const rhsp = vtxp->rhsp();
Optimize commutative vertex operands in Dfg for better combining
2026-03-24 07:53:19 +01:00
Optimize more patterns in DfgPeephole (#7332)
2026-03-27 08:47:50 +01:00
// If constant LHS, push through cond if used once. (Enables branch combining)
if (DfgConst* const lConstp = lhsp->cast<DfgConst>()) {
if (DfgCond* const rCondp = rhsp->cast<DfgCond>()) {
if (!rCondp->hasMultipleSinks()) {
APPLYING(PUSH_COMMUTATIVE_BINARY_THROUGH_COND) {
DfgVertex* const tp = make<Vertex>(vtxp, lConstp, rCondp->thenp());
DfgVertex* const ep = make<Vertex>(vtxp, lConstp, rCondp->elsep());
replace(make<DfgCond>(vtxp, rCondp->condp(), tp, ep));
return true;
}
}
}
return false;
}
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
// Ensure Const is on left-hand side to simplify other patterns
Optimize commutative vertex operands in Dfg for better combining
2026-03-24 07:53:19 +01:00
{
const bool lIsConst = lhsp->is<DfgConst>();
const bool rIsConst = rhsp->is<DfgConst>();
if (lIsConst != rIsConst) {
if (rIsConst) {
APPLYING(SWAP_CONST_IN_COMMUTATIVE_BINARY) {
replace(make<Vertex>(vtxp, rhsp, lhsp));
return true;
}
}
return false;
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
}
}
Optimize commutative vertex operands in Dfg for better combining
2026-03-24 07:53:19 +01:00
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
// Ensure Not is on the left-hand side to simplify other patterns
Optimize commutative vertex operands in Dfg for better combining
2026-03-24 07:53:19 +01:00
{
const bool lIsNot = lhsp->is<DfgNot>();
const bool rIsNot = rhsp->is<DfgNot>();
if (lIsNot != rIsNot) {
if (rIsNot) {
APPLYING(SWAP_NOT_IN_COMMUTATIVE_BINARY) {
replace(make<Vertex>(vtxp, rhsp, lhsp));
return true;
}
}
return false;
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
}
}
Optimize commutative vertex operands in Dfg for better combining
2026-03-24 07:53:19 +01:00
// Ensure same vertex is on the right-hand side to simplify other patterns
{
const bool lIsSame = lhsp->is<Vertex>();
const bool rIsSame = rhsp->is<Vertex>();
if (lIsSame != rIsSame) {
if (lIsSame) {
APPLYING(SWAP_SAME_IN_COMMUTATIVE_BINARY) {
replace(make<Vertex>(vtxp, rhsp, lhsp));
return true;
}
}
return false;
}
}
// Otherwise put sides in order based on unique iD, this makes
// 'a op b' and 'b op a' end up the same for better combining.
{
const VertexInfo& lInfo = m_vInfo[lhsp];
const VertexInfo& rInfo = m_vInfo[rhsp];
if (lInfo.m_id > rInfo.m_id) {
APPLYING(SWAP_SIDE_IN_COMMUTATIVE_BINARY) {
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(make<Vertex>(vtxp, rhsp, lhsp));
Avoid creating redundant vertices in V3DfgPeephole (#4944) Add a new data-structure V3DfgCache, which can be used to retrieve existing vertices with some given inputs vertices. Use this in V3DfgPeephole to eliminate the creation of redundant vertices. Overall this is performance neutral, but is in prep for some future work.
2024-03-06 19:01:52 +01:00
return true;
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
}
}
}
Avoid creating redundant vertices in V3DfgPeephole (#4944) Add a new data-structure V3DfgCache, which can be used to retrieve existing vertices with some given inputs vertices. Use this in V3DfgPeephole to eliminate the creation of redundant vertices. Overall this is performance neutral, but is in prep for some future work.
2024-03-06 19:01:52 +01:00
return false;
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
}
Add DfgPeephole patterns (#6149)
2025-07-07 17:25:29 +02:00
// Transformations that apply to all distributive and associative binary
// vertices 'Other' is the type that is distributive over 'Vertex',
// that is: a Other (b Vertex c) == (a Other b) Vertex (a Other c)
template <typename Other, typename Vertex>
VL_ATTR_WARN_UNUSED_RESULT bool distributiveAndAssociativeBinary(Vertex* vtxp) {
DfgVertex* const lhsp = vtxp->lhsp();
DfgVertex* const rhsp = vtxp->rhsp();
if (!lhsp->hasMultipleSinks() && !rhsp->hasMultipleSinks()) {
// Convert '(a Other b) Vertex (a Other c)' and associative
// variations to 'a Other (b Vertex c)'
if (Other* const lp = lhsp->cast<Other>()) {
if (Other* const rp = rhsp->cast<Other>()) {
DfgVertex* const llp = lp->lhsp();
DfgVertex* const lrp = lp->rhsp();
DfgVertex* const rlp = rp->lhsp();
DfgVertex* const rrp = rp->rhsp();
DfgVertex* ap = nullptr;
DfgVertex* bp = nullptr;
DfgVertex* cp = nullptr;
if (llp == rlp) {
ap = llp;
bp = lrp;
cp = rrp;
} else if (llp == rrp) {
ap = llp;
bp = lrp;
cp = rlp;
} else if (lrp == rlp) {
ap = lrp;
bp = llp;
cp = rrp;
} else if (lrp == rrp) {
ap = lrp;
bp = llp;
cp = rlp;
}
if (ap) {
APPLYING(REPLACE_DISTRIBUTIVE_BINARY) {
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(make<Other>(vtxp, ap, make<Vertex>(lhsp, bp, cp)));
Add DfgPeephole patterns (#6149)
2025-07-07 17:25:29 +02:00
return true;
}
}
}
}
}
return false;
}
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
// Bitwise operation with one side Const, and the other side a Concat
template <typename Vertex>
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
VL_ATTR_WARN_UNUSED_RESULT bool
tryPushBitwiseOpThroughConcat(Vertex* const vtxp, DfgConst* constp, DfgConcat* concatp) {
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
FileLine* const flp = vtxp->fileline();
// If at least one of the sides of the Concat constant, or width 1 (i.e.: can be
// further simplified), then push the Vertex past the Concat
if (concatp->lhsp()->is<DfgConst>() || concatp->rhsp()->is<DfgConst>() //
Improve Dfg type system (#6390) Added a mini type system for Dfg using DfgDataType to replace Dfg's use of AstNodeDType. This is much more restricted and represents only the types Dfg can handle in a canonical form. This will be needed when adding more support for unpacked arrays and maybe unpacked structs one day. Also added an internal type checker for DfgGraphs which encodes all the assumptions the code makes about type relationships in the graph. Run this in a few places with --debug-check. Fix resulting fallout.
2025-09-07 21:38:50 +02:00
|| concatp->lhsp()->dtype() == m_bitDType || concatp->rhsp()->dtype() == m_bitDType) {
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
APPLYING(PUSH_BITWISE_OP_THROUGH_CONCAT) {
const uint32_t width = concatp->width();
Improve Dfg type system (#6390) Added a mini type system for Dfg using DfgDataType to replace Dfg's use of AstNodeDType. This is much more restricted and represents only the types Dfg can handle in a canonical form. This will be needed when adding more support for unpacked arrays and maybe unpacked structs one day. Also added an internal type checker for DfgGraphs which encodes all the assumptions the code makes about type relationships in the graph. Run this in a few places with --debug-check. Fix resulting fallout.
2025-09-07 21:38:50 +02:00
const DfgDataType& lDtype = concatp->lhsp()->dtype();
const DfgDataType& rDtype = concatp->rhsp()->dtype();
const uint32_t lWidth = lDtype.size();
const uint32_t rWidth = rDtype.size();
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
// The new Lhs vertex
DfgConst* const newLhsConstp = makeZero(constp->fileline(), lWidth);
newLhsConstp->num().opSel(constp->num(), width - 1, rWidth);
Improve Dfg type system (#6390) Added a mini type system for Dfg using DfgDataType to replace Dfg's use of AstNodeDType. This is much more restricted and represents only the types Dfg can handle in a canonical form. This will be needed when adding more support for unpacked arrays and maybe unpacked structs one day. Also added an internal type checker for DfgGraphs which encodes all the assumptions the code makes about type relationships in the graph. Run this in a few places with --debug-check. Fix resulting fallout.
2025-09-07 21:38:50 +02:00
Vertex* const newLhsp = make<Vertex>(flp, lDtype, newLhsConstp, concatp->lhsp());
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
// The new Rhs vertex
DfgConst* const newRhsConstp = makeZero(constp->fileline(), rWidth);
newRhsConstp->num().opSel(constp->num(), rWidth - 1, 0);
Improve Dfg type system (#6390) Added a mini type system for Dfg using DfgDataType to replace Dfg's use of AstNodeDType. This is much more restricted and represents only the types Dfg can handle in a canonical form. This will be needed when adding more support for unpacked arrays and maybe unpacked structs one day. Also added an internal type checker for DfgGraphs which encodes all the assumptions the code makes about type relationships in the graph. Run this in a few places with --debug-check. Fix resulting fallout.
2025-09-07 21:38:50 +02:00
Vertex* const newRhsp = make<Vertex>(flp, rDtype, newRhsConstp, concatp->rhsp());
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
// Replace this vertex
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(make<DfgConcat>(concatp, newLhsp, newRhsp));
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
return true;
}
}
return false;
}
template <typename Vertex>
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
VL_ATTR_WARN_UNUSED_RESULT bool
tryPushCompareOpThroughConcat(Vertex* const vtxp, DfgConst* constp, DfgConcat* concatp) {
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
FileLine* const flp = vtxp->fileline();
Avoid creating redundant vertices in V3DfgPeephole (#4944) Add a new data-structure V3DfgCache, which can be used to retrieve existing vertices with some given inputs vertices. Use this in V3DfgPeephole to eliminate the creation of redundant vertices. Overall this is performance neutral, but is in prep for some future work.
2024-03-06 19:01:52 +01:00
// If at least one of the sides of the Concat is constant, then push the Vertex past
// the Concat
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
if (concatp->lhsp()->is<DfgConst>() || concatp->rhsp()->is<DfgConst>()) {
APPLYING(PUSH_COMPARE_OP_THROUGH_CONCAT) {
const uint32_t width = concatp->width();
const uint32_t lWidth = concatp->lhsp()->width();
const uint32_t rWidth = concatp->rhsp()->width();
// The new Lhs vertex
DfgConst* const newLhsConstp = makeZero(constp->fileline(), lWidth);
newLhsConstp->num().opSel(constp->num(), width - 1, rWidth);
Avoid creating redundant vertices in V3DfgPeephole (#4944) Add a new data-structure V3DfgCache, which can be used to retrieve existing vertices with some given inputs vertices. Use this in V3DfgPeephole to eliminate the creation of redundant vertices. Overall this is performance neutral, but is in prep for some future work.
2024-03-06 19:01:52 +01:00
Vertex* const newLhsp
= make<Vertex>(flp, m_bitDType, newLhsConstp, concatp->lhsp());
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
// The new Rhs vertex
DfgConst* const newRhsConstp = makeZero(constp->fileline(), rWidth);
newRhsConstp->num().opSel(constp->num(), rWidth - 1, 0);
Avoid creating redundant vertices in V3DfgPeephole (#4944) Add a new data-structure V3DfgCache, which can be used to retrieve existing vertices with some given inputs vertices. Use this in V3DfgPeephole to eliminate the creation of redundant vertices. Overall this is performance neutral, but is in prep for some future work.
2024-03-06 19:01:52 +01:00
Vertex* const newRhsp
= make<Vertex>(flp, m_bitDType, newRhsConstp, concatp->rhsp());
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
// The replacement Vertex
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
DfgVertexBinary* const resp
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
= std::is_same<Vertex, DfgEq>::value
Avoid creating redundant vertices in V3DfgPeephole (#4944) Add a new data-structure V3DfgCache, which can be used to retrieve existing vertices with some given inputs vertices. Use this in V3DfgPeephole to eliminate the creation of redundant vertices. Overall this is performance neutral, but is in prep for some future work.
2024-03-06 19:01:52 +01:00
? make<DfgAnd>(concatp->fileline(), m_bitDType, newLhsp, newRhsp)
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
: nullptr;
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
UASSERT_OBJ(resp, vtxp,
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
"Unhandled vertex type in 'tryPushCompareOpThroughConcat': "
<< vtxp->typeName());
// Replace this vertex
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(resp);
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
return true;
}
}
return false;
}
DFG: Add more peephole patterns
2022-09-30 17:19:53 +02:00
template <typename Bitwise>
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
VL_ATTR_WARN_UNUSED_RESULT bool tryPushBitwiseOpThroughReductions(Bitwise* const vtxp) {
DFG: Add more peephole patterns
2022-09-30 17:19:53 +02:00
using Reduction = BitwiseToReduction<Bitwise>;
if (Reduction* const lRedp = vtxp->lhsp()->template cast<Reduction>()) {
if (Reduction* const rRedp = vtxp->rhsp()->template cast<Reduction>()) {
DfgVertex* const lSrcp = lRedp->srcp();
DfgVertex* const rSrcp = rRedp->srcp();
Optimize more patterns in DfgPeephole (#7332)
2026-03-27 08:47:50 +01:00
if (lSrcp->dtype() == rSrcp->dtype() && lSrcp->width() <= VL_QUADSIZE
&& !lSrcp->is<DfgConcat>() && !rSrcp->is<DfgConcat>()
Try to ensure DFG peephole patterns don't grow the graph Some optimizations are only a net win if they help us remove a graph node (or at least ensure they don't grow the graph), or yields otherwise special logic, so try to apply them only in these cases.
2022-10-04 19:26:53 +02:00
&& !lSrcp->hasMultipleSinks() && !rSrcp->hasMultipleSinks()) {
DFG: Add more peephole patterns
2022-09-30 17:19:53 +02:00
APPLYING(PUSH_BITWISE_THROUGH_REDUCTION) {
FileLine* const flp = vtxp->fileline();
Improve Dfg type system (#6390) Added a mini type system for Dfg using DfgDataType to replace Dfg's use of AstNodeDType. This is much more restricted and represents only the types Dfg can handle in a canonical form. This will be needed when adding more support for unpacked arrays and maybe unpacked structs one day. Also added an internal type checker for DfgGraphs which encodes all the assumptions the code makes about type relationships in the graph. Run this in a few places with --debug-check. Fix resulting fallout.
2025-09-07 21:38:50 +02:00
Bitwise* const bwp = make<Bitwise>(flp, lSrcp->dtype(), lSrcp, rSrcp);
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(make<Reduction>(flp, m_bitDType, bwp));
DFG: Add more peephole patterns
2022-09-30 17:19:53 +02:00
return true;
}
}
}
}
return false;
}
Optimize more patterns in DfgPeephole (#7332)
2026-03-27 08:47:50 +01:00
template <typename Bitwise>
VL_ATTR_WARN_UNUSED_RESULT bool tryReplaceBitwiseWithReduction(Bitwise* vtxp) {
UASSERT_OBJ(vtxp->width() == 1, vtxp, "Width must be 1");
using Reduction = BitwiseToReduction<Bitwise>;
DfgVertex* const lhsp = vtxp->lhsp();
DfgVertex* const rhsp = vtxp->rhsp();
if (DfgSel* const lSelp = lhsp->template cast<DfgSel>()) {
DfgSel* rSelp = rhsp->template cast<DfgSel>();
DfgVertex* extrap = nullptr;
if (!rSelp) {
if (Bitwise* const rBitwisep = rhsp->template cast<Bitwise>()) {
rSelp = rBitwisep->lhsp()->template cast<DfgSel>();
extrap = rBitwisep->rhsp();
}
}
if (rSelp) {
uint32_t lsb = 0;
if (areAdjacent(lsb, lSelp, rSelp)) {
APPLYING(REPLACE_BITWISE_OF_SELS_WITH_REDUCTION) {
const DfgDataType& dtype
= DfgDataType::packed(lSelp->width() + rSelp->width());
DfgSel* const newSelp
= make<DfgSel>(lSelp->fileline(), dtype, lSelp->srcp(), lsb);
DfgVertex* resp = make<Reduction>(vtxp, newSelp);
if (extrap) resp = make<Bitwise>(vtxp, resp, extrap);
replace(resp);
return true;
}
}
}
}
if (Reduction* const lRedp = lhsp->template cast<Reduction>()) {
Reduction* rRedp = rhsp->template cast<Reduction>();
DfgVertex* extrap = nullptr;
if (!rRedp) {
if (Bitwise* const rBitwisep = rhsp->template cast<Bitwise>()) {
rRedp = rBitwisep->lhsp()->template cast<Reduction>();
extrap = rBitwisep->rhsp();
}
}
if (rRedp) {
if (DfgSel* const lSelp = lRedp->srcp()->template cast<DfgSel>()) {
if (DfgSel* const rSelp = rRedp->srcp()->template cast<DfgSel>()) {
uint32_t lsb = 0;
if (areAdjacent(lsb, lSelp, rSelp)) {
APPLYING(REPLACE_BITWISE_OF_REDUCTION_OF_SELS_WITH_REDUCTION) {
const DfgDataType& dtype
= DfgDataType::packed(lSelp->width() + rSelp->width());
DfgSel* const newSelp
= make<DfgSel>(lSelp->fileline(), dtype, lSelp->srcp(), lsb);
DfgVertex* resp = make<Reduction>(vtxp, newSelp);
if (extrap) resp = make<Bitwise>(vtxp, resp, extrap);
replace(resp);
return true;
}
}
}
}
}
}
return false;
}
DFG: Add more peephole patterns
2022-09-30 17:19:53 +02:00
template <typename Reduction>
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
VL_ATTR_WARN_UNUSED_RESULT bool optimizeReduction(Reduction* const vtxp) {
DFG: Add more peephole patterns
2022-09-30 17:19:53 +02:00
using Bitwise = ReductionToBitwise<Reduction>;
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
Avoid creating redundant vertices in V3DfgPeephole (#4944) Add a new data-structure V3DfgCache, which can be used to retrieve existing vertices with some given inputs vertices. Use this in V3DfgPeephole to eliminate the creation of redundant vertices. Overall this is performance neutral, but is in prep for some future work.
2024-03-06 19:01:52 +01:00
if (foldUnary(vtxp)) return true;
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
DfgVertex* const srcp = vtxp->srcp();
DFG: Add more peephole patterns
2022-09-30 17:19:53 +02:00
FileLine* const flp = vtxp->fileline();
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
DFG: Add more peephole patterns
2022-09-30 17:19:53 +02:00
// Reduction of 1-bit value
Improve Dfg type system (#6390) Added a mini type system for Dfg using DfgDataType to replace Dfg's use of AstNodeDType. This is much more restricted and represents only the types Dfg can handle in a canonical form. This will be needed when adding more support for unpacked arrays and maybe unpacked structs one day. Also added an internal type checker for DfgGraphs which encodes all the assumptions the code makes about type relationships in the graph. Run this in a few places with --debug-check. Fix resulting fallout.
2025-09-07 21:38:50 +02:00
if (srcp->dtype() == m_bitDType) {
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
APPLYING(REMOVE_WIDTH_ONE_REDUCTION) {
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(srcp);
Avoid creating redundant vertices in V3DfgPeephole (#4944) Add a new data-structure V3DfgCache, which can be used to retrieve existing vertices with some given inputs vertices. Use this in V3DfgPeephole to eliminate the creation of redundant vertices. Overall this is performance neutral, but is in prep for some future work.
2024-03-06 19:01:52 +01:00
return true;
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
}
DFG: Add more peephole patterns
2022-09-30 17:19:53 +02:00
}
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
if (DfgCond* const condp = srcp->cast<DfgCond>()) {
if (condp->thenp()->is<DfgConst>() || condp->elsep()->is<DfgConst>()) {
APPLYING(PUSH_REDUCTION_THROUGH_COND_WITH_CONST_BRANCH) {
// The new 'then' vertex
Avoid creating redundant vertices in V3DfgPeephole (#4944) Add a new data-structure V3DfgCache, which can be used to retrieve existing vertices with some given inputs vertices. Use this in V3DfgPeephole to eliminate the creation of redundant vertices. Overall this is performance neutral, but is in prep for some future work.
2024-03-06 19:01:52 +01:00
Reduction* const newThenp = make<Reduction>(flp, m_bitDType, condp->thenp());
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
// The new 'else' vertex
Avoid creating redundant vertices in V3DfgPeephole (#4944) Add a new data-structure V3DfgCache, which can be used to retrieve existing vertices with some given inputs vertices. Use this in V3DfgPeephole to eliminate the creation of redundant vertices. Overall this is performance neutral, but is in prep for some future work.
2024-03-06 19:01:52 +01:00
Reduction* const newElsep = make<Reduction>(flp, m_bitDType, condp->elsep());
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
// The replacement Cond vertex
Avoid creating redundant vertices in V3DfgPeephole (#4944) Add a new data-structure V3DfgCache, which can be used to retrieve existing vertices with some given inputs vertices. Use this in V3DfgPeephole to eliminate the creation of redundant vertices. Overall this is performance neutral, but is in prep for some future work.
2024-03-06 19:01:52 +01:00
DfgCond* const newCondp = make<DfgCond>(condp->fileline(), m_bitDType,
condp->condp(), newThenp, newElsep);
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
// Replace this vertex
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(newCondp);
Avoid creating redundant vertices in V3DfgPeephole (#4944) Add a new data-structure V3DfgCache, which can be used to retrieve existing vertices with some given inputs vertices. Use this in V3DfgPeephole to eliminate the creation of redundant vertices. Overall this is performance neutral, but is in prep for some future work.
2024-03-06 19:01:52 +01:00
return true;
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
}
}
}
DFG: Add more peephole patterns
2022-09-30 17:19:53 +02:00
if (DfgConcat* const concatp = srcp->cast<DfgConcat>()) {
Optimize more patterns in DfgPeephole (#7332)
2026-03-27 08:47:50 +01:00
if (concatp->lhsp()->is<DfgConst>() || concatp->rhsp()->is<DfgConst>()
|| concatp->lhsp()->dtype() == m_bitDType
|| concatp->rhsp()->dtype() == m_bitDType) {
Try to ensure DFG peephole patterns don't grow the graph Some optimizations are only a net win if they help us remove a graph node (or at least ensure they don't grow the graph), or yields otherwise special logic, so try to apply them only in these cases.
2022-10-04 19:26:53 +02:00
APPLYING(PUSH_REDUCTION_THROUGH_CONCAT) {
// Reduce the parts of the concatenation
Avoid creating redundant vertices in V3DfgPeephole (#4944) Add a new data-structure V3DfgCache, which can be used to retrieve existing vertices with some given inputs vertices. Use this in V3DfgPeephole to eliminate the creation of redundant vertices. Overall this is performance neutral, but is in prep for some future work.
2024-03-06 19:01:52 +01:00
Reduction* const lRedp
= make<Reduction>(concatp->fileline(), m_bitDType, concatp->lhsp());
Reduction* const rRedp
= make<Reduction>(concatp->fileline(), m_bitDType, concatp->rhsp());
Try to ensure DFG peephole patterns don't grow the graph Some optimizations are only a net win if they help us remove a graph node (or at least ensure they don't grow the graph), or yields otherwise special logic, so try to apply them only in these cases.
2022-10-04 19:26:53 +02:00
// Bitwise reduce the results
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(make<Bitwise>(flp, m_bitDType, lRedp, rRedp));
Avoid creating redundant vertices in V3DfgPeephole (#4944) Add a new data-structure V3DfgCache, which can be used to retrieve existing vertices with some given inputs vertices. Use this in V3DfgPeephole to eliminate the creation of redundant vertices. Overall this is performance neutral, but is in prep for some future work.
2024-03-06 19:01:52 +01:00
return true;
Try to ensure DFG peephole patterns don't grow the graph Some optimizations are only a net win if they help us remove a graph node (or at least ensure they don't grow the graph), or yields otherwise special logic, so try to apply them only in these cases.
2022-10-04 19:26:53 +02:00
}
DFG: Add more peephole patterns
2022-09-30 17:19:53 +02:00
}
}
Avoid creating redundant vertices in V3DfgPeephole (#4944) Add a new data-structure V3DfgCache, which can be used to retrieve existing vertices with some given inputs vertices. Use this in V3DfgPeephole to eliminate the creation of redundant vertices. Overall this is performance neutral, but is in prep for some future work.
2024-03-06 19:01:52 +01:00
Optimize more patterns in DfgPeephole (#7332)
2026-03-27 08:47:50 +01:00
if (Bitwise* const bitwisep = vtxp->srcp()->template cast<Bitwise>()) {
if (!bitwisep->hasMultipleSinks()) {
if (bitwisep->lhsp()->template is<DfgConcat>()
|| bitwisep->rhsp()->template is<DfgConcat>()) {
APPLYING(PUSH_REDUCTION_THROUGH_BITWISE_OF_CONCAT) {
Reduction* const newLhsp
= make<Reduction>(flp, m_bitDType, bitwisep->lhsp());
Reduction* const newRhsp
= make<Reduction>(flp, m_bitDType, bitwisep->rhsp());
replace(make<Bitwise>(flp, m_bitDType, newLhsp, newRhsp));
return true;
}
}
if (DfgSel* const lSelp = bitwisep->lhsp()->template cast<DfgSel>()) {
if (DfgSel* const rSelp = bitwisep->rhsp()->template cast<DfgSel>()) {
uint32_t lsb = 0;
if (areAdjacent(lsb, lSelp, rSelp)) {
APPLYING(PUSH_REDUCTION_THROUGH_BITWISE_OF_SELS) {
const DfgDataType& dtype
= DfgDataType::packed(lSelp->width() + rSelp->width());
DfgSel* const newSelp
= make<DfgSel>(lSelp->fileline(), dtype, lSelp->srcp(), lsb);
replace(make<Reduction>(vtxp, newSelp));
return true;
}
}
}
}
}
}
Avoid creating redundant vertices in V3DfgPeephole (#4944) Add a new data-structure V3DfgCache, which can be used to retrieve existing vertices with some given inputs vertices. Use this in V3DfgPeephole to eliminate the creation of redundant vertices. Overall this is performance neutral, but is in prep for some future work.
2024-03-06 19:01:52 +01:00
return false;
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
}
Avoid creating redundant vertices in V3DfgPeephole (#4944) Add a new data-structure V3DfgCache, which can be used to retrieve existing vertices with some given inputs vertices. Use this in V3DfgPeephole to eliminate the creation of redundant vertices. Overall this is performance neutral, but is in prep for some future work.
2024-03-06 19:01:52 +01:00
template <typename Shift>
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
VL_ATTR_WARN_UNUSED_RESULT bool optimizeShiftRHS(Shift* const vtxp) {
Avoid creating redundant vertices in V3DfgPeephole (#4944) Add a new data-structure V3DfgCache, which can be used to retrieve existing vertices with some given inputs vertices. Use this in V3DfgPeephole to eliminate the creation of redundant vertices. Overall this is performance neutral, but is in prep for some future work.
2024-03-06 19:01:52 +01:00
static_assert(std::is_base_of<DfgVertexBinary, Shift>::value, "Must invoke on binary");
static_assert(std::is_final<Shift>::value, "Must invoke on final class");
if (const DfgConcat* const concatp = vtxp->rhsp()->template cast<DfgConcat>()) {
Internals: Improve DFG implementation details (#6355) Large scale refactoring to simplify some of the more obtuse internals of DFG. Remove multiple redundant internal APIs, simplify representation of variables, fix potential unsoundness in circular decomposition. No functional change intended.
2025-09-02 17:50:40 +02:00
if (isZero(concatp->lhsp())) { // Drop redundant zero extension
DfgPeephole: Use a work list driven algorithm for speed Replace the 'run to fixed point' algorithm with a work list driven approach. Instead of marking the graph as changed, we explicitly add vertices to the work list, to be visited, when a vertex is changed. This improves both memory locality (as the work list is processed in last in first out order), and removed unnecessary visitations when only a few nodes changes.
2022-11-04 16:50:20 +01:00
APPLYING(REMOVE_REDUNDANT_ZEXT_ON_RHS_OF_SHIFT) {
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(make<Shift>(vtxp, vtxp->lhsp(), concatp->rhsp()));
Avoid creating redundant vertices in V3DfgPeephole (#4944) Add a new data-structure V3DfgCache, which can be used to retrieve existing vertices with some given inputs vertices. Use this in V3DfgPeephole to eliminate the creation of redundant vertices. Overall this is performance neutral, but is in prep for some future work.
2024-03-06 19:01:52 +01:00
return true;
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
}
}
}
Avoid creating redundant vertices in V3DfgPeephole (#4944) Add a new data-structure V3DfgCache, which can be used to retrieve existing vertices with some given inputs vertices. Use this in V3DfgPeephole to eliminate the creation of redundant vertices. Overall this is performance neutral, but is in prep for some future work.
2024-03-06 19:01:52 +01:00
return false;
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
}
Optimize CountOnes in Dfg
2026-03-22 14:34:33 +01:00
// Given an operand of an Add, return the term that could be used for conveting to CountOnes
// Result is a tulpe of (Vertex, Lsb, Width)
std::tuple<DfgVertex*, uint32_t, uint32_t> addToCountOnesTerm(DfgVertex* vtxp) {
if (DfgConcat* const oCatp = vtxp->cast<DfgConcat>()) {
if (isZero(oCatp->lhsp())) {
if (DfgCountOnes* const countOnesp = oCatp->rhsp()->cast<DfgCountOnes>()) {
// Zero extended count ones
if (DfgSel* const selp = countOnesp->srcp()->cast<DfgSel>()) {
return {selp->fromp(), selp->lsb(), selp->width()};
}
} else if (DfgSel* const selp = oCatp->rhsp()->cast<DfgSel>()) {
// Zero extended single bit select
if (selp->dtype() == m_bitDType) { //
return {selp->fromp(), selp->lsb(), selp->width()};
}
}
}
return {nullptr, 0, 0};
}
if (DfgCountOnes* const countOnesp = vtxp->cast<DfgCountOnes>()) {
// Simple count ones
if (DfgSel* const selp = countOnesp->srcp()->cast<DfgSel>()) {
return {selp->fromp(), selp->lsb(), selp->width()};
}
return {nullptr, 0, 0};
}
if (DfgSel* const oSelp = vtxp->cast<DfgSel>()) {
if (oSelp->lsb() == 0) {
// Truncated count ones
if (DfgCountOnes* const countOnesp = oSelp->fromp()->cast<DfgCountOnes>()) {
// Zero extended count ones
if (DfgSel* const selp = countOnesp->srcp()->cast<DfgSel>()) {
return {selp->fromp(), selp->lsb(), selp->width()};
}
}
}
// Single bit select
if (oSelp->dtype() == m_bitDType) { //
return {oSelp->fromp(), oSelp->lsb(), 1};
}
return {nullptr, 0, 0};
}
// Altered form of extended MSB
if (DfgShiftR* const shiftrp = vtxp->cast<DfgShiftR>()) {
if (DfgConst* const rConstp = shiftrp->rhsp()->cast<DfgConst>()) {
if (rConstp->toU32() == shiftrp->width() - 1) {
return {shiftrp->lhsp(), shiftrp->width() - 1, 1};
}
}
return {nullptr, 0, 0};
}
// Not applicable
return {nullptr, 0, 0};
}
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
// VISIT methods
void visit(DfgVertex*) override {}
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
//=========================================================================
// DfgVertexUnary
//=========================================================================
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
void visit(DfgExtend* const vtxp) override {
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
if (foldUnary(vtxp)) return;
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
Avoid creating redundant vertices in V3DfgPeephole (#4944) Add a new data-structure V3DfgCache, which can be used to retrieve existing vertices with some given inputs vertices. Use this in V3DfgPeephole to eliminate the creation of redundant vertices. Overall this is performance neutral, but is in prep for some future work.
2024-03-06 19:01:52 +01:00
// Convert all Extend into Concat with zeros. This simplifies other patterns as they
// only need to handle Concat, which is more generic, and don't need special cases for
DFG: Add more peephole patterns
2022-09-30 17:19:53 +02:00
// Extend.
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
APPLYING(REPLACE_EXTEND) {
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
DfgVertex* const zerop
= makeZero(vtxp->fileline(), vtxp->width() - vtxp->srcp()->width());
replace(make<DfgConcat>(vtxp, zerop, vtxp->srcp()));
DfgPeephole: Use a work list driven algorithm for speed Replace the 'run to fixed point' algorithm with a work list driven approach. Instead of marking the graph as changed, we explicitly add vertices to the work list, to be visited, when a vertex is changed. This improves both memory locality (as the work list is processed in last in first out order), and removed unnecessary visitations when only a few nodes changes.
2022-11-04 16:50:20 +01:00
return;
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
}
}
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
void visit(DfgExtendS* const vtxp) override {
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
if (foldUnary(vtxp)) return;
}
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
void visit(DfgLogNot* const vtxp) override {
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
if (foldUnary(vtxp)) return;
}
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
void visit(DfgNegate* const vtxp) override {
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
if (foldUnary(vtxp)) return;
}
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
void visit(DfgNot* const vtxp) override {
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
if (foldUnary(vtxp)) return;
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
// Not of Cond
if (DfgCond* const condp = vtxp->srcp()->cast<DfgCond>()) {
// If at least one of the branches are a constant, push the Not past the Cond
if (condp->thenp()->is<DfgConst>() || condp->elsep()->is<DfgConst>()) {
APPLYING(PUSH_NOT_THROUGH_COND) {
// The new 'then' vertex
Avoid creating redundant vertices in V3DfgPeephole (#4944) Add a new data-structure V3DfgCache, which can be used to retrieve existing vertices with some given inputs vertices. Use this in V3DfgPeephole to eliminate the creation of redundant vertices. Overall this is performance neutral, but is in prep for some future work.
2024-03-06 19:01:52 +01:00
DfgNot* const newThenp = make<DfgNot>(vtxp, condp->thenp());
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
// The new 'else' vertex
Avoid creating redundant vertices in V3DfgPeephole (#4944) Add a new data-structure V3DfgCache, which can be used to retrieve existing vertices with some given inputs vertices. Use this in V3DfgPeephole to eliminate the creation of redundant vertices. Overall this is performance neutral, but is in prep for some future work.
2024-03-06 19:01:52 +01:00
DfgNot* const newElsep = make<DfgNot>(vtxp, condp->elsep());
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
// The replacement Cond vertex
Improve Dfg type system (#6390) Added a mini type system for Dfg using DfgDataType to replace Dfg's use of AstNodeDType. This is much more restricted and represents only the types Dfg can handle in a canonical form. This will be needed when adding more support for unpacked arrays and maybe unpacked structs one day. Also added an internal type checker for DfgGraphs which encodes all the assumptions the code makes about type relationships in the graph. Run this in a few places with --debug-check. Fix resulting fallout.
2025-09-07 21:38:50 +02:00
DfgCond* const newCondp = make<DfgCond>(condp->fileline(), vtxp->dtype(),
Avoid creating redundant vertices in V3DfgPeephole (#4944) Add a new data-structure V3DfgCache, which can be used to retrieve existing vertices with some given inputs vertices. Use this in V3DfgPeephole to eliminate the creation of redundant vertices. Overall this is performance neutral, but is in prep for some future work.
2024-03-06 19:01:52 +01:00
condp->condp(), newThenp, newElsep);
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
// Replace this vertex
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(newCondp);
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
return;
}
}
}
// Not of Not
if (DfgNot* const notp = vtxp->srcp()->cast<DfgNot>()) {
APPLYING(REMOVE_NOT_NOT) {
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(notp->srcp());
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
return;
}
}
Try to ensure DFG peephole patterns don't grow the graph Some optimizations are only a net win if they help us remove a graph node (or at least ensure they don't grow the graph), or yields otherwise special logic, so try to apply them only in these cases.
2022-10-04 19:26:53 +02:00
if (!vtxp->srcp()->hasMultipleSinks()) {
// Not of Eq
if (DfgEq* const eqp = vtxp->srcp()->cast<DfgEq>()) {
APPLYING(REPLACE_NOT_EQ) {
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(
make<DfgNeq>(eqp->fileline(), vtxp->dtype(), eqp->lhsp(), eqp->rhsp()));
Try to ensure DFG peephole patterns don't grow the graph Some optimizations are only a net win if they help us remove a graph node (or at least ensure they don't grow the graph), or yields otherwise special logic, so try to apply them only in these cases.
2022-10-04 19:26:53 +02:00
return;
}
DFG: Add more peephole patterns
2022-09-30 17:19:53 +02:00
}
Try to ensure DFG peephole patterns don't grow the graph Some optimizations are only a net win if they help us remove a graph node (or at least ensure they don't grow the graph), or yields otherwise special logic, so try to apply them only in these cases.
2022-10-04 19:26:53 +02:00
// Not of Neq
if (DfgNeq* const neqp = vtxp->srcp()->cast<DfgNeq>()) {
APPLYING(REPLACE_NOT_NEQ) {
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(
make<DfgEq>(neqp->fileline(), vtxp->dtype(), neqp->lhsp(), neqp->rhsp()));
Try to ensure DFG peephole patterns don't grow the graph Some optimizations are only a net win if they help us remove a graph node (or at least ensure they don't grow the graph), or yields otherwise special logic, so try to apply them only in these cases.
2022-10-04 19:26:53 +02:00
return;
}
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
}
}
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
}
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
void visit(DfgRedOr* const vtxp) override {
Avoid creating redundant vertices in V3DfgPeephole (#4944) Add a new data-structure V3DfgCache, which can be used to retrieve existing vertices with some given inputs vertices. Use this in V3DfgPeephole to eliminate the creation of redundant vertices. Overall this is performance neutral, but is in prep for some future work.
2024-03-06 19:01:52 +01:00
if (optimizeReduction(vtxp)) return;
}
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
void visit(DfgRedAnd* const vtxp) override {
Avoid creating redundant vertices in V3DfgPeephole (#4944) Add a new data-structure V3DfgCache, which can be used to retrieve existing vertices with some given inputs vertices. Use this in V3DfgPeephole to eliminate the creation of redundant vertices. Overall this is performance neutral, but is in prep for some future work.
2024-03-06 19:01:52 +01:00
if (optimizeReduction(vtxp)) return;
}
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
void visit(DfgRedXor* const vtxp) override {
Avoid creating redundant vertices in V3DfgPeephole (#4944) Add a new data-structure V3DfgCache, which can be used to retrieve existing vertices with some given inputs vertices. Use this in V3DfgPeephole to eliminate the creation of redundant vertices. Overall this is performance neutral, but is in prep for some future work.
2024-03-06 19:01:52 +01:00
if (optimizeReduction(vtxp)) return;
}
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
void visit(DfgSel* const vtxp) override {
DFG: Special case representation of AstSel AstSel is a ternary node, but the 'widthp' is always constant and is hence redundant, and 'lsbp' is very often constant. As AstSel is fairly common, we special case as a DfgSel for the constant 'lsbp', and as 'DfgMux` for the non-constant 'lsbp'.
2022-10-06 19:34:18 +02:00
DfgVertex* const fromp = vtxp->fromp();
FileLine* const flp = vtxp->fileline();
const uint32_t lsb = vtxp->lsb();
const uint32_t width = vtxp->width();
const uint32_t msb = lsb + width - 1;
if (DfgConst* const constp = fromp->cast<DfgConst>()) {
APPLYING(FOLD_SEL) {
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
DfgConst* const resp = makeZero(flp, width);
resp->num().opSel(constp->num(), msb, lsb);
replace(resp);
DFG: Special case representation of AstSel AstSel is a ternary node, but the 'widthp' is always constant and is hence redundant, and 'lsbp' is very often constant. As AstSel is fairly common, we special case as a DfgSel for the constant 'lsbp', and as 'DfgMux` for the non-constant 'lsbp'.
2022-10-06 19:34:18 +02:00
return;
}
}
// Full width select, replace with the source.
if (fromp->width() == width) {
Remove out of range Sel in V3Unknown (#5443) Fixes #5393
2024-09-09 15:09:29 +02:00
UASSERT_OBJ(lsb == 0, fromp, "Out of range select should have been fixed up earlier");
DFG: Special case representation of AstSel AstSel is a ternary node, but the 'widthp' is always constant and is hence redundant, and 'lsbp' is very often constant. As AstSel is fairly common, we special case as a DfgSel for the constant 'lsbp', and as 'DfgMux` for the non-constant 'lsbp'.
2022-10-06 19:34:18 +02:00
APPLYING(REMOVE_FULL_WIDTH_SEL) {
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(fromp);
DFG: Special case representation of AstSel AstSel is a ternary node, but the 'widthp' is always constant and is hence redundant, and 'lsbp' is very often constant. As AstSel is fairly common, we special case as a DfgSel for the constant 'lsbp', and as 'DfgMux` for the non-constant 'lsbp'.
2022-10-06 19:34:18 +02:00
return;
}
}
// Sel from Concat
if (DfgConcat* const concatp = fromp->cast<DfgConcat>()) {
DfgVertex* const lhsp = concatp->lhsp();
DfgVertex* const rhsp = concatp->rhsp();
if (msb < rhsp->width()) {
// If the select is entirely from rhs, then replace with sel from rhs
APPLYING(REMOVE_SEL_FROM_RHS_OF_CONCAT) { //
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(make<DfgSel>(vtxp, rhsp, vtxp->lsb()));
return;
DFG: Special case representation of AstSel AstSel is a ternary node, but the 'widthp' is always constant and is hence redundant, and 'lsbp' is very often constant. As AstSel is fairly common, we special case as a DfgSel for the constant 'lsbp', and as 'DfgMux` for the non-constant 'lsbp'.
2022-10-06 19:34:18 +02:00
}
} else if (lsb >= rhsp->width()) {
// If the select is entirely from the lhs, then replace with sel from lhs
APPLYING(REMOVE_SEL_FROM_LHS_OF_CONCAT) {
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(make<DfgSel>(vtxp, lhsp, lsb - rhsp->width()));
return;
DFG: Special case representation of AstSel AstSel is a ternary node, but the 'widthp' is always constant and is hence redundant, and 'lsbp' is very often constant. As AstSel is fairly common, we special case as a DfgSel for the constant 'lsbp', and as 'DfgMux` for the non-constant 'lsbp'.
2022-10-06 19:34:18 +02:00
}
}
}
if (DfgReplicate* const repp = fromp->cast<DfgReplicate>()) {
// If the Sel is wholly into the source of the Replicate, push the Sel through the
// Replicate and apply it directly to the source of the Replicate.
const uint32_t srcWidth = repp->srcp()->width();
if (width <= srcWidth) {
const uint32_t newLsb = lsb % srcWidth;
if (newLsb + width <= srcWidth) {
APPLYING(PUSH_SEL_THROUGH_REPLICATE) {
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(make<DfgSel>(vtxp, repp->srcp(), newLsb));
return;
DFG: Special case representation of AstSel AstSel is a ternary node, but the 'widthp' is always constant and is hence redundant, and 'lsbp' is very often constant. As AstSel is fairly common, we special case as a DfgSel for the constant 'lsbp', and as 'DfgMux` for the non-constant 'lsbp'.
2022-10-06 19:34:18 +02:00
}
}
}
}
// Sel from Not
if (DfgNot* const notp = fromp->cast<DfgNot>()) {
// Replace "Sel from Not" with "Not of Sel"
if (!notp->hasMultipleSinks()) {
APPLYING(PUSH_SEL_THROUGH_NOT) {
// Make Sel select from source of Not
Avoid creating redundant vertices in V3DfgPeephole (#4944) Add a new data-structure V3DfgCache, which can be used to retrieve existing vertices with some given inputs vertices. Use this in V3DfgPeephole to eliminate the creation of redundant vertices. Overall this is performance neutral, but is in prep for some future work.
2024-03-06 19:01:52 +01:00
DfgSel* const newSelp = make<DfgSel>(vtxp, notp->srcp(), vtxp->lsb());
DFG: Special case representation of AstSel AstSel is a ternary node, but the 'widthp' is always constant and is hence redundant, and 'lsbp' is very often constant. As AstSel is fairly common, we special case as a DfgSel for the constant 'lsbp', and as 'DfgMux` for the non-constant 'lsbp'.
2022-10-06 19:34:18 +02:00
// Add Not after Sel
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(make<DfgNot>(notp->fileline(), vtxp->dtype(), newSelp));
return;
DFG: Special case representation of AstSel AstSel is a ternary node, but the 'widthp' is always constant and is hence redundant, and 'lsbp' is very often constant. As AstSel is fairly common, we special case as a DfgSel for the constant 'lsbp', and as 'DfgMux` for the non-constant 'lsbp'.
2022-10-06 19:34:18 +02:00
}
}
}
// Sel from Sel
if (DfgSel* const selp = fromp->cast<DfgSel>()) {
APPLYING(REPLACE_SEL_FROM_SEL) {
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
// Select from the source of the source Sel with adjusted LSB
replace(make<DfgSel>(vtxp, selp->fromp(), lsb + selp->lsb()));
return;
DFG: Special case representation of AstSel AstSel is a ternary node, but the 'widthp' is always constant and is hence redundant, and 'lsbp' is very often constant. As AstSel is fairly common, we special case as a DfgSel for the constant 'lsbp', and as 'DfgMux` for the non-constant 'lsbp'.
2022-10-06 19:34:18 +02:00
}
}
// Sel from Cond
if (DfgCond* const condp = fromp->cast<DfgCond>()) {
Optimize more patterns in DfgPeephole (#7332)
2026-03-27 08:47:50 +01:00
if (!condp->hasMultipleSinks()) {
DFG: Special case representation of AstSel AstSel is a ternary node, but the 'widthp' is always constant and is hence redundant, and 'lsbp' is very often constant. As AstSel is fairly common, we special case as a DfgSel for the constant 'lsbp', and as 'DfgMux` for the non-constant 'lsbp'.
2022-10-06 19:34:18 +02:00
APPLYING(PUSH_SEL_THROUGH_COND) {
// The new 'then' vertex
Avoid creating redundant vertices in V3DfgPeephole (#4944) Add a new data-structure V3DfgCache, which can be used to retrieve existing vertices with some given inputs vertices. Use this in V3DfgPeephole to eliminate the creation of redundant vertices. Overall this is performance neutral, but is in prep for some future work.
2024-03-06 19:01:52 +01:00
DfgSel* const newThenp = make<DfgSel>(vtxp, condp->thenp(), lsb);
DFG: Special case representation of AstSel AstSel is a ternary node, but the 'widthp' is always constant and is hence redundant, and 'lsbp' is very often constant. As AstSel is fairly common, we special case as a DfgSel for the constant 'lsbp', and as 'DfgMux` for the non-constant 'lsbp'.
2022-10-06 19:34:18 +02:00
// The new 'else' vertex
Avoid creating redundant vertices in V3DfgPeephole (#4944) Add a new data-structure V3DfgCache, which can be used to retrieve existing vertices with some given inputs vertices. Use this in V3DfgPeephole to eliminate the creation of redundant vertices. Overall this is performance neutral, but is in prep for some future work.
2024-03-06 19:01:52 +01:00
DfgSel* const newElsep = make<DfgSel>(vtxp, condp->elsep(), lsb);
DFG: Special case representation of AstSel AstSel is a ternary node, but the 'widthp' is always constant and is hence redundant, and 'lsbp' is very often constant. As AstSel is fairly common, we special case as a DfgSel for the constant 'lsbp', and as 'DfgMux` for the non-constant 'lsbp'.
2022-10-06 19:34:18 +02:00
// The replacement Cond vertex
Improve Dfg type system (#6390) Added a mini type system for Dfg using DfgDataType to replace Dfg's use of AstNodeDType. This is much more restricted and represents only the types Dfg can handle in a canonical form. This will be needed when adding more support for unpacked arrays and maybe unpacked structs one day. Also added an internal type checker for DfgGraphs which encodes all the assumptions the code makes about type relationships in the graph. Run this in a few places with --debug-check. Fix resulting fallout.
2025-09-07 21:38:50 +02:00
DfgCond* const newCondp = make<DfgCond>(condp->fileline(), vtxp->dtype(),
Avoid creating redundant vertices in V3DfgPeephole (#4944) Add a new data-structure V3DfgCache, which can be used to retrieve existing vertices with some given inputs vertices. Use this in V3DfgPeephole to eliminate the creation of redundant vertices. Overall this is performance neutral, but is in prep for some future work.
2024-03-06 19:01:52 +01:00
condp->condp(), newThenp, newElsep);
DFG: Special case representation of AstSel AstSel is a ternary node, but the 'widthp' is always constant and is hence redundant, and 'lsbp' is very often constant. As AstSel is fairly common, we special case as a DfgSel for the constant 'lsbp', and as 'DfgMux` for the non-constant 'lsbp'.
2022-10-06 19:34:18 +02:00
// Replace this vertex
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(newCondp);
DFG: Special case representation of AstSel AstSel is a ternary node, but the 'widthp' is always constant and is hence redundant, and 'lsbp' is very often constant. As AstSel is fairly common, we special case as a DfgSel for the constant 'lsbp', and as 'DfgMux` for the non-constant 'lsbp'.
2022-10-06 19:34:18 +02:00
return;
}
}
}
// Sel from ShiftL
if (DfgShiftL* const shiftLp = fromp->cast<DfgShiftL>()) {
// If selecting bottom bits of left shift, push the Sel before the shift
if (lsb == 0) {
UASSERT_OBJ(shiftLp->lhsp()->width() >= width, vtxp, "input of shift narrow");
APPLYING(PUSH_SEL_THROUGH_SHIFTL) {
Avoid creating redundant vertices in V3DfgPeephole (#4944) Add a new data-structure V3DfgCache, which can be used to retrieve existing vertices with some given inputs vertices. Use this in V3DfgPeephole to eliminate the creation of redundant vertices. Overall this is performance neutral, but is in prep for some future work.
2024-03-06 19:01:52 +01:00
DfgSel* const newSelp = make<DfgSel>(vtxp, shiftLp->lhsp(), vtxp->lsb());
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(make<DfgShiftL>(vtxp, newSelp, shiftLp->rhsp()));
return;
DFG: Special case representation of AstSel AstSel is a ternary node, but the 'widthp' is always constant and is hence redundant, and 'lsbp' is very often constant. As AstSel is fairly common, we special case as a DfgSel for the constant 'lsbp', and as 'DfgMux` for the non-constant 'lsbp'.
2022-10-06 19:34:18 +02:00
}
}
}
Optimize complex combinational logic in DFG (#6298) This patch adds DfgLogic, which is a vertex that represents a whole, arbitrarily complex combinational AstAlways or AstAssignW in the DfgGraph. Implementing this requires computing the variables live at entry to the AstAlways (variables read by the block), so there is a new ControlFlowGraph data structure and a classical data-flow analysis based live variable analysis to do that at the variable level (as opposed to bit/element level). The actual CFG construction and live variable analysis is best effort, and might fail for currently unhandled constructs or data types. This can be extended later. V3DfgAstToDfg is changed to convert the Ast into an initial DfgGraph containing only DfgLogic, DfgVertexSplice and DfgVertexVar vertices. The DfgLogic are then subsequently synthesized into primitive operations by the new V3DfgSynthesize pass, which is a combination of the old V3DfgAstToDfg conversion and new code to handle AstAlways blocks with complex flow control. V3DfgSynthesize by default will synthesize roughly the same constructs as V3DfgAstToDfg used to handle before, plus any logic that is part of a combinational cycle within the DfgGraph. This enables breaking up these cycles, for which there are extensions to V3DfgBreakCycles in this patch as well. V3DfgSynthesize will then delete all non synthesized or non synthesizable DfgLogic vertices and the rest of the Dfg pipeline is identical, with minor changes to adjust for the changed representation. Because with this change we can now eliminate many more UNOPTFLAT, DFG has been disabled in all the tests that specifically target testing the scheduling and reporting of circular combinational logic.
2025-08-19 16:06:38 +02:00
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
// Sel from a partial variable or narrowed vertex
{
DfgSplicePacked* splicep = fromp->cast<DfgSplicePacked>();
if (DfgVarPacked* const varp = fromp->cast<DfgVarPacked>()) {
Optimize complex combinational logic in DFG (#6298) This patch adds DfgLogic, which is a vertex that represents a whole, arbitrarily complex combinational AstAlways or AstAssignW in the DfgGraph. Implementing this requires computing the variables live at entry to the AstAlways (variables read by the block), so there is a new ControlFlowGraph data structure and a classical data-flow analysis based live variable analysis to do that at the variable level (as opposed to bit/element level). The actual CFG construction and live variable analysis is best effort, and might fail for currently unhandled constructs or data types. This can be extended later. V3DfgAstToDfg is changed to convert the Ast into an initial DfgGraph containing only DfgLogic, DfgVertexSplice and DfgVertexVar vertices. The DfgLogic are then subsequently synthesized into primitive operations by the new V3DfgSynthesize pass, which is a combination of the old V3DfgAstToDfg conversion and new code to handle AstAlways blocks with complex flow control. V3DfgSynthesize by default will synthesize roughly the same constructs as V3DfgAstToDfg used to handle before, plus any logic that is part of a combinational cycle within the DfgGraph. This enables breaking up these cycles, for which there are extensions to V3DfgBreakCycles in this patch as well. V3DfgSynthesize will then delete all non synthesized or non synthesizable DfgLogic vertices and the rest of the Dfg pipeline is identical, with minor changes to adjust for the changed representation. Because with this change we can now eliminate many more UNOPTFLAT, DFG has been disabled in all the tests that specifically target testing the scheduling and reporting of circular combinational logic.
2025-08-19 16:06:38 +02:00
// Must be a splice, otherwise it would have been inlined
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
if (varp->tmpForp() && varp->srcp()) splicep = varp->srcp()->as<DfgSplicePacked>();
}
if (splicep) {
Internals: Remove unused vertices immediately in DfgPeephole
2026-03-23 16:06:07 +01:00
DfgVertex* driverp = nullptr;
uint32_t driverLsb = 0;
Internals: Improve DFG implementation details (#6355) Large scale refactoring to simplify some of the more obtuse internals of DFG. Remove multiple redundant internal APIs, simplify representation of variables, fix potential unsoundness in circular decomposition. No functional change intended.
2025-09-02 17:50:40 +02:00
splicep->foreachDriver([&](DfgVertex& src, const uint32_t dLsb) {
const uint32_t dMsb = dLsb + src.width() - 1;
Optimize complex combinational logic in DFG (#6298) This patch adds DfgLogic, which is a vertex that represents a whole, arbitrarily complex combinational AstAlways or AstAssignW in the DfgGraph. Implementing this requires computing the variables live at entry to the AstAlways (variables read by the block), so there is a new ControlFlowGraph data structure and a classical data-flow analysis based live variable analysis to do that at the variable level (as opposed to bit/element level). The actual CFG construction and live variable analysis is best effort, and might fail for currently unhandled constructs or data types. This can be extended later. V3DfgAstToDfg is changed to convert the Ast into an initial DfgGraph containing only DfgLogic, DfgVertexSplice and DfgVertexVar vertices. The DfgLogic are then subsequently synthesized into primitive operations by the new V3DfgSynthesize pass, which is a combination of the old V3DfgAstToDfg conversion and new code to handle AstAlways blocks with complex flow control. V3DfgSynthesize by default will synthesize roughly the same constructs as V3DfgAstToDfg used to handle before, plus any logic that is part of a combinational cycle within the DfgGraph. This enables breaking up these cycles, for which there are extensions to V3DfgBreakCycles in this patch as well. V3DfgSynthesize will then delete all non synthesized or non synthesizable DfgLogic vertices and the rest of the Dfg pipeline is identical, with minor changes to adjust for the changed representation. Because with this change we can now eliminate many more UNOPTFLAT, DFG has been disabled in all the tests that specifically target testing the scheduling and reporting of circular combinational logic.
2025-08-19 16:06:38 +02:00
// If it does not cover the whole searched bit range, move on
Internals: Improve DFG implementation details (#6355) Large scale refactoring to simplify some of the more obtuse internals of DFG. Remove multiple redundant internal APIs, simplify representation of variables, fix potential unsoundness in circular decomposition. No functional change intended.
2025-09-02 17:50:40 +02:00
if (lsb < dLsb || dMsb < msb) return false;
Internals: Remove unused vertices immediately in DfgPeephole
2026-03-23 16:06:07 +01:00
// Save the driver
driverp = &src;
driverLsb = dLsb;
Internals: Improve DFG implementation details (#6355) Large scale refactoring to simplify some of the more obtuse internals of DFG. Remove multiple redundant internal APIs, simplify representation of variables, fix potential unsoundness in circular decomposition. No functional change intended.
2025-09-02 17:50:40 +02:00
return true;
});
Internals: Remove unused vertices immediately in DfgPeephole
2026-03-23 16:06:07 +01:00
if (driverp) {
Optimize complex combinational logic in DFG (#6298) This patch adds DfgLogic, which is a vertex that represents a whole, arbitrarily complex combinational AstAlways or AstAssignW in the DfgGraph. Implementing this requires computing the variables live at entry to the AstAlways (variables read by the block), so there is a new ControlFlowGraph data structure and a classical data-flow analysis based live variable analysis to do that at the variable level (as opposed to bit/element level). The actual CFG construction and live variable analysis is best effort, and might fail for currently unhandled constructs or data types. This can be extended later. V3DfgAstToDfg is changed to convert the Ast into an initial DfgGraph containing only DfgLogic, DfgVertexSplice and DfgVertexVar vertices. The DfgLogic are then subsequently synthesized into primitive operations by the new V3DfgSynthesize pass, which is a combination of the old V3DfgAstToDfg conversion and new code to handle AstAlways blocks with complex flow control. V3DfgSynthesize by default will synthesize roughly the same constructs as V3DfgAstToDfg used to handle before, plus any logic that is part of a combinational cycle within the DfgGraph. This enables breaking up these cycles, for which there are extensions to V3DfgBreakCycles in this patch as well. V3DfgSynthesize will then delete all non synthesized or non synthesizable DfgLogic vertices and the rest of the Dfg pipeline is identical, with minor changes to adjust for the changed representation. Because with this change we can now eliminate many more UNOPTFLAT, DFG has been disabled in all the tests that specifically target testing the scheduling and reporting of circular combinational logic.
2025-08-19 16:06:38 +02:00
APPLYING(PUSH_SEL_THROUGH_SPLICE) {
Internals: Remove unused vertices immediately in DfgPeephole
2026-03-23 16:06:07 +01:00
replace(make<DfgSel>(vtxp, driverp, lsb - driverLsb));
Optimize: Remove redundant variables during Dfg Peephole pass (#7076) Many rules in the Dfg Peephole pass check if a node has more than one sinks. Redundant variables that will ultimately be removed can prevent these from matching. Remove such variables during the Peeophole pass itself to enable more matches.
2026-02-14 13:33:20 +01:00
return;
Optimize complex combinational logic in DFG (#6298) This patch adds DfgLogic, which is a vertex that represents a whole, arbitrarily complex combinational AstAlways or AstAssignW in the DfgGraph. Implementing this requires computing the variables live at entry to the AstAlways (variables read by the block), so there is a new ControlFlowGraph data structure and a classical data-flow analysis based live variable analysis to do that at the variable level (as opposed to bit/element level). The actual CFG construction and live variable analysis is best effort, and might fail for currently unhandled constructs or data types. This can be extended later. V3DfgAstToDfg is changed to convert the Ast into an initial DfgGraph containing only DfgLogic, DfgVertexSplice and DfgVertexVar vertices. The DfgLogic are then subsequently synthesized into primitive operations by the new V3DfgSynthesize pass, which is a combination of the old V3DfgAstToDfg conversion and new code to handle AstAlways blocks with complex flow control. V3DfgSynthesize by default will synthesize roughly the same constructs as V3DfgAstToDfg used to handle before, plus any logic that is part of a combinational cycle within the DfgGraph. This enables breaking up these cycles, for which there are extensions to V3DfgBreakCycles in this patch as well. V3DfgSynthesize will then delete all non synthesized or non synthesizable DfgLogic vertices and the rest of the Dfg pipeline is identical, with minor changes to adjust for the changed representation. Because with this change we can now eliminate many more UNOPTFLAT, DFG has been disabled in all the tests that specifically target testing the scheduling and reporting of circular combinational logic.
2025-08-19 16:06:38 +02:00
}
}
}
}
DFG: Special case representation of AstSel AstSel is a ternary node, but the 'widthp' is always constant and is hence redundant, and 'lsbp' is very often constant. As AstSel is fairly common, we special case as a DfgSel for the constant 'lsbp', and as 'DfgMux` for the non-constant 'lsbp'.
2022-10-06 19:34:18 +02:00
}
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
//=========================================================================
// DfgVertexBinary - bitwise
//=========================================================================
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
void visit(DfgAnd* const vtxp) override {
Add DfgPeephole patterns (#6149)
2025-07-07 17:25:29 +02:00
DfgVertex* const lhsp = vtxp->lhsp();
DfgVertex* const rhsp = vtxp->rhsp();
Improve Dfg type system (#6390) Added a mini type system for Dfg using DfgDataType to replace Dfg's use of AstNodeDType. This is much more restricted and represents only the types Dfg can handle in a canonical form. This will be needed when adding more support for unpacked arrays and maybe unpacked structs one day. Also added an internal type checker for DfgGraphs which encodes all the assumptions the code makes about type relationships in the graph. Run this in a few places with --debug-check. Fix resulting fallout.
2025-09-07 21:38:50 +02:00
if (isSame(lhsp, rhsp)) {
Add DfgPeephole patterns (#6149)
2025-07-07 17:25:29 +02:00
APPLYING(REMOVE_AND_WITH_SELF) {
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(lhsp);
Add DfgPeephole patterns (#6149)
2025-07-07 17:25:29 +02:00
return;
}
}
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
DfgPeephole: Rework folding of associative operations Allow constant folding through adjacent nodes of all associative operations, for example '((a & 2) & 3)' or '(3 & (2 & a))' can now be folded into '(a & 2)' and '(2 & a)' respectively. Also improve speed of making associative expression trees right leaning by using rotation of the existing vertices whenever instead of allocation of new nodes.
2022-10-05 11:36:28 +02:00
if (associativeBinary(vtxp)) return;
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
Avoid creating redundant vertices in V3DfgPeephole (#4944) Add a new data-structure V3DfgCache, which can be used to retrieve existing vertices with some given inputs vertices. Use this in V3DfgPeephole to eliminate the creation of redundant vertices. Overall this is performance neutral, but is in prep for some future work.
2024-03-06 19:01:52 +01:00
if (commutativeBinary(vtxp)) return;
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
FileLine* const flp = vtxp->fileline();
Optimize DFG De Morgan patterns (#6090) It's ok if the replaced vertex has multiple sinks, this pattern cannot increase the size of logic even then.
2025-06-15 17:00:11 +02:00
// Bubble pushing (De Morgan)
Optimize more patterns in DfgPeephole (#7332)
2026-03-27 08:47:50 +01:00
if (!lhsp->hasMultipleSinks() || !rhsp->hasMultipleSinks()) {
Try to ensure DFG peephole patterns don't grow the graph Some optimizations are only a net win if they help us remove a graph node (or at least ensure they don't grow the graph), or yields otherwise special logic, so try to apply them only in these cases.
2022-10-04 19:26:53 +02:00
if (DfgNot* const lhsNotp = lhsp->cast<DfgNot>()) {
if (DfgNot* const rhsNotp = rhsp->cast<DfgNot>()) {
APPLYING(REPLACE_AND_OF_NOT_AND_NOT) {
Avoid creating redundant vertices in V3DfgPeephole (#4944) Add a new data-structure V3DfgCache, which can be used to retrieve existing vertices with some given inputs vertices. Use this in V3DfgPeephole to eliminate the creation of redundant vertices. Overall this is performance neutral, but is in prep for some future work.
2024-03-06 19:01:52 +01:00
DfgOr* const orp = make<DfgOr>(vtxp, lhsNotp->srcp(), rhsNotp->srcp());
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(make<DfgNot>(vtxp, orp));
Try to ensure DFG peephole patterns don't grow the graph Some optimizations are only a net win if they help us remove a graph node (or at least ensure they don't grow the graph), or yields otherwise special logic, so try to apply them only in these cases.
2022-10-04 19:26:53 +02:00
return;
}
DFG: Add more peephole patterns
2022-09-30 17:19:53 +02:00
}
Try to ensure DFG peephole patterns don't grow the graph Some optimizations are only a net win if they help us remove a graph node (or at least ensure they don't grow the graph), or yields otherwise special logic, so try to apply them only in these cases.
2022-10-04 19:26:53 +02:00
if (DfgNeq* const rhsNeqp = rhsp->cast<DfgNeq>()) {
APPLYING(REPLACE_AND_OF_NOT_AND_NEQ) {
Avoid creating redundant vertices in V3DfgPeephole (#4944) Add a new data-structure V3DfgCache, which can be used to retrieve existing vertices with some given inputs vertices. Use this in V3DfgPeephole to eliminate the creation of redundant vertices. Overall this is performance neutral, but is in prep for some future work.
2024-03-06 19:01:52 +01:00
DfgEq* const newRhsp = make<DfgEq>(rhsp, rhsNeqp->lhsp(), rhsNeqp->rhsp());
DfgOr* const orp = make<DfgOr>(vtxp, lhsNotp->srcp(), newRhsp);
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(make<DfgNot>(vtxp, orp));
Try to ensure DFG peephole patterns don't grow the graph Some optimizations are only a net win if they help us remove a graph node (or at least ensure they don't grow the graph), or yields otherwise special logic, so try to apply them only in these cases.
2022-10-04 19:26:53 +02:00
return;
}
DFG: Add more peephole patterns
2022-09-30 17:19:53 +02:00
}
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
}
}
Optimize more patterns in DfgPeephole (#7332)
2026-03-27 08:47:50 +01:00
if (DfgConst* const lConstp = lhsp->cast<DfgConst>()) {
if (lConstp->isZero()) {
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
APPLYING(REPLACE_AND_WITH_ZERO) {
Optimize more patterns in DfgPeephole (#7332)
2026-03-27 08:47:50 +01:00
replace(lConstp);
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
return;
}
}
Optimize more patterns in DfgPeephole (#7332)
2026-03-27 08:47:50 +01:00
if (lConstp->isOnes()) {
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
APPLYING(REMOVE_AND_WITH_ONES) {
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(rhsp);
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
return;
}
}
if (DfgConcat* const rhsConcatp = rhsp->cast<DfgConcat>()) {
Optimize more patterns in DfgPeephole (#7332)
2026-03-27 08:47:50 +01:00
if (tryPushBitwiseOpThroughConcat(vtxp, lConstp, rhsConcatp)) return;
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
}
}
Add DfgPeephole patterns (#6149)
2025-07-07 17:25:29 +02:00
if (distributiveAndAssociativeBinary<DfgOr, DfgAnd>(vtxp)) return;
DFG: Add more peephole patterns
2022-09-30 17:19:53 +02:00
if (tryPushBitwiseOpThroughReductions(vtxp)) return;
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
if (DfgNot* const lhsNotp = lhsp->cast<DfgNot>()) {
// ~A & A is all zeroes
if (lhsNotp->srcp() == rhsp) {
APPLYING(REPLACE_CONTRADICTORY_AND) {
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(makeZero(flp, vtxp->width()));
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
return;
}
}
Add DfgPeephole patterns (#6149)
2025-07-07 17:25:29 +02:00
// ~A & (A & _) or ~A & (_ & A) is all zeroes
if (DfgAnd* const rhsAndp = rhsp->cast<DfgAnd>()) {
if (lhsNotp->srcp() == rhsAndp->lhsp() || lhsNotp->srcp() == rhsAndp->rhsp()) {
APPLYING(REPLACE_CONTRADICTORY_AND_3) {
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(makeZero(flp, vtxp->width()));
Add DfgPeephole patterns (#6149)
2025-07-07 17:25:29 +02:00
return;
}
}
}
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
}
Optimize more patterns in DfgPeephole (#7332)
2026-03-27 08:47:50 +01:00
if (vtxp->dtype() == m_bitDType) {
if (tryReplaceBitwiseWithReduction(vtxp)) return;
}
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
}
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
void visit(DfgOr* const vtxp) override {
Add DfgPeephole patterns (#6149)
2025-07-07 17:25:29 +02:00
DfgVertex* const lhsp = vtxp->lhsp();
DfgVertex* const rhsp = vtxp->rhsp();
Improve Dfg type system (#6390) Added a mini type system for Dfg using DfgDataType to replace Dfg's use of AstNodeDType. This is much more restricted and represents only the types Dfg can handle in a canonical form. This will be needed when adding more support for unpacked arrays and maybe unpacked structs one day. Also added an internal type checker for DfgGraphs which encodes all the assumptions the code makes about type relationships in the graph. Run this in a few places with --debug-check. Fix resulting fallout.
2025-09-07 21:38:50 +02:00
if (isSame(lhsp, rhsp)) {
Add DfgPeephole patterns (#6149)
2025-07-07 17:25:29 +02:00
APPLYING(REMOVE_OR_WITH_SELF) {
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(lhsp);
Add DfgPeephole patterns (#6149)
2025-07-07 17:25:29 +02:00
return;
}
}
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
DfgPeephole: Rework folding of associative operations Allow constant folding through adjacent nodes of all associative operations, for example '((a & 2) & 3)' or '(3 & (2 & a))' can now be folded into '(a & 2)' and '(2 & a)' respectively. Also improve speed of making associative expression trees right leaning by using rotation of the existing vertices whenever instead of allocation of new nodes.
2022-10-05 11:36:28 +02:00
if (associativeBinary(vtxp)) return;
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
Avoid creating redundant vertices in V3DfgPeephole (#4944) Add a new data-structure V3DfgCache, which can be used to retrieve existing vertices with some given inputs vertices. Use this in V3DfgPeephole to eliminate the creation of redundant vertices. Overall this is performance neutral, but is in prep for some future work.
2024-03-06 19:01:52 +01:00
if (commutativeBinary(vtxp)) return;
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
FileLine* const flp = vtxp->fileline();
Optimize DFG De Morgan patterns (#6090) It's ok if the replaced vertex has multiple sinks, this pattern cannot increase the size of logic even then.
2025-06-15 17:00:11 +02:00
// Bubble pushing (De Morgan)
Optimize more patterns in DfgPeephole (#7332)
2026-03-27 08:47:50 +01:00
if (!lhsp->hasMultipleSinks() || !rhsp->hasMultipleSinks()) {
Try to ensure DFG peephole patterns don't grow the graph Some optimizations are only a net win if they help us remove a graph node (or at least ensure they don't grow the graph), or yields otherwise special logic, so try to apply them only in these cases.
2022-10-04 19:26:53 +02:00
if (DfgNot* const lhsNotp = lhsp->cast<DfgNot>()) {
if (DfgNot* const rhsNotp = rhsp->cast<DfgNot>()) {
APPLYING(REPLACE_OR_OF_NOT_AND_NOT) {
Avoid creating redundant vertices in V3DfgPeephole (#4944) Add a new data-structure V3DfgCache, which can be used to retrieve existing vertices with some given inputs vertices. Use this in V3DfgPeephole to eliminate the creation of redundant vertices. Overall this is performance neutral, but is in prep for some future work.
2024-03-06 19:01:52 +01:00
DfgAnd* const andp = make<DfgAnd>(vtxp, lhsNotp->srcp(), rhsNotp->srcp());
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(make<DfgNot>(vtxp, andp));
Try to ensure DFG peephole patterns don't grow the graph Some optimizations are only a net win if they help us remove a graph node (or at least ensure they don't grow the graph), or yields otherwise special logic, so try to apply them only in these cases.
2022-10-04 19:26:53 +02:00
return;
}
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
}
Try to ensure DFG peephole patterns don't grow the graph Some optimizations are only a net win if they help us remove a graph node (or at least ensure they don't grow the graph), or yields otherwise special logic, so try to apply them only in these cases.
2022-10-04 19:26:53 +02:00
if (DfgNeq* const rhsNeqp = rhsp->cast<DfgNeq>()) {
APPLYING(REPLACE_OR_OF_NOT_AND_NEQ) {
Avoid creating redundant vertices in V3DfgPeephole (#4944) Add a new data-structure V3DfgCache, which can be used to retrieve existing vertices with some given inputs vertices. Use this in V3DfgPeephole to eliminate the creation of redundant vertices. Overall this is performance neutral, but is in prep for some future work.
2024-03-06 19:01:52 +01:00
DfgEq* const newRhsp = make<DfgEq>(rhsp, rhsNeqp->lhsp(), rhsNeqp->rhsp());
DfgAnd* const andp = make<DfgAnd>(vtxp, lhsNotp->srcp(), newRhsp);
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(make<DfgNot>(vtxp, andp));
Try to ensure DFG peephole patterns don't grow the graph Some optimizations are only a net win if they help us remove a graph node (or at least ensure they don't grow the graph), or yields otherwise special logic, so try to apply them only in these cases.
2022-10-04 19:26:53 +02:00
return;
}
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
}
}
}
if (DfgConcat* const lhsConcatp = lhsp->cast<DfgConcat>()) {
if (DfgConcat* const rhsConcatp = rhsp->cast<DfgConcat>()) {
Improve Dfg type system (#6390) Added a mini type system for Dfg using DfgDataType to replace Dfg's use of AstNodeDType. This is much more restricted and represents only the types Dfg can handle in a canonical form. This will be needed when adding more support for unpacked arrays and maybe unpacked structs one day. Also added an internal type checker for DfgGraphs which encodes all the assumptions the code makes about type relationships in the graph. Run this in a few places with --debug-check. Fix resulting fallout.
2025-09-07 21:38:50 +02:00
if (lhsConcatp->lhsp()->dtype() == rhsConcatp->lhsp()->dtype()) {
Internals: Improve DFG implementation details (#6355) Large scale refactoring to simplify some of the more obtuse internals of DFG. Remove multiple redundant internal APIs, simplify representation of variables, fix potential unsoundness in circular decomposition. No functional change intended.
2025-09-02 17:50:40 +02:00
if (isZero(lhsConcatp->lhsp()) && isZero(rhsConcatp->rhsp())) {
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
APPLYING(REPLACE_OR_OF_CONCAT_ZERO_LHS_AND_CONCAT_RHS_ZERO) {
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(make<DfgConcat>(vtxp, rhsConcatp->lhsp(), lhsConcatp->rhsp()));
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
return;
}
}
Internals: Improve DFG implementation details (#6355) Large scale refactoring to simplify some of the more obtuse internals of DFG. Remove multiple redundant internal APIs, simplify representation of variables, fix potential unsoundness in circular decomposition. No functional change intended.
2025-09-02 17:50:40 +02:00
if (isZero(lhsConcatp->rhsp()) && isZero(rhsConcatp->lhsp())) {
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
APPLYING(REPLACE_OR_OF_CONCAT_LHS_ZERO_AND_CONCAT_ZERO_RHS) {
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(make<DfgConcat>(vtxp, lhsConcatp->lhsp(), rhsConcatp->rhsp()));
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
return;
}
}
}
}
}
Optimize more patterns in DfgPeephole (#7332)
2026-03-27 08:47:50 +01:00
if (DfgConst* const lConstp = lhsp->cast<DfgConst>()) {
if (lConstp->isZero()) {
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
APPLYING(REMOVE_OR_WITH_ZERO) {
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(rhsp);
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
return;
}
}
Optimize more patterns in DfgPeephole (#7332)
2026-03-27 08:47:50 +01:00
if (lConstp->isOnes()) {
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
APPLYING(REPLACE_OR_WITH_ONES) {
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(lhsp);
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
return;
}
}
if (DfgConcat* const rhsConcatp = rhsp->cast<DfgConcat>()) {
Optimize more patterns in DfgPeephole (#7332)
2026-03-27 08:47:50 +01:00
if (tryPushBitwiseOpThroughConcat(vtxp, lConstp, rhsConcatp)) return;
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
}
}
Add DfgPeephole patterns (#6149)
2025-07-07 17:25:29 +02:00
if (distributiveAndAssociativeBinary<DfgAnd, DfgOr>(vtxp)) return;
DFG: Add more peephole patterns
2022-09-30 17:19:53 +02:00
if (tryPushBitwiseOpThroughReductions(vtxp)) return;
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
if (DfgNot* const lhsNotp = lhsp->cast<DfgNot>()) {
// ~A | A is all ones
if (lhsNotp->srcp() == rhsp) {
APPLYING(REPLACE_TAUTOLOGICAL_OR) {
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
DfgConst* const resp = makeZero(flp, vtxp->width());
resp->num().setAllBits1();
replace(resp);
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
return;
}
}
Add DfgPeephole patterns (#6149)
2025-07-07 17:25:29 +02:00
// ~A | (A | _) or ~A | (_ | A) is all ones
if (DfgOr* const rhsOrp = rhsp->cast<DfgOr>()) {
if (lhsNotp->srcp() == rhsOrp->lhsp() || lhsNotp->srcp() == rhsOrp->rhsp()) {
APPLYING(REPLACE_TAUTOLOGICAL_OR_3) {
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
DfgConst* const resp = makeZero(flp, vtxp->width());
resp->num().setAllBits1();
replace(resp);
Add DfgPeephole patterns (#6149)
2025-07-07 17:25:29 +02:00
return;
}
}
}
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
}
Optimize more patterns in DfgPeephole (#7332)
2026-03-27 08:47:50 +01:00
if (vtxp->dtype() == m_bitDType) {
if (tryReplaceBitwiseWithReduction(vtxp)) return;
}
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
}
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
void visit(DfgXor* const vtxp) override {
Add DfgPeephole patterns (#6149)
2025-07-07 17:25:29 +02:00
DfgVertex* const lhsp = vtxp->lhsp();
DfgVertex* const rhsp = vtxp->rhsp();
Improve Dfg type system (#6390) Added a mini type system for Dfg using DfgDataType to replace Dfg's use of AstNodeDType. This is much more restricted and represents only the types Dfg can handle in a canonical form. This will be needed when adding more support for unpacked arrays and maybe unpacked structs one day. Also added an internal type checker for DfgGraphs which encodes all the assumptions the code makes about type relationships in the graph. Run this in a few places with --debug-check. Fix resulting fallout.
2025-09-07 21:38:50 +02:00
if (isSame(lhsp, rhsp)) {
Add DfgPeephole patterns (#6149)
2025-07-07 17:25:29 +02:00
APPLYING(REPLACE_XOR_WITH_SELF) {
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(makeZero(vtxp->fileline(), vtxp->width()));
Add DfgPeephole patterns (#6149)
2025-07-07 17:25:29 +02:00
return;
}
}
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
DfgPeephole: Rework folding of associative operations Allow constant folding through adjacent nodes of all associative operations, for example '((a & 2) & 3)' or '(3 & (2 & a))' can now be folded into '(a & 2)' and '(2 & a)' respectively. Also improve speed of making associative expression trees right leaning by using rotation of the existing vertices whenever instead of allocation of new nodes.
2022-10-05 11:36:28 +02:00
if (associativeBinary(vtxp)) return;
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
Avoid creating redundant vertices in V3DfgPeephole (#4944) Add a new data-structure V3DfgCache, which can be used to retrieve existing vertices with some given inputs vertices. Use this in V3DfgPeephole to eliminate the creation of redundant vertices. Overall this is performance neutral, but is in prep for some future work.
2024-03-06 19:01:52 +01:00
if (commutativeBinary(vtxp)) return;
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
DFG: Add more peephole patterns
2022-09-30 17:19:53 +02:00
if (DfgConst* const lConstp = lhsp->cast<DfgConst>()) {
if (lConstp->isZero()) {
APPLYING(REMOVE_XOR_WITH_ZERO) {
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(rhsp);
DFG: Add more peephole patterns
2022-09-30 17:19:53 +02:00
return;
}
}
if (lConstp->isOnes()) {
APPLYING(REPLACE_XOR_WITH_ONES) {
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(make<DfgNot>(vtxp, rhsp));
DFG: Add more peephole patterns
2022-09-30 17:19:53 +02:00
return;
}
}
if (DfgConcat* const rConcatp = rhsp->cast<DfgConcat>()) {
Avoid creating redundant vertices in V3DfgPeephole (#4944) Add a new data-structure V3DfgCache, which can be used to retrieve existing vertices with some given inputs vertices. Use this in V3DfgPeephole to eliminate the creation of redundant vertices. Overall this is performance neutral, but is in prep for some future work.
2024-03-06 19:01:52 +01:00
if (tryPushBitwiseOpThroughConcat(vtxp, lConstp, rConcatp)) return;
DFG: Add more peephole patterns
2022-09-30 17:19:53 +02:00
}
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
}
DFG: Add more peephole patterns
2022-09-30 17:19:53 +02:00
if (tryPushBitwiseOpThroughReductions(vtxp)) return;
Optimize more patterns in DfgPeephole (#7332)
2026-03-27 08:47:50 +01:00
if (vtxp->dtype() == m_bitDType) {
if (tryReplaceBitwiseWithReduction(vtxp)) return;
}
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
}
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
//=========================================================================
// DfgVertexBinary - other
//=========================================================================
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
void visit(DfgAdd* const vtxp) override {
DfgPeephole: Rework folding of associative operations Allow constant folding through adjacent nodes of all associative operations, for example '((a & 2) & 3)' or '(3 & (2 & a))' can now be folded into '(a & 2)' and '(2 & a)' respectively. Also improve speed of making associative expression trees right leaning by using rotation of the existing vertices whenever instead of allocation of new nodes.
2022-10-05 11:36:28 +02:00
if (associativeBinary(vtxp)) return;
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
Avoid creating redundant vertices in V3DfgPeephole (#4944) Add a new data-structure V3DfgCache, which can be used to retrieve existing vertices with some given inputs vertices. Use this in V3DfgPeephole to eliminate the creation of redundant vertices. Overall this is performance neutral, but is in prep for some future work.
2024-03-06 19:01:52 +01:00
if (commutativeBinary(vtxp)) return;
Optimize CountOnes in Dfg
2026-03-22 14:34:33 +01:00
DfgVertex* const lhsp = vtxp->lhsp();
DfgVertex* const rhsp = vtxp->rhsp();
FileLine* const flp = vtxp->fileline();
if (isZero(lhsp)) {
APPLYING(REMOVE_ADD_ZERO) {
replace(rhsp);
return;
}
}
const std::tuple<DfgVertex*, uint32_t, uint32_t> lTerm = addToCountOnesTerm(lhsp);
if (DfgVertex* const lVtxp = std::get<0>(lTerm)) {
std::tuple<DfgVertex*, uint32_t, uint32_t> rTerm = addToCountOnesTerm(rhsp);
DfgVertex* extrap = nullptr;
if (!std::get<0>(rTerm)) {
if (DfgAdd* const rAddp = rhsp->cast<DfgAdd>()) {
rTerm = addToCountOnesTerm(rAddp->lhsp());
extrap = rAddp->rhsp();
}
}
if (DfgVertex* const rVtxp = std::get<0>(rTerm)) {
if (isSame(lVtxp, rVtxp)) {
const uint32_t lLsb = std::get<1>(lTerm);
const uint32_t rLsb = std::get<1>(rTerm);
const uint32_t lWidth = std::get<2>(lTerm);
const uint32_t rWidth = std::get<2>(rTerm);
bool adjoined = true;
uint32_t lsb = 0;
if (lLsb + lWidth == rLsb) {
lsb = lLsb;
} else if (lLsb == rLsb + rWidth) {
lsb = rLsb;
} else {
adjoined = false;
}
if (adjoined) {
APPLYING(REPLACE_ADD_WITH_COUNT_ONES) {
DfgSel* const selp
= make<DfgSel>(vtxp->fileline(),
DfgDataType::packed(lWidth + rWidth), lVtxp, lsb);
DfgVertex* resp
= make<DfgCountOnes>(flp, DfgDataType::packed(32), selp);
if (vtxp->width() > 32U) {
resp = make<DfgConcat>(vtxp, makeZero(flp, vtxp->width() - 32U),
resp);
} else if (vtxp->width() < 32U) {
resp = make<DfgSel>(vtxp, resp, 0U);
}
if (extrap) resp = make<DfgAdd>(vtxp, resp, extrap);
replace(resp);
return;
}
}
}
}
}
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
}
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
void visit(DfgArraySel* const vtxp) override {
Optimize complex combinational logic in DFG (#6298) This patch adds DfgLogic, which is a vertex that represents a whole, arbitrarily complex combinational AstAlways or AstAssignW in the DfgGraph. Implementing this requires computing the variables live at entry to the AstAlways (variables read by the block), so there is a new ControlFlowGraph data structure and a classical data-flow analysis based live variable analysis to do that at the variable level (as opposed to bit/element level). The actual CFG construction and live variable analysis is best effort, and might fail for currently unhandled constructs or data types. This can be extended later. V3DfgAstToDfg is changed to convert the Ast into an initial DfgGraph containing only DfgLogic, DfgVertexSplice and DfgVertexVar vertices. The DfgLogic are then subsequently synthesized into primitive operations by the new V3DfgSynthesize pass, which is a combination of the old V3DfgAstToDfg conversion and new code to handle AstAlways blocks with complex flow control. V3DfgSynthesize by default will synthesize roughly the same constructs as V3DfgAstToDfg used to handle before, plus any logic that is part of a combinational cycle within the DfgGraph. This enables breaking up these cycles, for which there are extensions to V3DfgBreakCycles in this patch as well. V3DfgSynthesize will then delete all non synthesized or non synthesizable DfgLogic vertices and the rest of the Dfg pipeline is identical, with minor changes to adjust for the changed representation. Because with this change we can now eliminate many more UNOPTFLAT, DFG has been disabled in all the tests that specifically target testing the scheduling and reporting of circular combinational logic.
2025-08-19 16:06:38 +02:00
DfgConst* const idxp = vtxp->bitp()->cast<DfgConst>();
if (!idxp) return;
DfgVarArray* const varp = vtxp->fromp()->cast<DfgVarArray>();
if (!varp) return;
if (varp->varp()->isForced()) return;
Fix corner case bugs in module and variable inlining (#6322) There were a couple corner case bugs in V3Inline, and one in Dfg when dealing with inlining of modules/variables. V3Inline: - Invalid code generated when inlining an input that also had an assignment to it (Throws an ASSIGNIN, but this is sometimes reasonable to do, e.g. hiererchical reference to an unonnected input port) - Inlining (aliasing) publicly writeable input port. - Inlining forcable port connected to constant. Dfg: - Inining publicly writeable variables The tests that cover these are the same and fixing one will trigger the other bug, so fixing them all in one go. Also cleanup V3Inline to be less out of order and rely less on unique APIs only used by V3Inine (will remove those in follow up patch). Small step towards #6280.
2025-08-22 22:43:49 +02:00
if (varp->varp()->isSigUserRWPublic()) return;
Optimize complex combinational logic in DFG (#6298) This patch adds DfgLogic, which is a vertex that represents a whole, arbitrarily complex combinational AstAlways or AstAssignW in the DfgGraph. Implementing this requires computing the variables live at entry to the AstAlways (variables read by the block), so there is a new ControlFlowGraph data structure and a classical data-flow analysis based live variable analysis to do that at the variable level (as opposed to bit/element level). The actual CFG construction and live variable analysis is best effort, and might fail for currently unhandled constructs or data types. This can be extended later. V3DfgAstToDfg is changed to convert the Ast into an initial DfgGraph containing only DfgLogic, DfgVertexSplice and DfgVertexVar vertices. The DfgLogic are then subsequently synthesized into primitive operations by the new V3DfgSynthesize pass, which is a combination of the old V3DfgAstToDfg conversion and new code to handle AstAlways blocks with complex flow control. V3DfgSynthesize by default will synthesize roughly the same constructs as V3DfgAstToDfg used to handle before, plus any logic that is part of a combinational cycle within the DfgGraph. This enables breaking up these cycles, for which there are extensions to V3DfgBreakCycles in this patch as well. V3DfgSynthesize will then delete all non synthesized or non synthesizable DfgLogic vertices and the rest of the Dfg pipeline is identical, with minor changes to adjust for the changed representation. Because with this change we can now eliminate many more UNOPTFLAT, DFG has been disabled in all the tests that specifically target testing the scheduling and reporting of circular combinational logic.
2025-08-19 16:06:38 +02:00
DfgVertex* const srcp = varp->srcp();
if (!srcp) return;
if (DfgSpliceArray* const splicep = srcp->cast<DfgSpliceArray>()) {
DfgVertex* const driverp = splicep->driverAt(idxp->toSizeT());
if (!driverp) return;
DfgUnitArray* const uap = driverp->cast<DfgUnitArray>();
if (!uap) return;
if (uap->srcp()->is<DfgVertexSplice>()) return;
// If driven by a variable that had a Driver in DFG, it is partial
if (DfgVertexVar* const dvarp = uap->srcp()->cast<DfgVertexVar>()) {
if (dvarp->srcp()) return;
}
APPLYING(INLINE_ARRAYSEL_SPLICE) {
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(uap->srcp());
Optimize complex combinational logic in DFG (#6298) This patch adds DfgLogic, which is a vertex that represents a whole, arbitrarily complex combinational AstAlways or AstAssignW in the DfgGraph. Implementing this requires computing the variables live at entry to the AstAlways (variables read by the block), so there is a new ControlFlowGraph data structure and a classical data-flow analysis based live variable analysis to do that at the variable level (as opposed to bit/element level). The actual CFG construction and live variable analysis is best effort, and might fail for currently unhandled constructs or data types. This can be extended later. V3DfgAstToDfg is changed to convert the Ast into an initial DfgGraph containing only DfgLogic, DfgVertexSplice and DfgVertexVar vertices. The DfgLogic are then subsequently synthesized into primitive operations by the new V3DfgSynthesize pass, which is a combination of the old V3DfgAstToDfg conversion and new code to handle AstAlways blocks with complex flow control. V3DfgSynthesize by default will synthesize roughly the same constructs as V3DfgAstToDfg used to handle before, plus any logic that is part of a combinational cycle within the DfgGraph. This enables breaking up these cycles, for which there are extensions to V3DfgBreakCycles in this patch as well. V3DfgSynthesize will then delete all non synthesized or non synthesizable DfgLogic vertices and the rest of the Dfg pipeline is identical, with minor changes to adjust for the changed representation. Because with this change we can now eliminate many more UNOPTFLAT, DFG has been disabled in all the tests that specifically target testing the scheduling and reporting of circular combinational logic.
2025-08-19 16:06:38 +02:00
return;
}
}
if (DfgUnitArray* const uap = srcp->cast<DfgUnitArray>()) {
UASSERT_OBJ(idxp->toSizeT() == 0, vtxp, "Array index out of range");
if (uap->srcp()->is<DfgSplicePacked>()) return;
// If driven by a variable that had a Driver in DFG, it is partial
if (DfgVertexVar* const dvarp = uap->srcp()->cast<DfgVertexVar>()) {
if (dvarp->srcp()) return;
}
APPLYING(INLINE_ARRAYSEL_UNIT) {
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(uap->srcp());
Optimize complex combinational logic in DFG (#6298) This patch adds DfgLogic, which is a vertex that represents a whole, arbitrarily complex combinational AstAlways or AstAssignW in the DfgGraph. Implementing this requires computing the variables live at entry to the AstAlways (variables read by the block), so there is a new ControlFlowGraph data structure and a classical data-flow analysis based live variable analysis to do that at the variable level (as opposed to bit/element level). The actual CFG construction and live variable analysis is best effort, and might fail for currently unhandled constructs or data types. This can be extended later. V3DfgAstToDfg is changed to convert the Ast into an initial DfgGraph containing only DfgLogic, DfgVertexSplice and DfgVertexVar vertices. The DfgLogic are then subsequently synthesized into primitive operations by the new V3DfgSynthesize pass, which is a combination of the old V3DfgAstToDfg conversion and new code to handle AstAlways blocks with complex flow control. V3DfgSynthesize by default will synthesize roughly the same constructs as V3DfgAstToDfg used to handle before, plus any logic that is part of a combinational cycle within the DfgGraph. This enables breaking up these cycles, for which there are extensions to V3DfgBreakCycles in this patch as well. V3DfgSynthesize will then delete all non synthesized or non synthesizable DfgLogic vertices and the rest of the Dfg pipeline is identical, with minor changes to adjust for the changed representation. Because with this change we can now eliminate many more UNOPTFLAT, DFG has been disabled in all the tests that specifically target testing the scheduling and reporting of circular combinational logic.
2025-08-19 16:06:38 +02:00
return;
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
}
}
}
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
void visit(DfgConcat* const vtxp) override {
Speed up DfgPeephole in various ways
2022-10-06 13:02:46 +02:00
if (associativeBinary(vtxp)) return;
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
DfgVertex* const lhsp = vtxp->lhsp();
DfgVertex* const rhsp = vtxp->rhsp();
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
FileLine* const flp = vtxp->fileline();
Internals: Improve DFG implementation details (#6355) Large scale refactoring to simplify some of the more obtuse internals of DFG. Remove multiple redundant internal APIs, simplify representation of variables, fix potential unsoundness in circular decomposition. No functional change intended.
2025-09-02 17:50:40 +02:00
if (isZero(lhsp)) {
DfgPeephole: Rework folding of associative operations Allow constant folding through adjacent nodes of all associative operations, for example '((a & 2) & 3)' or '(3 & (2 & a))' can now be folded into '(a & 2)' and '(2 & a)' respectively. Also improve speed of making associative expression trees right leaning by using rotation of the existing vertices whenever instead of allocation of new nodes.
2022-10-05 11:36:28 +02:00
DfgConst* const lConstp = lhsp->as<DfgConst>();
if (DfgSel* const rSelp = rhsp->cast<DfgSel>()) {
Improve Dfg type system (#6390) Added a mini type system for Dfg using DfgDataType to replace Dfg's use of AstNodeDType. This is much more restricted and represents only the types Dfg can handle in a canonical form. This will be needed when adding more support for unpacked arrays and maybe unpacked structs one day. Also added an internal type checker for DfgGraphs which encodes all the assumptions the code makes about type relationships in the graph. Run this in a few places with --debug-check. Fix resulting fallout.
2025-09-07 21:38:50 +02:00
if (vtxp->dtype() == rSelp->fromp()->dtype() && rSelp->lsb() == lConstp->width()) {
DFG: Special case representation of AstSel AstSel is a ternary node, but the 'widthp' is always constant and is hence redundant, and 'lsbp' is very often constant. As AstSel is fairly common, we special case as a DfgSel for the constant 'lsbp', and as 'DfgMux` for the non-constant 'lsbp'.
2022-10-06 19:34:18 +02:00
APPLYING(REPLACE_CONCAT_ZERO_AND_SEL_TOP_WITH_SHIFTR) {
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(
make<DfgShiftR>(vtxp, rSelp->fromp(), makeI32(flp, lConstp->width())));
DFG: Special case representation of AstSel AstSel is a ternary node, but the 'widthp' is always constant and is hence redundant, and 'lsbp' is very often constant. As AstSel is fairly common, we special case as a DfgSel for the constant 'lsbp', and as 'DfgMux` for the non-constant 'lsbp'.
2022-10-06 19:34:18 +02:00
return;
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
}
}
}
DfgPeephole: Rework folding of associative operations Allow constant folding through adjacent nodes of all associative operations, for example '((a & 2) & 3)' or '(3 & (2 & a))' can now be folded into '(a & 2)' and '(2 & a)' respectively. Also improve speed of making associative expression trees right leaning by using rotation of the existing vertices whenever instead of allocation of new nodes.
2022-10-05 11:36:28 +02:00
}
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
Internals: Improve DFG implementation details (#6355) Large scale refactoring to simplify some of the more obtuse internals of DFG. Remove multiple redundant internal APIs, simplify representation of variables, fix potential unsoundness in circular decomposition. No functional change intended.
2025-09-02 17:50:40 +02:00
if (isZero(rhsp)) {
DfgPeephole: Rework folding of associative operations Allow constant folding through adjacent nodes of all associative operations, for example '((a & 2) & 3)' or '(3 & (2 & a))' can now be folded into '(a & 2)' and '(2 & a)' respectively. Also improve speed of making associative expression trees right leaning by using rotation of the existing vertices whenever instead of allocation of new nodes.
2022-10-05 11:36:28 +02:00
DfgConst* const rConstp = rhsp->as<DfgConst>();
if (DfgSel* const lSelp = lhsp->cast<DfgSel>()) {
Improve Dfg type system (#6390) Added a mini type system for Dfg using DfgDataType to replace Dfg's use of AstNodeDType. This is much more restricted and represents only the types Dfg can handle in a canonical form. This will be needed when adding more support for unpacked arrays and maybe unpacked structs one day. Also added an internal type checker for DfgGraphs which encodes all the assumptions the code makes about type relationships in the graph. Run this in a few places with --debug-check. Fix resulting fallout.
2025-09-07 21:38:50 +02:00
if (vtxp->dtype() == lSelp->fromp()->dtype() && lSelp->lsb() == 0) {
DFG: Special case representation of AstSel AstSel is a ternary node, but the 'widthp' is always constant and is hence redundant, and 'lsbp' is very often constant. As AstSel is fairly common, we special case as a DfgSel for the constant 'lsbp', and as 'DfgMux` for the non-constant 'lsbp'.
2022-10-06 19:34:18 +02:00
APPLYING(REPLACE_CONCAT_SEL_BOTTOM_AND_ZERO_WITH_SHIFTL) {
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(
make<DfgShiftL>(vtxp, lSelp->fromp(), makeI32(flp, rConstp->width())));
DFG: Special case representation of AstSel AstSel is a ternary node, but the 'widthp' is always constant and is hence redundant, and 'lsbp' is very often constant. As AstSel is fairly common, we special case as a DfgSel for the constant 'lsbp', and as 'DfgMux` for the non-constant 'lsbp'.
2022-10-06 19:34:18 +02:00
return;
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
}
}
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
}
}
DfgPeephole: Rework folding of associative operations Allow constant folding through adjacent nodes of all associative operations, for example '((a & 2) & 3)' or '(3 & (2 & a))' can now be folded into '(a & 2)' and '(2 & a)' respectively. Also improve speed of making associative expression trees right leaning by using rotation of the existing vertices whenever instead of allocation of new nodes.
2022-10-05 11:36:28 +02:00
if (DfgNot* const lNot = lhsp->cast<DfgNot>()) {
if (DfgNot* const rNot = rhsp->cast<DfgNot>()) {
if (!lNot->hasMultipleSinks() && !rNot->hasMultipleSinks()) {
APPLYING(PUSH_CONCAT_THROUGH_NOTS) {
Avoid creating redundant vertices in V3DfgPeephole (#4944) Add a new data-structure V3DfgCache, which can be used to retrieve existing vertices with some given inputs vertices. Use this in V3DfgPeephole to eliminate the creation of redundant vertices. Overall this is performance neutral, but is in prep for some future work.
2024-03-06 19:01:52 +01:00
DfgConcat* const newCatp
= make<DfgConcat>(vtxp, lNot->srcp(), rNot->srcp());
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(make<DfgNot>(vtxp, newCatp));
DfgPeephole: Rework folding of associative operations Allow constant folding through adjacent nodes of all associative operations, for example '((a & 2) & 3)' or '(3 & (2 & a))' can now be folded into '(a & 2)' and '(2 & a)' respectively. Also improve speed of making associative expression trees right leaning by using rotation of the existing vertices whenever instead of allocation of new nodes.
2022-10-05 11:36:28 +02:00
return;
}
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
}
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
}
}
Internals: Remove unused vertices immediately in DfgPeephole
2026-03-23 16:06:07 +01:00
if (DfgSel* const lSelp = lhsp->cast<DfgSel>()) {
if (DfgSel* const rSelp = rhsp->cast<DfgSel>()) {
Internals: Remove use of DfgVetex::equals from V3DfgPeephole This is both unnecessary and slower than the existing `isSame`.
2025-09-03 11:26:47 +02:00
if (isSame(lSelp->fromp(), rSelp->fromp())) {
DFG: Special case representation of AstSel AstSel is a ternary node, but the 'widthp' is always constant and is hence redundant, and 'lsbp' is very often constant. As AstSel is fairly common, we special case as a DfgSel for the constant 'lsbp', and as 'DfgMux` for the non-constant 'lsbp'.
2022-10-06 19:34:18 +02:00
if (lSelp->lsb() == rSelp->lsb() + rSelp->width()) {
Internals: Remove unused vertices immediately in DfgPeephole
2026-03-23 16:06:07 +01:00
APPLYING(REMOVE_CONCAT_OF_ADJOINING_SELS) {
replace(make<DfgSel>(vtxp, rSelp->fromp(), rSelp->lsb()));
return;
}
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
}
}
}
Internals: Remove unused vertices immediately in DfgPeephole
2026-03-23 16:06:07 +01:00
if (DfgConcat* const rConcatp = rhsp->cast<DfgConcat>()) {
if (DfgSel* const rlSelp = rConcatp->lhsp()->cast<DfgSel>()) {
if (isSame(lSelp->fromp(), rlSelp->fromp())) {
if (lSelp->lsb() == rlSelp->lsb() + rlSelp->width()) {
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
APPLYING(REPLACE_NESTED_CONCAT_OF_ADJOINING_SELS_ON_LHS) {
Internals: Remove unused vertices immediately in DfgPeephole
2026-03-23 16:06:07 +01:00
const uint32_t width = lSelp->width() + rlSelp->width();
const DfgDataType& dtype = DfgDataType::packed(width);
DfgSel* const selp
= make<DfgSel>(flp, dtype, rlSelp->fromp(), rlSelp->lsb());
replace(make<DfgConcat>(vtxp, selp, rConcatp->rhsp()));
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
return;
}
}
}
}
}
Internals: Remove unused vertices immediately in DfgPeephole
2026-03-23 16:06:07 +01:00
}
if (DfgSel* const rSelp = rhsp->cast<DfgSel>()) {
if (DfgConcat* const lConcatp = lhsp->cast<DfgConcat>()) {
if (DfgSel* const lrSelp = lConcatp->rhsp()->cast<DfgSel>()) {
if (isSame(lrSelp->fromp(), rSelp->fromp())) {
if (lrSelp->lsb() == rSelp->lsb() + rSelp->width()) {
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
APPLYING(REPLACE_NESTED_CONCAT_OF_ADJOINING_SELS_ON_RHS) {
Internals: Remove unused vertices immediately in DfgPeephole
2026-03-23 16:06:07 +01:00
const uint32_t width = lrSelp->width() + rSelp->width();
const DfgDataType& dtype = DfgDataType::packed(width);
DfgSel* const selp
= make<DfgSel>(flp, dtype, rSelp->fromp(), rSelp->lsb());
replace(make<DfgConcat>(vtxp, lConcatp->lhsp(), selp));
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
return;
}
}
}
}
}
}
Optimize concatenations that produce unused bits in DFG (#6971) Concatenations that are only used by Sel expressions that do not consume some bits on the edges can be narrowed to not compute the unused bits. E.g.: `{a[4:0], b[4:0]}[5:4]` -> `{a[0], b[4]}[1:0]` This is a superset or the PUSH_SEL_THROUGH_CONCAT DFG pattern, which is removed.
2026-01-31 18:28:14 +01:00
Optimize additional DFG peephole Shift and Concat patterns (#7077)
2026-02-14 13:25:17 +01:00
if (DfgConst* const lConstp = lhsp->cast<DfgConst>()) {
if (DfgCond* const rCondp = rhsp->cast<DfgCond>()) {
Optimize more patterns in DfgPeephole (#7332)
2026-03-27 08:47:50 +01:00
DfgVertex* const rtVtxp = rCondp->thenp();
DfgVertex* const reVtxp = rCondp->elsep();
DfgConst* const rtConstp = rtVtxp->cast<DfgConst>();
DfgConst* const reConstp = reVtxp->cast<DfgConst>();
if (!rCondp->hasMultipleSinks() && (rtConstp || reConstp)) {
Optimize additional DFG peephole Shift and Concat patterns (#7077)
2026-02-14 13:25:17 +01:00
APPLYING(PUSH_CONCAT_THROUGH_COND_LHS) {
Optimize more patterns in DfgPeephole (#7332)
2026-03-27 08:47:50 +01:00
DfgVertex* const thenp = [&]() -> DfgVertex* {
FileLine* const rtFlp = rtVtxp->fileline();
if (rtConstp) {
DfgConst* const constp = makeZero(rtFlp, vtxp->width());
constp->num().opConcat(lConstp->num(), rtConstp->num());
return constp;
}
return make<DfgConcat>(rtFlp, vtxp->dtype(), lConstp, rtVtxp);
}();
DfgVertex* const elsep = [&]() -> DfgVertex* {
FileLine* const reFlp = reVtxp->fileline();
if (reConstp) {
DfgConst* const constp = makeZero(reFlp, vtxp->width());
constp->num().opConcat(lConstp->num(), reConstp->num());
return constp;
}
return make<DfgConcat>(reFlp, vtxp->dtype(), lConstp, reVtxp);
}();
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(make<DfgCond>(vtxp, rCondp->condp(), thenp, elsep));
Optimize additional DFG peephole Shift and Concat patterns (#7077)
2026-02-14 13:25:17 +01:00
return;
}
}
}
}
if (DfgConst* const rConstp = rhsp->cast<DfgConst>()) {
if (DfgCond* const lCondp = lhsp->cast<DfgCond>()) {
Optimize more patterns in DfgPeephole (#7332)
2026-03-27 08:47:50 +01:00
DfgVertex* const ltVtxp = lCondp->thenp();
DfgVertex* const leVtxp = lCondp->elsep();
DfgConst* const ltConstp = ltVtxp->cast<DfgConst>();
DfgConst* const leConstp = leVtxp->cast<DfgConst>();
if (!lCondp->hasMultipleSinks() && (ltConstp || leConstp)) {
Optimize additional DFG peephole Shift and Concat patterns (#7077)
2026-02-14 13:25:17 +01:00
APPLYING(PUSH_CONCAT_THROUGH_COND_RHS) {
Optimize more patterns in DfgPeephole (#7332)
2026-03-27 08:47:50 +01:00
DfgVertex* const thenp = [&]() -> DfgVertex* {
FileLine* const ltFlp = ltVtxp->fileline();
if (ltConstp) {
DfgConst* const constp = makeZero(ltFlp, vtxp->width());
constp->num().opConcat(ltConstp->num(), rConstp->num());
return constp;
}
return make<DfgConcat>(ltFlp, vtxp->dtype(), ltVtxp, rConstp);
}();
DfgVertex* const elsep = [&]() -> DfgVertex* {
FileLine* const leFlp = leVtxp->fileline();
if (leConstp) {
DfgConst* const constp = makeZero(leFlp, vtxp->width());
constp->num().opConcat(leConstp->num(), rConstp->num());
return constp;
}
return make<DfgConcat>(leFlp, vtxp->dtype(), leVtxp, rConstp);
}();
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(make<DfgCond>(vtxp, lCondp->condp(), thenp, elsep));
Optimize additional DFG peephole Shift and Concat patterns (#7077)
2026-02-14 13:25:17 +01:00
return;
}
}
}
}
Optimize concatenations that produce unused bits in DFG (#6971) Concatenations that are only used by Sel expressions that do not consume some bits on the edges can be narrowed to not compute the unused bits. E.g.: `{a[4:0], b[4:0]}[5:4]` -> `{a[0], b[4]}[1:0]` This is a superset or the PUSH_SEL_THROUGH_CONCAT DFG pattern, which is removed.
2026-01-31 18:28:14 +01:00
// Attempt to narrow a concatenation that produces unused bits on the edges
{
const uint32_t vMsb = vtxp->width() - 1; // MSB of the concatenation
const uint32_t lLsb = vtxp->rhsp()->width(); // LSB of the LHS
const uint32_t rMsb = lLsb - 1; // MSB of the RHS
// Check each sink, and record the range of bits used by them
uint32_t lsb = vMsb; // LSB used by a sink
uint32_t msb = 0; // MSB used by a sink
bool hasCrossSink = false; // True if some sinks use bits from both sides
vtxp->foreachSink([&](DfgVertex& sink) {
// Record bits used by DfgSel sinks
if (const DfgSel* const selp = sink.cast<DfgSel>()) {
const uint32_t selLsb = selp->lsb();
const uint32_t selMsb = selLsb + selp->width() - 1;
lsb = std::min(lsb, selLsb);
msb = std::max(msb, selMsb);
hasCrossSink |= selMsb >= lLsb && rMsb >= selLsb;
return false;
}
Commentary: Fix non-grammar
2026-03-12 00:53:23 +01:00
// Ignore non-observable variable sinks. These will be eliminated.
Optimize concatenations that produce unused bits in DFG (#6971) Concatenations that are only used by Sel expressions that do not consume some bits on the edges can be narrowed to not compute the unused bits. E.g.: `{a[4:0], b[4:0]}[5:4]` -> `{a[0], b[4]}[1:0]` This is a superset or the PUSH_SEL_THROUGH_CONCAT DFG pattern, which is removed.
2026-01-31 18:28:14 +01:00
if (const DfgVarPacked* const varp = sink.cast<DfgVarPacked>()) {
if (!varp->hasSinks() && !varp->isObserved()) return false;
}
// Otherwise the whole value is used
lsb = 0;
msb = vMsb;
return true;
});
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
if (hasCrossSink && (vMsb > msb || lsb > 0)) {
Optimize concatenations that produce unused bits in DFG (#6971) Concatenations that are only used by Sel expressions that do not consume some bits on the edges can be narrowed to not compute the unused bits. E.g.: `{a[4:0], b[4:0]}[5:4]` -> `{a[0], b[4]}[1:0]` This is a superset or the PUSH_SEL_THROUGH_CONCAT DFG pattern, which is removed.
2026-01-31 18:28:14 +01:00
APPLYING(NARROW_CONCAT) {
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
// Narrowed RHS
DfgVertex* nRhsp = rhsp;
if (lsb != 0)
nRhsp = make<DfgSel>(flp, DfgDataType::packed(rMsb - lsb + 1), rhsp, lsb);
// Narrowed LHS
DfgVertex* nLhsp = lhsp;
if (msb != vMsb)
nLhsp = make<DfgSel>(flp, DfgDataType::packed(msb - lLsb + 1), lhsp, 0U);
// Narrowed concatenation
DfgVertex* const catp
= make<DfgConcat>(flp, DfgDataType::packed(msb - lsb + 1), nLhsp, nRhsp);
// Need to insert via a partial splice to avoid infinite matching,
// this splice will be eliminated on later visits to its sinks.
DfgVertex::ScopeCache scopeCache;
DfgSplicePacked* const sp = new DfgSplicePacked{m_dfg, flp, vtxp->dtype()};
sp->addDriver(catp, lsb, flp);
Optimize commutative vertex operands in Dfg for better combining
2026-03-24 07:53:19 +01:00
m_vInfo[sp].m_id = ++m_lastId;
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(sp);
Optimize concatenations that produce unused bits in DFG (#6971) Concatenations that are only used by Sel expressions that do not consume some bits on the edges can be narrowed to not compute the unused bits. E.g.: `{a[4:0], b[4:0]}[5:4]` -> `{a[0], b[4]}[1:0]` This is a superset or the PUSH_SEL_THROUGH_CONCAT DFG pattern, which is removed.
2026-01-31 18:28:14 +01:00
return;
}
}
}
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
}
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
void visit(DfgDiv* const vtxp) override {
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
if (foldBinary(vtxp)) return;
}
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
void visit(DfgDivS* const vtxp) override {
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
if (foldBinary(vtxp)) return;
}
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
void visit(DfgEq* const vtxp) override {
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
if (foldBinary(vtxp)) return;
Avoid creating redundant vertices in V3DfgPeephole (#4944) Add a new data-structure V3DfgCache, which can be used to retrieve existing vertices with some given inputs vertices. Use this in V3DfgPeephole to eliminate the creation of redundant vertices. Overall this is performance neutral, but is in prep for some future work.
2024-03-06 19:01:52 +01:00
if (commutativeBinary(vtxp)) return;
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
DfgVertex* const lhsp = vtxp->lhsp();
DfgVertex* const rhsp = vtxp->rhsp();
if (DfgConst* const lhsConstp = lhsp->cast<DfgConst>()) {
if (DfgConcat* const rhsConcatp = rhsp->cast<DfgConcat>()) {
if (tryPushCompareOpThroughConcat(vtxp, lhsConstp, rhsConcatp)) return;
}
}
}
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
void visit(DfgGt* const vtxp) override {
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
if (foldBinary(vtxp)) return;
}
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
void visit(DfgGtS* const vtxp) override {
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
if (foldBinary(vtxp)) return;
}
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
void visit(DfgGte* const vtxp) override {
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
if (foldBinary(vtxp)) return;
}
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
void visit(DfgGteS* const vtxp) override {
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
if (foldBinary(vtxp)) return;
}
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
void visit(DfgLogAnd* const vtxp) override {
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
if (foldBinary(vtxp)) return;
Add DfgPeephole patterns (#6149)
2025-07-07 17:25:29 +02:00
DfgVertex* const lhsp = vtxp->lhsp();
DfgVertex* const rhsp = vtxp->rhsp();
if (lhsp->width() == 1 && rhsp->width() == 1) {
APPLYING(REPLACE_LOGAND_WITH_AND) {
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(make<DfgAnd>(vtxp, lhsp, rhsp));
Add DfgPeephole patterns (#6149)
2025-07-07 17:25:29 +02:00
return;
}
}
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
}
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
void visit(DfgLogEq* const vtxp) override {
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
if (foldBinary(vtxp)) return;
}
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
void visit(DfgLogIf* const vtxp) override {
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
if (foldBinary(vtxp)) return;
}
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
void visit(DfgLogOr* const vtxp) override {
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
if (foldBinary(vtxp)) return;
Add DfgPeephole patterns (#6149)
2025-07-07 17:25:29 +02:00
DfgVertex* const lhsp = vtxp->lhsp();
DfgVertex* const rhsp = vtxp->rhsp();
if (lhsp->width() == 1 && rhsp->width() == 1) {
APPLYING(REPLACE_LOGOR_WITH_OR) {
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(make<DfgOr>(vtxp, lhsp, rhsp));
Add DfgPeephole patterns (#6149)
2025-07-07 17:25:29 +02:00
return;
}
}
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
}
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
void visit(DfgLt* const vtxp) override {
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
if (foldBinary(vtxp)) return;
}
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
void visit(DfgLtS* const vtxp) override {
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
if (foldBinary(vtxp)) return;
}
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
void visit(DfgLte* const vtxp) override {
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
if (foldBinary(vtxp)) return;
}
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
void visit(DfgLteS* const vtxp) override {
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
if (foldBinary(vtxp)) return;
}
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
void visit(DfgModDiv* const vtxp) override {
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
if (foldBinary(vtxp)) return;
}
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
void visit(DfgModDivS* const vtxp) override {
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
if (foldBinary(vtxp)) return;
}
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
void visit(DfgMul* const vtxp) override {
DfgPeephole: Rework folding of associative operations Allow constant folding through adjacent nodes of all associative operations, for example '((a & 2) & 3)' or '(3 & (2 & a))' can now be folded into '(a & 2)' and '(2 & a)' respectively. Also improve speed of making associative expression trees right leaning by using rotation of the existing vertices whenever instead of allocation of new nodes.
2022-10-05 11:36:28 +02:00
if (associativeBinary(vtxp)) return;
Avoid creating redundant vertices in V3DfgPeephole (#4944) Add a new data-structure V3DfgCache, which can be used to retrieve existing vertices with some given inputs vertices. Use this in V3DfgPeephole to eliminate the creation of redundant vertices. Overall this is performance neutral, but is in prep for some future work.
2024-03-06 19:01:52 +01:00
if (commutativeBinary(vtxp)) return;
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
}
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
void visit(DfgMulS* const vtxp) override {
DfgPeephole: Rework folding of associative operations Allow constant folding through adjacent nodes of all associative operations, for example '((a & 2) & 3)' or '(3 & (2 & a))' can now be folded into '(a & 2)' and '(2 & a)' respectively. Also improve speed of making associative expression trees right leaning by using rotation of the existing vertices whenever instead of allocation of new nodes.
2022-10-05 11:36:28 +02:00
if (associativeBinary(vtxp)) return;
Avoid creating redundant vertices in V3DfgPeephole (#4944) Add a new data-structure V3DfgCache, which can be used to retrieve existing vertices with some given inputs vertices. Use this in V3DfgPeephole to eliminate the creation of redundant vertices. Overall this is performance neutral, but is in prep for some future work.
2024-03-06 19:01:52 +01:00
if (commutativeBinary(vtxp)) return;
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
}
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
void visit(DfgNeq* const vtxp) override {
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
if (foldBinary(vtxp)) return;
}
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
void visit(DfgPow* const vtxp) override {
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
if (foldBinary(vtxp)) return;
}
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
void visit(DfgPowSS* const vtxp) override {
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
if (foldBinary(vtxp)) return;
}
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
void visit(DfgPowSU* const vtxp) override {
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
if (foldBinary(vtxp)) return;
}
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
void visit(DfgPowUS* const vtxp) override {
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
if (foldBinary(vtxp)) return;
}
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
void visit(DfgReplicate* const vtxp) override {
Improve Dfg type system (#6390) Added a mini type system for Dfg using DfgDataType to replace Dfg's use of AstNodeDType. This is much more restricted and represents only the types Dfg can handle in a canonical form. This will be needed when adding more support for unpacked arrays and maybe unpacked structs one day. Also added an internal type checker for DfgGraphs which encodes all the assumptions the code makes about type relationships in the graph. Run this in a few places with --debug-check. Fix resulting fallout.
2025-09-07 21:38:50 +02:00
if (vtxp->dtype() == vtxp->srcp()->dtype()) {
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
APPLYING(REMOVE_REPLICATE_ONCE) {
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(vtxp->srcp());
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
return;
}
}
if (foldBinary(vtxp)) return;
}
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
void visit(DfgShiftL* const vtxp) override {
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
if (foldBinary(vtxp)) return;
Avoid creating redundant vertices in V3DfgPeephole (#4944) Add a new data-structure V3DfgCache, which can be used to retrieve existing vertices with some given inputs vertices. Use this in V3DfgPeephole to eliminate the creation of redundant vertices. Overall this is performance neutral, but is in prep for some future work.
2024-03-06 19:01:52 +01:00
if (optimizeShiftRHS(vtxp)) return;
Optimize additional DFG peephole Shift and Concat patterns (#7077)
2026-02-14 13:25:17 +01:00
DfgVertex* const lhsp = vtxp->lhsp();
DfgVertex* const rhsp = vtxp->rhsp();
Optimize more patterns in DfgPeephole (#7332)
2026-03-27 08:47:50 +01:00
if (DfgShiftL* const lShiftLp = lhsp->cast<DfgShiftL>()) {
if (!lShiftLp->hasMultipleSinks() && rhsp->dtype() == lShiftLp->rhsp()->dtype()) {
APPLYING(REPLACE_SHIFTL_SHIFTL) {
DfgAdd* const addp = make<DfgAdd>(rhsp, rhsp, lShiftLp->rhsp());
replace(make<DfgShiftL>(vtxp, lShiftLp->lhsp(), addp));
return;
}
}
}
Optimize additional DFG peephole Shift and Concat patterns (#7077)
2026-02-14 13:25:17 +01:00
if (DfgConst* const rConstp = rhsp->cast<DfgConst>()) {
Optimize more patterns in DfgPeephole (#7332)
2026-03-27 08:47:50 +01:00
const uint32_t shiftAmount = rConstp->toU32();
if (shiftAmount == 0) {
APPLYING(REMOVE_SHIFTL_ZERO) {
replace(lhsp);
return;
}
}
if (shiftAmount >= vtxp->width()) {
APPLYING(REPLACE_SHIFTL_OVER) {
replace(makeZero(vtxp->fileline(), vtxp->width()));
return;
}
}
Optimize additional DFG peephole Shift and Concat patterns (#7077)
2026-02-14 13:25:17 +01:00
if (DfgConcat* const lConcatp = lhsp->cast<DfgConcat>()) {
Optimize more patterns in DfgPeephole (#7332)
2026-03-27 08:47:50 +01:00
if (!lConcatp->hasMultipleSinks()) {
Optimize additional DFG peephole Shift and Concat patterns (#7077)
2026-02-14 13:25:17 +01:00
APPLYING(REPLACE_SHIFTL_CAT) {
Optimize more patterns in DfgPeephole (#7332)
2026-03-27 08:47:50 +01:00
DfgVertex* const lRhsp = lConcatp->rhsp();
DfgVertex* const lLhsp = lConcatp->lhsp();
// Compute widths of 3 possible result parts
const uint32_t rW = shiftAmount;
const uint32_t mW = std::min(lRhsp->width(), vtxp->width() - shiftAmount);
const uint32_t lW = vtxp->width() - mW - rW;
// Construct the result
FileLine* const flp = lConcatp->fileline();
DfgVertex* const rp = makeZero(flp, shiftAmount);
DfgVertex* const mp
= make<DfgSel>(flp, DfgDataType::packed(mW), lRhsp, 0U);
DfgVertex* np = make<DfgConcat>(flp, DfgDataType::packed(mW + rW), mp, rp);
if (!lW) {
replace(np);
return;
}
DfgVertex* const lp
= make<DfgSel>(flp, DfgDataType::packed(lW), lLhsp, 0U);
np = make<DfgConcat>(vtxp, lp, np);
replace(np);
Optimize additional DFG peephole Shift and Concat patterns (#7077)
2026-02-14 13:25:17 +01:00
return;
}
}
}
Optimize more patterns in DfgPeephole (#7332)
2026-03-27 08:47:50 +01:00
if (DfgSel* const lSelp = lhsp->cast<DfgSel>()) {
if (!lSelp->hasMultipleSinks()) {
APPLYING(REPLACE_SHIFTL_SEL) {
const uint32_t nSelWidth = lSelp->width() - shiftAmount;
DfgVertex* const nSelp
= make<DfgSel>(lSelp->fileline(), DfgDataType::packed(nSelWidth),
lSelp->fromp(), lSelp->lsb());
replace(
make<DfgConcat>(vtxp, nSelp, makeZero(vtxp->fileline(), shiftAmount)));
return;
}
}
}
if (DfgCond* const lCondp = lhsp->cast<DfgCond>()) {
if (!lCondp->hasMultipleSinks()) {
APPLYING(PUSH_SHIFTL_THROUGH_COND) {
DfgShiftL* const tp = make<DfgShiftL>(vtxp, lCondp->thenp(), rConstp);
DfgShiftL* const ep = make<DfgShiftL>(vtxp, lCondp->elsep(), rConstp);
replace(make<DfgCond>(vtxp, lCondp->condp(), tp, ep));
return;
Optimize additional DFG peephole Shift and Concat patterns (#7077)
2026-02-14 13:25:17 +01:00
}
}
}
}
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
}
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
void visit(DfgShiftR* const vtxp) override {
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
if (foldBinary(vtxp)) return;
Avoid creating redundant vertices in V3DfgPeephole (#4944) Add a new data-structure V3DfgCache, which can be used to retrieve existing vertices with some given inputs vertices. Use this in V3DfgPeephole to eliminate the creation of redundant vertices. Overall this is performance neutral, but is in prep for some future work.
2024-03-06 19:01:52 +01:00
if (optimizeShiftRHS(vtxp)) return;
Optimize more patterns in DfgPeephole (#7332)
2026-03-27 08:47:50 +01:00
DfgVertex* const lhsp = vtxp->lhsp();
DfgVertex* const rhsp = vtxp->rhsp();
if (DfgShiftR* const lShiftRp = lhsp->cast<DfgShiftR>()) {
if (!lShiftRp->hasMultipleSinks() && rhsp->dtype() == lShiftRp->rhsp()->dtype()) {
APPLYING(REPLACE_SHIFTR_SHIFTR) {
DfgAdd* const addp = make<DfgAdd>(rhsp, rhsp, lShiftRp->rhsp());
replace(make<DfgShiftR>(vtxp, lShiftRp->lhsp(), addp));
return;
}
}
}
if (DfgConst* const rConstp = rhsp->cast<DfgConst>()) {
const uint32_t shiftAmount = rConstp->toU32();
if (shiftAmount == 0) {
APPLYING(REMOVE_SHIFTR_ZERO) {
replace(lhsp);
return;
}
}
if (shiftAmount >= vtxp->width()) {
APPLYING(REPLACE_SHIFTR_OVER) {
replace(makeZero(vtxp->fileline(), vtxp->width()));
return;
}
}
if (DfgConcat* const lConcatp = lhsp->cast<DfgConcat>()) {
if (!lConcatp->hasMultipleSinks()) {
APPLYING(REPLACE_SHIFTR_CAT) {
DfgVertex* const lRhsp = lConcatp->rhsp();
DfgVertex* const lLhsp = lConcatp->lhsp();
// Compute widths of 3 possible result parts
const uint32_t lW = shiftAmount;
const uint32_t mW = std::min(lLhsp->width(), vtxp->width() - shiftAmount);
const uint32_t rW = vtxp->width() - mW - lW;
// Construct the result
FileLine* const flp = lConcatp->fileline();
DfgVertex* const lp = makeZero(flp, shiftAmount);
DfgVertex* const mp = make<DfgSel>(flp, DfgDataType::packed(mW), lLhsp,
lLhsp->width() - mW);
DfgVertex* np = make<DfgConcat>(flp, DfgDataType::packed(lW + mW), lp, mp);
if (!rW) {
replace(np);
return;
}
DfgVertex* const rp = make<DfgSel>(flp, DfgDataType::packed(rW), lRhsp,
lRhsp->width() - rW);
np = make<DfgConcat>(vtxp, np, rp);
replace(np);
return;
}
}
}
if (DfgSel* const lSelp = lhsp->cast<DfgSel>()) {
if (!lSelp->hasMultipleSinks()) {
APPLYING(REPLACE_SHIFTR_SEL) {
const uint32_t nSelWidth = lSelp->width() - shiftAmount;
DfgVertex* const nSelp
= make<DfgSel>(lSelp->fileline(), DfgDataType::packed(nSelWidth),
lSelp->fromp(), lSelp->lsb() + shiftAmount);
replace(
make<DfgConcat>(vtxp, makeZero(vtxp->fileline(), shiftAmount), nSelp));
return;
}
}
}
if (DfgCond* const lCondp = lhsp->cast<DfgCond>()) {
if (!lCondp->hasMultipleSinks()) {
APPLYING(PUSH_SHIFTR_THROUGH_COND) {
DfgShiftR* const tp = make<DfgShiftR>(vtxp, lCondp->thenp(), rConstp);
DfgShiftR* const ep = make<DfgShiftR>(vtxp, lCondp->elsep(), rConstp);
replace(make<DfgCond>(vtxp, lCondp->condp(), tp, ep));
return;
}
}
}
}
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
}
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
void visit(DfgShiftRS* const vtxp) override {
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
if (foldBinary(vtxp)) return;
Avoid creating redundant vertices in V3DfgPeephole (#4944) Add a new data-structure V3DfgCache, which can be used to retrieve existing vertices with some given inputs vertices. Use this in V3DfgPeephole to eliminate the creation of redundant vertices. Overall this is performance neutral, but is in prep for some future work.
2024-03-06 19:01:52 +01:00
if (optimizeShiftRHS(vtxp)) return;
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
}
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
void visit(DfgSub* const vtxp) override {
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
if (foldBinary(vtxp)) return;
DfgVertex* const lhsp = vtxp->lhsp();
DfgVertex* const rhsp = vtxp->rhsp();
if (DfgConst* const rConstp = rhsp->cast<DfgConst>()) {
if (rConstp->isZero()) {
APPLYING(REMOVE_SUB_ZERO) {
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(lhsp);
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
return;
}
}
Improve Dfg type system (#6390) Added a mini type system for Dfg using DfgDataType to replace Dfg's use of AstNodeDType. This is much more restricted and represents only the types Dfg can handle in a canonical form. This will be needed when adding more support for unpacked arrays and maybe unpacked structs one day. Also added an internal type checker for DfgGraphs which encodes all the assumptions the code makes about type relationships in the graph. Run this in a few places with --debug-check. Fix resulting fallout.
2025-09-07 21:38:50 +02:00
if (vtxp->dtype() == m_bitDType && rConstp->hasValue(1)) {
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
APPLYING(REPLACE_SUB_WITH_NOT) {
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(make<DfgNot>(vtxp->fileline(), m_bitDType, lhsp));
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
return;
}
}
}
}
//=========================================================================
// DfgVertexTernary
//=========================================================================
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
void visit(DfgCond* const vtxp) override {
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
DfgVertex* const condp = vtxp->condp();
DfgVertex* const thenp = vtxp->thenp();
DfgVertex* const elsep = vtxp->elsep();
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
FileLine* const flp = vtxp->fileline();
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
Improve Dfg type system (#6390) Added a mini type system for Dfg using DfgDataType to replace Dfg's use of AstNodeDType. This is much more restricted and represents only the types Dfg can handle in a canonical form. This will be needed when adding more support for unpacked arrays and maybe unpacked structs one day. Also added an internal type checker for DfgGraphs which encodes all the assumptions the code makes about type relationships in the graph. Run this in a few places with --debug-check. Fix resulting fallout.
2025-09-07 21:38:50 +02:00
if (condp->dtype() != m_bitDType) return;
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
Internals: Improve DFG implementation details (#6355) Large scale refactoring to simplify some of the more obtuse internals of DFG. Remove multiple redundant internal APIs, simplify representation of variables, fix potential unsoundness in circular decomposition. No functional change intended.
2025-09-02 17:50:40 +02:00
if (isOnes(condp)) {
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
APPLYING(REMOVE_COND_WITH_TRUE_CONDITION) {
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(thenp);
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
return;
}
}
Internals: Improve DFG implementation details (#6355) Large scale refactoring to simplify some of the more obtuse internals of DFG. Remove multiple redundant internal APIs, simplify representation of variables, fix potential unsoundness in circular decomposition. No functional change intended.
2025-09-02 17:50:40 +02:00
if (isZero(condp)) {
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
APPLYING(REMOVE_COND_WITH_FALSE_CONDITION) {
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(elsep);
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
return;
}
}
Add DfgPeephole patterns (#6149)
2025-07-07 17:25:29 +02:00
if (isSame(thenp, elsep)) {
APPLYING(REMOVE_COND_WITH_BRANCHES_SAME) {
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(elsep);
Add DfgPeephole patterns (#6149)
2025-07-07 17:25:29 +02:00
return;
}
}
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
if (DfgNot* const condNotp = condp->cast<DfgNot>()) {
Try to ensure DFG peephole patterns don't grow the graph Some optimizations are only a net win if they help us remove a graph node (or at least ensure they don't grow the graph), or yields otherwise special logic, so try to apply them only in these cases.
2022-10-04 19:26:53 +02:00
if (!condp->hasMultipleSinks() || condNotp->hasMultipleSinks()) {
APPLYING(SWAP_COND_WITH_NOT_CONDITION) {
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(make<DfgCond>(vtxp, condNotp->srcp(), elsep, thenp));
Try to ensure DFG peephole patterns don't grow the graph Some optimizations are only a net win if they help us remove a graph node (or at least ensure they don't grow the graph), or yields otherwise special logic, so try to apply them only in these cases.
2022-10-04 19:26:53 +02:00
return;
}
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
}
}
if (DfgNeq* const condNeqp = condp->cast<DfgNeq>()) {
Try to ensure DFG peephole patterns don't grow the graph Some optimizations are only a net win if they help us remove a graph node (or at least ensure they don't grow the graph), or yields otherwise special logic, so try to apply them only in these cases.
2022-10-04 19:26:53 +02:00
if (!condp->hasMultipleSinks()) {
APPLYING(SWAP_COND_WITH_NEQ_CONDITION) {
Avoid creating redundant vertices in V3DfgPeephole (#4944) Add a new data-structure V3DfgCache, which can be used to retrieve existing vertices with some given inputs vertices. Use this in V3DfgPeephole to eliminate the creation of redundant vertices. Overall this is performance neutral, but is in prep for some future work.
2024-03-06 19:01:52 +01:00
DfgEq* const newCondp = make<DfgEq>(condp, condNeqp->lhsp(), condNeqp->rhsp());
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(make<DfgCond>(vtxp, newCondp, elsep, thenp));
Try to ensure DFG peephole patterns don't grow the graph Some optimizations are only a net win if they help us remove a graph node (or at least ensure they don't grow the graph), or yields otherwise special logic, so try to apply them only in these cases.
2022-10-04 19:26:53 +02:00
return;
}
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
}
}
if (DfgNot* const thenNotp = thenp->cast<DfgNot>()) {
if (DfgNot* const elseNotp = elsep->cast<DfgNot>()) {
DfgPeephole: Use a work list driven algorithm for speed Replace the 'run to fixed point' algorithm with a work list driven approach. Instead of marking the graph as changed, we explicitly add vertices to the work list, to be visited, when a vertex is changed. This improves both memory locality (as the work list is processed in last in first out order), and removed unnecessary visitations when only a few nodes changes.
2022-11-04 16:50:20 +01:00
if (!thenNotp->srcp()->is<DfgConst>() && !elseNotp->srcp()->is<DfgConst>()
Fix tautological predicate in V3DfgPeephole Fixes #3771
2022-11-22 14:48:49 +01:00
&& !thenNotp->hasMultipleSinks() && !elseNotp->hasMultipleSinks()) {
Try to ensure DFG peephole patterns don't grow the graph Some optimizations are only a net win if they help us remove a graph node (or at least ensure they don't grow the graph), or yields otherwise special logic, so try to apply them only in these cases.
2022-10-04 19:26:53 +02:00
APPLYING(PULL_NOTS_THROUGH_COND) {
Avoid creating redundant vertices in V3DfgPeephole (#4944) Add a new data-structure V3DfgCache, which can be used to retrieve existing vertices with some given inputs vertices. Use this in V3DfgPeephole to eliminate the creation of redundant vertices. Overall this is performance neutral, but is in prep for some future work.
2024-03-06 19:01:52 +01:00
DfgCond* const newCondp = make<DfgCond>(
vtxp, vtxp->condp(), thenNotp->srcp(), elseNotp->srcp());
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(make<DfgNot>(thenp->fileline(), vtxp->dtype(), newCondp));
Try to ensure DFG peephole patterns don't grow the graph Some optimizations are only a net win if they help us remove a graph node (or at least ensure they don't grow the graph), or yields otherwise special logic, so try to apply them only in these cases.
2022-10-04 19:26:53 +02:00
return;
}
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
}
}
}
Add DfgPeephole patterns (#6149)
2025-07-07 17:25:29 +02:00
if (DfgOr* const condOrp = condp->cast<DfgOr>()) {
if (DfgCond* const thenCondp = thenp->cast<DfgCond>()) {
if (!thenCondp->hasMultipleSinks()) {
if (condOrp->lhsp() == thenCondp->condp()) {
// '(a | b) ? (a ? x : y) : z' -> 'a ? x : b ? y : z'
APPLYING(REPLACE_COND_OR_THEN_COND_LHS) {
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
DfgCond* const ep = make<DfgCond>(thenCondp, condOrp->rhsp(),
thenCondp->elsep(), elsep);
replace(make<DfgCond>(vtxp, condOrp->lhsp(), thenCondp->thenp(), ep));
Add DfgPeephole patterns (#6149)
2025-07-07 17:25:29 +02:00
return;
}
}
if (condOrp->rhsp() == thenCondp->condp()) {
// '(a | b) ? (a ? x : y) : z' -> 'a ? x : b ? y : z'
APPLYING(REPLACE_COND_OR_THEN_COND_RHS) {
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
DfgCond* const ep = make<DfgCond>(thenCondp, condOrp->lhsp(),
thenCondp->elsep(), elsep);
replace(make<DfgCond>(vtxp, condOrp->rhsp(), thenCondp->thenp(), ep));
Add DfgPeephole patterns (#6149)
2025-07-07 17:25:29 +02:00
return;
}
}
}
}
}
DFG: Add more peephole patterns
2022-09-30 17:19:53 +02:00
if (vtxp->width() > 1) {
// 'cond ? a + 1 : a' -> 'a + cond'
if (DfgAdd* const thenAddp = thenp->cast<DfgAdd>()) {
if (DfgConst* const constp = thenAddp->lhsp()->cast<DfgConst>()) {
DFG: Fix attempted evaluation of constants wider than 32 bits Fixes #3718
2022-10-28 18:00:32 +02:00
if (constp->hasValue(1)) {
DFG: Add more peephole patterns
2022-09-30 17:19:53 +02:00
if (thenAddp->rhsp() == elsep) {
APPLYING(REPLACE_COND_INC) {
Avoid creating redundant vertices in V3DfgPeephole (#4944) Add a new data-structure V3DfgCache, which can be used to retrieve existing vertices with some given inputs vertices. Use this in V3DfgPeephole to eliminate the creation of redundant vertices. Overall this is performance neutral, but is in prep for some future work.
2024-03-06 19:01:52 +01:00
DfgConcat* const extp = make<DfgConcat>(
vtxp, makeZero(flp, vtxp->width() - 1), condp);
DFG: Add more peephole patterns
2022-09-30 17:19:53 +02:00
FileLine* const thenFlp = thenAddp->fileline();
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(
make<DfgAdd>(thenFlp, vtxp->dtype(), thenAddp->rhsp(), extp));
DFG: Add more peephole patterns
2022-09-30 17:19:53 +02:00
return;
}
}
}
}
}
// 'cond ? a - 1 : a' -> 'a - cond'
if (DfgSub* const thenSubp = thenp->cast<DfgSub>()) {
if (DfgConst* const constp = thenSubp->rhsp()->cast<DfgConst>()) {
DFG: Fix attempted evaluation of constants wider than 32 bits Fixes #3718
2022-10-28 18:00:32 +02:00
if (constp->hasValue(1)) {
DFG: Add more peephole patterns
2022-09-30 17:19:53 +02:00
if (thenSubp->lhsp() == elsep) {
APPLYING(REPLACE_COND_DEC) {
Avoid creating redundant vertices in V3DfgPeephole (#4944) Add a new data-structure V3DfgCache, which can be used to retrieve existing vertices with some given inputs vertices. Use this in V3DfgPeephole to eliminate the creation of redundant vertices. Overall this is performance neutral, but is in prep for some future work.
2024-03-06 19:01:52 +01:00
DfgConcat* const extp = make<DfgConcat>(
vtxp, makeZero(flp, vtxp->width() - 1), condp);
DFG: Add more peephole patterns
2022-09-30 17:19:53 +02:00
FileLine* const thenFlp = thenSubp->fileline();
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(
make<DfgSub>(thenFlp, vtxp->dtype(), thenSubp->lhsp(), extp));
DFG: Add more peephole patterns
2022-09-30 17:19:53 +02:00
return;
}
}
}
}
}
}
Improve Dfg type system (#6390) Added a mini type system for Dfg using DfgDataType to replace Dfg's use of AstNodeDType. This is much more restricted and represents only the types Dfg can handle in a canonical form. This will be needed when adding more support for unpacked arrays and maybe unpacked structs one day. Also added an internal type checker for DfgGraphs which encodes all the assumptions the code makes about type relationships in the graph. Run this in a few places with --debug-check. Fix resulting fallout.
2025-09-07 21:38:50 +02:00
if (vtxp->dtype() == m_bitDType) {
Internals: Improve DFG implementation details (#6355) Large scale refactoring to simplify some of the more obtuse internals of DFG. Remove multiple redundant internal APIs, simplify representation of variables, fix potential unsoundness in circular decomposition. No functional change intended.
2025-09-02 17:50:40 +02:00
if (isZero(thenp)) { // a ? 0 : b becomes ~a & b
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
APPLYING(REPLACE_COND_WITH_THEN_BRANCH_ZERO) {
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(make<DfgAnd>(vtxp, make<DfgNot>(vtxp, condp), elsep));
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
return;
}
}
Add DfgPeephole patterns (#6149)
2025-07-07 17:25:29 +02:00
if (thenp == condp) { // a ? a : b becomes a | b
APPLYING(REPLACE_COND_WITH_THEN_BRANCH_COND) {
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(make<DfgOr>(vtxp, condp, elsep));
Add DfgPeephole patterns (#6149)
2025-07-07 17:25:29 +02:00
return;
}
}
Optimize more patterns in DfgPeephole (#7332)
2026-03-27 08:47:50 +01:00
if (elsep == condp) { // a ? b : a becomes a & b
APPLYING(REPLACE_COND_WITH_ELSE_BRANCH_COND) {
replace(make<DfgAnd>(vtxp, condp, thenp));
return;
}
}
Internals: Improve DFG implementation details (#6355) Large scale refactoring to simplify some of the more obtuse internals of DFG. Remove multiple redundant internal APIs, simplify representation of variables, fix potential unsoundness in circular decomposition. No functional change intended.
2025-09-02 17:50:40 +02:00
if (isOnes(thenp)) { // a ? 1 : b becomes a | b
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
APPLYING(REPLACE_COND_WITH_THEN_BRANCH_ONES) {
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(make<DfgOr>(vtxp, condp, elsep));
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
return;
}
}
Internals: Improve DFG implementation details (#6355) Large scale refactoring to simplify some of the more obtuse internals of DFG. Remove multiple redundant internal APIs, simplify representation of variables, fix potential unsoundness in circular decomposition. No functional change intended.
2025-09-02 17:50:40 +02:00
if (isZero(elsep)) { // a ? b : 0 becomes a & b
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
APPLYING(REPLACE_COND_WITH_ELSE_BRANCH_ZERO) {
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(make<DfgAnd>(vtxp, condp, thenp));
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
return;
}
}
Internals: Improve DFG implementation details (#6355) Large scale refactoring to simplify some of the more obtuse internals of DFG. Remove multiple redundant internal APIs, simplify representation of variables, fix potential unsoundness in circular decomposition. No functional change intended.
2025-09-02 17:50:40 +02:00
if (isOnes(elsep)) { // a ? b : 1 becomes ~a | b
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
APPLYING(REPLACE_COND_WITH_ELSE_BRANCH_ONES) {
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(make<DfgOr>(vtxp, make<DfgNot>(vtxp, condp), thenp));
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
return;
}
}
}
Optimize more patterns in DfgPeephole (#7332)
2026-03-27 08:47:50 +01:00
if (DfgConcat* const tConcatp = thenp->cast<DfgConcat>()) {
if (DfgConcat* const eConcatp = elsep->cast<DfgConcat>()) {
DfgVertex* const tRhsp = tConcatp->rhsp();
DfgVertex* const tLhsp = tConcatp->lhsp();
DfgVertex* const eRhsp = eConcatp->rhsp();
DfgVertex* const eLhsp = eConcatp->lhsp();
if (isSame(tRhsp, eRhsp)) {
APPLYING(REPLACE_COND_SAME_CAT_RHS) {
DfgCond* const newCondp
= make<DfgCond>(flp, tLhsp->dtype(), condp, tLhsp, eLhsp);
replace(make<DfgConcat>(vtxp, newCondp, tRhsp));
return;
}
}
if (isSame(tLhsp, eLhsp)) {
APPLYING(REPLACE_COND_SAME_CAT_LHS) {
DfgCond* const newCondp
= make<DfgCond>(flp, tRhsp->dtype(), condp, tRhsp, eRhsp);
replace(make<DfgConcat>(vtxp, tLhsp, newCondp));
return;
}
}
}
}
if (isZero(elsep) && isEqOne(thenp)) {
APPLYING(REPLACE_COND_CONST_ONE_ZERO) {
DfgVertex* resp = condp;
if (const uint32_t extend = vtxp->width() - 1) {
DfgConst* const zerop = makeZero(flp, extend);
resp = make<DfgConcat>(vtxp, zerop, resp);
}
replace(resp);
return;
}
}
if (isZero(thenp) && isEqOne(elsep)) {
APPLYING(REPLACE_COND_CONST_ZERO_ONE) {
DfgVertex* resp = make<DfgNot>(condp, condp);
if (const uint32_t extend = vtxp->width() - 1) {
DfgConst* const zerop = makeZero(vtxp->fileline(), extend);
resp = make<DfgConcat>(vtxp, zerop, resp);
}
replace(resp);
return;
}
}
if (DfgCond* const tCondp = thenp->cast<DfgCond>()) {
if (isSame(condp, tCondp->condp())) {
APPLYING(REPLACE_COND_SAME_COND_THEN) {
replace(make<DfgCond>(vtxp, condp, tCondp->thenp(), elsep));
return;
}
}
}
if (DfgCond* const eCondp = elsep->cast<DfgCond>()) {
if (isSame(condp, eCondp->condp())) {
APPLYING(REPLACE_COND_SAME_COND_ELSE) {
replace(make<DfgCond>(vtxp, condp, thenp, eCondp->elsep()));
return;
}
}
}
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
}
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
void visit(DfgVertexVar* const vtxp) override {
Optimize: Remove redundant variables during Dfg Peephole pass (#7076) Many rules in the Dfg Peephole pass check if a node has more than one sinks. Redundant variables that will ultimately be removed can prevent these from matching. Remove such variables during the Peeophole pass itself to enable more matches.
2026-02-14 13:33:20 +01:00
if (vtxp->hasSinks()) return;
if (vtxp->isObserved()) return;
if (vtxp->defaultp()) return;
// If undriven, or driven from another var, it is completely redundant.
if (!vtxp->srcp() || vtxp->srcp()->is<DfgVertexVar>()) {
APPLYING(REMOVE_VAR) {
deleteVertex(vtxp);
return;
}
}
// Otherwise remove if there is only one sink that is not a removable variable
bool foundOne = false;
const bool keep = vtxp->srcp()->foreachSink([&](DfgVertex& sink) {
Commentary: Fix non-grammar
2026-03-12 00:53:23 +01:00
// Ignore non-observable variable sinks. These can be eliminated.
Optimize: Remove redundant variables during Dfg Peephole pass (#7076) Many rules in the Dfg Peephole pass check if a node has more than one sinks. Redundant variables that will ultimately be removed can prevent these from matching. Remove such variables during the Peeophole pass itself to enable more matches.
2026-02-14 13:33:20 +01:00
if (const DfgVertexVar* const varp = sink.cast<DfgVertexVar>()) {
if (!varp->hasSinks() && !varp->isObserved()) return false;
}
if (foundOne) return true;
foundOne = true;
return false;
});
if (!keep) {
APPLYING(REMOVE_VAR) {
deleteVertex(vtxp);
return;
}
}
}
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
V3DfgPeephole(DfgGraph& dfg, V3DfgPeepholeContext& ctx)
: m_dfg{dfg}
Speed up DfgPeephole in various ways
2022-10-06 13:02:46 +02:00
, m_ctx{ctx} {
Optimize commutative vertex operands in Dfg for better combining
2026-03-24 07:53:19 +01:00
// Assign vertex IDs
m_dfg.forEachVertex([&](DfgVertex& vtx) { m_vInfo[vtx].m_id = ++m_lastId; });
Optimize DfgPeephole until fixed point (#7309)
2026-03-26 07:56:36 +01:00
// Initialize the work list and iter list. They can't get bigger than
// m_dfg.size(), but new vertices are created in the loop, so over alloacte
m_workList.reserve(m_dfg.size() * 2);
m_iterList.reserve(m_dfg.size() * 2);
Internals: Remove unused vertices immediately in DfgPeephole
2026-03-23 16:06:07 +01:00
Optimize DfgPeephole until fixed point (#7309)
2026-03-26 07:56:36 +01:00
// Need a nullptr at index 0 so VertexInfo::m_*ListIndex == 0 can check membership
m_workList.push_back(nullptr);
m_iterList.push_back(nullptr);
Internals: Use a custom worklist in DfgPeephole
2026-03-23 15:41:36 +01:00
Optimize DfgPeephole until fixed point (#7309)
2026-03-26 07:56:36 +01:00
// Add all variable vertices to the work list. Do this first so they are processed
// last. This order has a better chance of preserving original variables in case
// they are needed to hold intermediate results.
Optimize: Remove redundant variables during Dfg Peephole pass (#7076) Many rules in the Dfg Peephole pass check if a node has more than one sinks. Redundant variables that will ultimately be removed can prevent these from matching. Remove such variables during the Peeophole pass itself to enable more matches.
2026-02-14 13:33:20 +01:00
for (DfgVertexVar& vtx : m_dfg.varVertices()) addToWorkList(&vtx);
Internals: Use a custom worklist in DfgPeephole
2026-03-23 15:41:36 +01:00
// Add all operation vertices to the work list
Internals: Simplify V3DfgCache
2026-03-22 11:19:31 +01:00
for (DfgVertex& vtx : m_dfg.opVertices()) addToWorkList(&vtx);
DfgPeephole: Use a work list driven algorithm for speed Replace the 'run to fixed point' algorithm with a work list driven approach. Instead of marking the graph as changed, we explicitly add vertices to the work list, to be visited, when a vertex is changed. This improves both memory locality (as the work list is processed in last in first out order), and removed unnecessary visitations when only a few nodes changes.
2022-11-04 16:50:20 +01:00
Optimize DfgPeephole until fixed point (#7309)
2026-03-26 07:56:36 +01:00
// Process iteratively
while (true) {
// Process the work list - keep the placeholder at index 0
while (m_workList.size() > 1) {
VL_RESTORER(m_vtxp);
// Pop up head of work list
m_vtxp = m_workList.back();
m_workList.pop_back();
if (!m_vtxp) continue; // If removed from worklist (deleted), move on
m_vInfo[m_vtxp].m_workListIndex = 0; // No longer on work list
// Variables are special, just visit them, the visit might delete them
if (DfgVertexVar* const varp = m_vtxp->cast<DfgVertexVar>()) {
visit(varp);
continue;
}
Internals: Use a custom worklist in DfgPeephole
2026-03-23 15:41:36 +01:00
Optimize DfgPeephole until fixed point (#7309)
2026-03-26 07:56:36 +01:00
// Unsued vertices should have been removed immediately
UASSERT_OBJ(m_vtxp->hasSinks(), m_vtxp, "Operation vertex should have sinks");
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
Optimize DfgPeephole until fixed point (#7309)
2026-03-26 07:56:36 +01:00
// Check if an equivalent vertex exists, if so replace this vertex with it
if (DfgVertex* const sampep = m_cache.cache(m_vtxp)) {
APPLYING(REPLACE_WITH_EQUIVALENT) {
replace(sampep);
continue;
}
}
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
Optimize DfgPeephole until fixed point (#7309)
2026-03-26 07:56:36 +01:00
// Visit vertex, might get deleted in the process
iterate(m_vtxp);
}
// If nothing was added to the iter list, we can stop
if (m_iterList.size() == 1) break;
// Expand the iter list to visit the whole neighborhood of vertices
// within a fixed number of hops from the enqueued vertices. This
// enables patterns that look deeper across the graph to be reconsidered.
{
size_t begin = 0;
size_t end = m_iterList.size();
for (size_t hops = 1; hops <= 4; ++hops) {
for (size_t i = begin; i < end; ++i) {
DfgVertex* const vtxp = m_iterList[i];
if (!vtxp) continue;
vtxp->foreachSink([&](DfgVertex& dst) {
addToIterList(&dst);
return false;
});
vtxp->foreachSource([&](DfgVertex& src) {
addToIterList(&src);
return false;
});
}
begin = end;
end = m_iterList.size();
Internals: Remove unused vertices immediately in DfgPeephole
2026-03-23 16:06:07 +01:00
}
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
}
Optimize DfgPeephole until fixed point (#7309)
2026-03-26 07:56:36 +01:00
// Move vertices in the iter list to the work list for the next iteration
for (DfgVertex* const vtxp : m_iterList) {
if (!vtxp) continue;
removeFromIterList(vtxp);
addToWorkList(vtxp);
}
// Reset the iter list
m_iterList.resize(1);
Internals: Use a custom worklist in DfgPeephole
2026-03-23 15:41:36 +01:00
}
Speed up DfgPeephole in various ways
2022-10-06 13:02:46 +02:00
}
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
Internals: Remove unused vertices immediately in DfgPeephole
2026-03-23 16:06:07 +01:00
#undef APPLYING
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
public:
Speed up DfgPeephole in various ways
2022-10-06 13:02:46 +02:00
static void apply(DfgGraph& dfg, V3DfgPeepholeContext& ctx) { V3DfgPeephole{dfg, ctx}; }
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
};
Internals: Add utility to perform bisection search for debugging (#7294)
2026-03-21 11:13:27 +01:00
V3DebugBisect V3DfgPeephole::s_debugBisect{"DfgPeephole"};
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
void V3DfgPasses::peephole(DfgGraph& dfg, V3DfgPeepholeContext& ctx) {
Improve Dfg variable removal and temporary insertion (#6401) Combined Dfg variable elimination into the regularization pass that runs before converting back to Ast. This avoids introducing some unnecessary temporaries. Added replacing of variables with constants in the Ast if after the Dfg passes they are known to be constants. This is only done in final scoped Dfg application. Avoid introducing temporaries for common sub-expressions that are cheaper to re-compute than store in a temporary variable. Enable removal of redundant unpacked array variables. Also fixes #6394 as this patch involved changes to that code.
2025-09-10 13:38:49 +02:00
if (!v3Global.opt.fDfgPeephole()) return;
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
V3DfgPeephole::apply(dfg, ctx);
}