luke
/
verilator
mirror of https://github.com/verilator/verilator.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
verilator/src/V3DfgPeephole.cpp

1920 lines
78 KiB
C++
Raw Normal View History

Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
// -*- mode: C++; c-file-style: "cc-mode" -*-
//*************************************************************************
// DESCRIPTION: Verilator: Peephole optimizations over DfgGraph
//
// Code available from: https://verilator.org
//
//*************************************************************************
//
Add SPDX copyright identifiers, and get 'reuse' clean. No functional change.
2026-01-27 02:24:34 +01:00
// This program is free software; you can redistribute it and/or modify it
// under the terms of either the GNU Lesser General Public License Version 3
// or the Perl Artistic License Version 2.0.
// SPDX-FileCopyrightText: 2003-2026 Wilson Snyder
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
// SPDX-License-Identifier: LGPL-3.0-only OR Artistic-2.0
//
//*************************************************************************
//
// A pattern-matching based optimizer for DfgGraph. This is in some aspects similar to V3Const, but
// more powerful in that it does not care about ordering combinational statement. This is also less
// broadly applicable than V3Const, as it does not apply to procedural statements with sequential
// execution semantics.
//
//*************************************************************************
Fix header order botched by clang-format in recent commit.
2023-10-18 12:37:46 +02:00
#include "V3PchAstNoMT.h" // VL_MT_DISABLED_CODE_UNIT
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
#include "V3Dfg.h"
Avoid creating redundant vertices in V3DfgPeephole (#4944) Add a new data-structure V3DfgCache, which can be used to retrieve existing vertices with some given inputs vertices. Use this in V3DfgPeephole to eliminate the creation of redundant vertices. Overall this is performance neutral, but is in prep for some future work.
2024-03-06 19:01:52 +01:00
#include "V3DfgCache.h"
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
#include "V3DfgPasses.h"
Refactor DFG context objects (#6232) - Move All DFG context objects to V3DfgContext.h - Add separate object for ast2dfg and dfg2ast passes - Factor out commonalities No functional change
2025-07-26 21:37:01 +02:00
#include "V3DfgPeepholePatterns.h"
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
#include "V3Stats.h"
Internals: Remove unused vertices immediately in DfgPeephole
2026-03-23 16:06:07 +01:00
#include <algorithm>
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
#include <cctype>
Optimize concatenations that produce unused bits in DFG (#6971) Concatenations that are only used by Sel expressions that do not consume some bits on the edges can be narrowed to not compute the unused bits. E.g.: `{a[4:0], b[4:0]}[5:4]` -> `{a[0], b[4]}[1:0]` This is a superset or the PUSH_SEL_THROUGH_CONCAT DFG pattern, which is removed.
2026-01-31 18:28:14 +01:00
#include <vector>
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
VL_DEFINE_DEBUG_FUNCTIONS;
Refactor DFG context objects (#6232) - Move All DFG context objects to V3DfgContext.h - Add separate object for ast2dfg and dfg2ast passes - Factor out commonalities No functional change
2025-07-26 21:37:01 +02:00
V3DfgPeepholeContext::V3DfgPeepholeContext(V3DfgContext& ctx, const std::string& label)
: V3DfgSubContext{ctx, label, "Peephole"} {
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
const auto checkEnabled = [this](VDfgPeepholePattern id) {
Refactor DFG context objects (#6232) - Move All DFG context objects to V3DfgContext.h - Add separate object for ast2dfg and dfg2ast passes - Factor out commonalities No functional change
2025-07-26 21:37:01 +02:00
std::string str{id.ascii()};
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
std::transform(str.begin(), str.end(), str.begin(), [](unsigned char c) { //
return c == '_' ? '-' : std::tolower(c);
});
m_enabled[id] = v3Global.opt.fDfgPeepholeEnabled(str);
};
#define OPTIMIZATION_CHECK_ENABLED(id, name) checkEnabled(VDfgPeepholePattern::id);
FOR_EACH_DFG_PEEPHOLE_OPTIMIZATION(OPTIMIZATION_CHECK_ENABLED)
#undef OPTIMIZATION_CHECK_ENABLED
}
V3DfgPeepholeContext::~V3DfgPeepholeContext() {
const auto emitStat = [this](VDfgPeepholePattern id) {
Refactor DFG context objects (#6232) - Move All DFG context objects to V3DfgContext.h - Add separate object for ast2dfg and dfg2ast passes - Factor out commonalities No functional change
2025-07-26 21:37:01 +02:00
std::string str{id.ascii()};
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
std::transform(str.begin(), str.end(), str.begin(), [](unsigned char c) { //
return c == '_' ? ' ' : std::tolower(c);
});
Refactor DFG context objects (#6232) - Move All DFG context objects to V3DfgContext.h - Add separate object for ast2dfg and dfg2ast passes - Factor out commonalities No functional change
2025-07-26 21:37:01 +02:00
addStat(str, m_count[id]);
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
};
#define OPTIMIZATION_EMIT_STATS(id, name) emitStat(VDfgPeepholePattern::id);
FOR_EACH_DFG_PEEPHOLE_OPTIMIZATION(OPTIMIZATION_EMIT_STATS)
#undef OPTIMIZATION_EMIT_STATS
}
DFG: Add more peephole patterns
2022-09-30 17:19:53 +02:00
// clang-format off
template <typename T_Reduction>
struct ReductionToBitwiseImpl {};
template <> struct ReductionToBitwiseImpl<DfgRedAnd> { using type = DfgAnd; };
template <> struct ReductionToBitwiseImpl<DfgRedOr> { using type = DfgOr; };
template <> struct ReductionToBitwiseImpl<DfgRedXor> { using type = DfgXor; };
Fix spelling
2023-11-11 05:25:53 +01:00
template <typename T_Reduction>
using ReductionToBitwise = typename ReductionToBitwiseImpl<T_Reduction>::type;
DFG: Add more peephole patterns
2022-09-30 17:19:53 +02:00
template <typename T_Bitwise>
struct BitwiseToReductionImpl {};
template <> struct BitwiseToReductionImpl<DfgAnd> { using type = DfgRedAnd; };
template <> struct BitwiseToReductionImpl<DfgOr> { using type = DfgRedOr; };
template <> struct BitwiseToReductionImpl<DfgXor> { using type = DfgRedXor; };
Fix spelling
2023-11-11 05:25:53 +01:00
template <typename T_Reduction>
using BitwiseToReduction = typename BitwiseToReductionImpl<T_Reduction>::type;
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
namespace {
template<typename Vertex> void foldOp(V3Number& out, const V3Number& src);
template <> void foldOp<DfgExtend> (V3Number& out, const V3Number& src) { out.opAssign(src); }
template <> void foldOp<DfgExtendS> (V3Number& out, const V3Number& src) { out.opExtendS(src, src.width()); }
template <> void foldOp<DfgLogNot> (V3Number& out, const V3Number& src) { out.opLogNot(src); }
template <> void foldOp<DfgNegate> (V3Number& out, const V3Number& src) { out.opNegate(src); }
template <> void foldOp<DfgNot> (V3Number& out, const V3Number& src) { out.opNot(src); }
template <> void foldOp<DfgRedAnd> (V3Number& out, const V3Number& src) { out.opRedAnd(src); }
template <> void foldOp<DfgRedOr> (V3Number& out, const V3Number& src) { out.opRedOr(src); }
template <> void foldOp<DfgRedXor> (V3Number& out, const V3Number& src) { out.opRedXor(src); }
template<typename Vertex> void foldOp(V3Number& out, const V3Number& lhs, const V3Number& rhs);
template <> void foldOp<DfgAdd> (V3Number& out, const V3Number& lhs, const V3Number& rhs) { out.opAdd(lhs, rhs); }
template <> void foldOp<DfgAnd> (V3Number& out, const V3Number& lhs, const V3Number& rhs) { out.opAnd(lhs, rhs); }
template <> void foldOp<DfgConcat> (V3Number& out, const V3Number& lhs, const V3Number& rhs) { out.opConcat(lhs, rhs); }
template <> void foldOp<DfgDiv> (V3Number& out, const V3Number& lhs, const V3Number& rhs) { out.opDiv(lhs, rhs); }
template <> void foldOp<DfgDivS> (V3Number& out, const V3Number& lhs, const V3Number& rhs) { out.opDivS(lhs, rhs); }
template <> void foldOp<DfgEq> (V3Number& out, const V3Number& lhs, const V3Number& rhs) { out.opEq(lhs, rhs); }
template <> void foldOp<DfgGt> (V3Number& out, const V3Number& lhs, const V3Number& rhs) { out.opGt(lhs, rhs); }
template <> void foldOp<DfgGtS> (V3Number& out, const V3Number& lhs, const V3Number& rhs) { out.opGtS(lhs, rhs); }
template <> void foldOp<DfgGte> (V3Number& out, const V3Number& lhs, const V3Number& rhs) { out.opGte(lhs, rhs); }
template <> void foldOp<DfgGteS> (V3Number& out, const V3Number& lhs, const V3Number& rhs) { out.opGteS(lhs, rhs); }
template <> void foldOp<DfgLogAnd> (V3Number& out, const V3Number& lhs, const V3Number& rhs) { out.opLogAnd(lhs, rhs); }
template <> void foldOp<DfgLogEq> (V3Number& out, const V3Number& lhs, const V3Number& rhs) { out.opLogEq(lhs, rhs); }
template <> void foldOp<DfgLogIf> (V3Number& out, const V3Number& lhs, const V3Number& rhs) { out.opLogIf(lhs, rhs); }
template <> void foldOp<DfgLogOr> (V3Number& out, const V3Number& lhs, const V3Number& rhs) { out.opLogOr(lhs, rhs); }
template <> void foldOp<DfgLt> (V3Number& out, const V3Number& lhs, const V3Number& rhs) { out.opLt(lhs, rhs); }
template <> void foldOp<DfgLtS> (V3Number& out, const V3Number& lhs, const V3Number& rhs) { out.opLtS(lhs, rhs); }
template <> void foldOp<DfgLte> (V3Number& out, const V3Number& lhs, const V3Number& rhs) { out.opLte(lhs, rhs); }
Fix folding of LteS in DfgPeephole (#6004) Fixes #6000
2025-05-14 14:33:20 +02:00
template <> void foldOp<DfgLteS> (V3Number& out, const V3Number& lhs, const V3Number& rhs) { out.opLteS(lhs, rhs); }
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
template <> void foldOp<DfgModDiv> (V3Number& out, const V3Number& lhs, const V3Number& rhs) { out.opModDiv(lhs, rhs); }
template <> void foldOp<DfgModDivS> (V3Number& out, const V3Number& lhs, const V3Number& rhs) { out.opModDivS(lhs, rhs); }
template <> void foldOp<DfgMul> (V3Number& out, const V3Number& lhs, const V3Number& rhs) { out.opMul(lhs, rhs); }
template <> void foldOp<DfgMulS> (V3Number& out, const V3Number& lhs, const V3Number& rhs) { out.opMulS(lhs, rhs); }
template <> void foldOp<DfgNeq> (V3Number& out, const V3Number& lhs, const V3Number& rhs) { out.opNeq(lhs, rhs); }
template <> void foldOp<DfgOr> (V3Number& out, const V3Number& lhs, const V3Number& rhs) { out.opOr(lhs, rhs); }
template <> void foldOp<DfgPow> (V3Number& out, const V3Number& lhs, const V3Number& rhs) { out.opPow(lhs, rhs); }
template <> void foldOp<DfgPowSS> (V3Number& out, const V3Number& lhs, const V3Number& rhs) { out.opPowSS(lhs, rhs); }
template <> void foldOp<DfgPowSU> (V3Number& out, const V3Number& lhs, const V3Number& rhs) { out.opPowSU(lhs, rhs); }
template <> void foldOp<DfgPowUS> (V3Number& out, const V3Number& lhs, const V3Number& rhs) { out.opPowUS(lhs, rhs); }
template <> void foldOp<DfgReplicate> (V3Number& out, const V3Number& lhs, const V3Number& rhs) { out.opRepl(lhs, rhs); }
template <> void foldOp<DfgShiftL> (V3Number& out, const V3Number& lhs, const V3Number& rhs) { out.opShiftL(lhs, rhs); }
template <> void foldOp<DfgShiftR> (V3Number& out, const V3Number& lhs, const V3Number& rhs) { out.opShiftR(lhs, rhs); }
template <> void foldOp<DfgShiftRS> (V3Number& out, const V3Number& lhs, const V3Number& rhs) { out.opShiftRS(lhs, rhs, lhs.width()); }
template <> void foldOp<DfgSub> (V3Number& out, const V3Number& lhs, const V3Number& rhs) { out.opSub(lhs, rhs); }
template <> void foldOp<DfgXor> (V3Number& out, const V3Number& lhs, const V3Number& rhs) { out.opXor(lhs, rhs); }
}
DFG: Add more peephole patterns
2022-09-30 17:19:53 +02:00
// clang-format on
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
class V3DfgPeephole final : public DfgVisitor {
Internals: Use a custom worklist in DfgPeephole
2026-03-23 15:41:36 +01:00
// TYPES
struct VertexInfo final {
Internals: Remove unused vertices immediately in DfgPeephole
2026-03-23 16:06:07 +01:00
size_t m_workListIndex = 0; // Position of this vertx m_wlist (0 means not in list)
size_t m_generation = 0; // Generation number of this vertex
Internals: Use a custom worklist in DfgPeephole
2026-03-23 15:41:36 +01:00
};
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
// STATE
DfgGraph& m_dfg; // The DfgGraph being visited
V3DfgPeepholeContext& m_ctx; // The config structure
Improve Dfg type system (#6390) Added a mini type system for Dfg using DfgDataType to replace Dfg's use of AstNodeDType. This is much more restricted and represents only the types Dfg can handle in a canonical form. This will be needed when adding more support for unpacked arrays and maybe unpacked structs one day. Also added an internal type checker for DfgGraphs which encodes all the assumptions the code makes about type relationships in the graph. Run this in a few places with --debug-check. Fix resulting fallout.
2025-09-07 21:38:50 +02:00
const DfgDataType& m_bitDType = DfgDataType::packed(1); // Common, so grab it up front
Internals: Use a custom worklist in DfgPeephole
2026-03-23 15:41:36 +01:00
std::vector<DfgVertex*> m_wlist; // Using a manual work list as need special operations
DfgUserMap<VertexInfo> m_vInfo = m_dfg.makeUserMap<VertexInfo>(); // Map to VertexInfo
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
V3DfgCache m_cache{m_dfg}; // Vertex cache to avoid creating redundant vertices
DfgVertex* m_vtxp = nullptr; // Currently considered vertex
Internals: Remove unused vertices immediately in DfgPeephole
2026-03-23 16:06:07 +01:00
size_t m_currentGeneration = 0; // Current generation number
Internals: Add DfgWorklist commonly used in algorithms (#6361)
2025-09-03 14:55:33 +02:00
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
// STATIC STATE
static V3DebugBisect s_debugBisect; // Debug aid
Internals: Add utility to perform bisection search for debugging (#7294)
2026-03-21 11:13:27 +01:00
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
#define APPLYING(id) if (checkApplying(VDfgPeepholePattern::id))
// METHODS
bool checkApplying(VDfgPeepholePattern id) {
Internals: Add utility to perform bisection search for debugging (#7294)
2026-03-21 11:13:27 +01:00
if (VL_UNLIKELY(!m_ctx.m_enabled[id] || s_debugBisect.isStopped())) return false;
Internals: UINFO now includes newline itself. No functional change.
2025-05-23 02:29:32 +02:00
UINFO(9, "Applying DFG pattern " << id.ascii());
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
++m_ctx.m_count[id];
return true;
}
Internals: Remove unused vertices immediately in DfgPeephole
2026-03-23 16:06:07 +01:00
void incrementGeneration() {
++m_currentGeneration;
// TODO: could sweep on overflow
}
Internals: Use a custom worklist in DfgPeephole
2026-03-23 15:41:36 +01:00
void addToWorkList(DfgVertex* vtxp) {
VertexInfo& vInfo = m_vInfo[*vtxp];
// If already on work list, ignore
if (vInfo.m_workListIndex) return;
// Add to work list
vInfo.m_workListIndex = m_wlist.size();
m_wlist.push_back(vtxp);
}
DfgPeephole: Use a work list driven algorithm for speed Replace the 'run to fixed point' algorithm with a work list driven approach. Instead of marking the graph as changed, we explicitly add vertices to the work list, to be visited, when a vertex is changed. This improves both memory locality (as the work list is processed in last in first out order), and removed unnecessary visitations when only a few nodes changes.
2022-11-04 16:50:20 +01:00
Internals: Remove unused vertices immediately in DfgPeephole
2026-03-23 16:06:07 +01:00
void removeFromWorkList(DfgVertex* vtxp) {
VertexInfo& vInfo = m_vInfo[*vtxp];
// m_wlist[0] is always nullptr, fine to assign same here
m_wlist[vInfo.m_workListIndex] = nullptr;
vInfo.m_workListIndex = 0;
}
DfgPeephole: Use a work list driven algorithm for speed Replace the 'run to fixed point' algorithm with a work list driven approach. Instead of marking the graph as changed, we explicitly add vertices to the work list, to be visited, when a vertex is changed. This improves both memory locality (as the work list is processed in last in first out order), and removed unnecessary visitations when only a few nodes changes.
2022-11-04 16:50:20 +01:00
void addSourcesToWorkList(DfgVertex* vtxp) {
Internals: Improve DFG implementation details (#6355) Large scale refactoring to simplify some of the more obtuse internals of DFG. Remove multiple redundant internal APIs, simplify representation of variables, fix potential unsoundness in circular decomposition. No functional change intended.
2025-09-02 17:50:40 +02:00
vtxp->foreachSource([&](DfgVertex& src) {
addToWorkList(&src);
return false;
});
DfgPeephole: Use a work list driven algorithm for speed Replace the 'run to fixed point' algorithm with a work list driven approach. Instead of marking the graph as changed, we explicitly add vertices to the work list, to be visited, when a vertex is changed. This improves both memory locality (as the work list is processed in last in first out order), and removed unnecessary visitations when only a few nodes changes.
2022-11-04 16:50:20 +01:00
}
void addSinksToWorkList(DfgVertex* vtxp) {
Internals: Improve DFG implementation details (#6355) Large scale refactoring to simplify some of the more obtuse internals of DFG. Remove multiple redundant internal APIs, simplify representation of variables, fix potential unsoundness in circular decomposition. No functional change intended.
2025-09-02 17:50:40 +02:00
vtxp->foreachSink([&](DfgVertex& src) {
addToWorkList(&src);
return false;
});
DfgPeephole: Use a work list driven algorithm for speed Replace the 'run to fixed point' algorithm with a work list driven approach. Instead of marking the graph as changed, we explicitly add vertices to the work list, to be visited, when a vertex is changed. This improves both memory locality (as the work list is processed in last in first out order), and removed unnecessary visitations when only a few nodes changes.
2022-11-04 16:50:20 +01:00
}
void deleteVertex(DfgVertex* vtxp) {
Internals: Use a custom worklist in DfgPeephole
2026-03-23 15:41:36 +01:00
UASSERT_OBJ(!m_vInfo[vtxp].m_workListIndex, vtxp, "Deleted Vertex is in work list");
Optimize duplicate vertices in DfgPeephole (#7305)
2026-03-21 23:45:02 +01:00
UASSERT_OBJ(!vtxp->hasSinks(), vtxp, "Should not delete used vertex");
Internals: Remove unused vertices immediately in DfgPeephole
2026-03-23 16:06:07 +01:00
// Invalidate cache entry
m_cache.invalidate(vtxp);
// Gather source vertices - they might be duplicates, make unique using generation number
incrementGeneration();
std::vector<DfgVertex*> srcps;
srcps.reserve(vtxp->nInputs());
vtxp->foreachSource([&](DfgVertex& src) {
// If it's a variable, add to work list to see if it became redundant
if (src.is<DfgVertexVar>()) {
addToWorkList(&src);
return false;
}
// Gather unique sources
VertexInfo& vInfo = m_vInfo[src];
if (vInfo.m_generation != m_currentGeneration) srcps.push_back(&src);
vInfo.m_generation = m_currentGeneration;
return false;
});
Optimize: Remove redundant variables during Dfg Peephole pass (#7076) Many rules in the Dfg Peephole pass check if a node has more than one sinks. Redundant variables that will ultimately be removed can prevent these from matching. Remove such variables during the Peeophole pass itself to enable more matches.
2026-02-14 13:33:20 +01:00
// This pass only removes variables that are either not driven in this graph,
// or are not observable outside the graph. If there is also no external write
// to the variable and no references in other graph then delete the Ast var too.
const DfgVertexVar* const varp = vtxp->cast<DfgVertexVar>();
Internals: Remove unused vertices immediately in DfgPeephole
2026-03-23 16:06:07 +01:00
if (varp && !varp->isVolatile() && !varp->hasDfgRefs()) {
AstNode* const nodep = varp->nodep();
VL_DO_DANGLING(vtxp->unlinkDelete(m_dfg), vtxp);
VL_DO_DANGLING(nodep->unlinkFrBack()->deleteTree(), nodep);
} else {
VL_DO_DANGLING(vtxp->unlinkDelete(m_dfg), vtxp);
}
// Partition sources into used/unused after removing the vertex
const auto mid
= std::stable_partition(srcps.begin(), srcps.end(), [](DfgVertex* srcp) { //
return srcp->hasSinks();
});
// Add used sources to the work list
for (auto it = srcps.begin(); it != mid; ++it) addToWorkList(*it);
// Recursively delete unused sources
for (auto it = mid; it != srcps.end(); ++it) {
// Remove from work list
removeFromWorkList(*it);
// Delete vertex
deleteVertex(*it);
}
DfgPeephole: Use a work list driven algorithm for speed Replace the 'run to fixed point' algorithm with a work list driven approach. Instead of marking the graph as changed, we explicitly add vertices to the work list, to be visited, when a vertex is changed. This improves both memory locality (as the work list is processed in last in first out order), and removed unnecessary visitations when only a few nodes changes.
2022-11-04 16:50:20 +01:00
}
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
// Replace 'm_vtxp' with the given vertex
void replace(DfgVertex* resp) {
// Should not be in the work list
Internals: Use a custom worklist in DfgPeephole
2026-03-23 15:41:36 +01:00
UASSERT_OBJ(!m_vInfo[m_vtxp].m_workListIndex, m_vtxp, "Replaced Vertex is in work list");
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
// Debug bisect check
Internals: Add utility to perform bisection search for debugging (#7294)
2026-03-21 11:13:27 +01:00
const auto debugCallback = [&]() -> void {
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
UINFO(0, "Problematic DfgPeephole replacement: " << m_vtxp << " -> " << resp);
m_dfg.sourceCone({m_vtxp, resp});
const auto cone = m_dfg.sourceCone({m_vtxp, resp});
Internals: Add utility to perform bisection search for debugging (#7294)
2026-03-21 11:13:27 +01:00
m_dfg.dumpDotFilePrefixed("peephole-broken", [&](const DfgVertex& v) { //
return cone->count(&v);
});
};
if (VL_UNLIKELY(s_debugBisect.stop(debugCallback))) return;
Internals: Remove unused vertices immediately in DfgPeephole
2026-03-23 16:06:07 +01:00
// Remove sinks from cache, their inputs are about to change
m_vtxp->foreachSink([&](DfgVertex& dst) {
m_cache.invalidate(&dst);
Internals: Improve DFG implementation details (#6355) Large scale refactoring to simplify some of the more obtuse internals of DFG. Remove multiple redundant internal APIs, simplify representation of variables, fix potential unsoundness in circular decomposition. No functional change intended.
2025-09-02 17:50:40 +02:00
return false;
});
Optimize duplicate vertices in DfgPeephole (#7305)
2026-03-21 23:45:02 +01:00
// Replace vertex with the replacement
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
m_vtxp->replaceWith(resp);
// Re-cache all sinks of the replacement
Internals: Remove unused vertices immediately in DfgPeephole
2026-03-23 16:06:07 +01:00
resp->foreachSink([&](DfgVertex& dst) {
m_cache.cache(&dst);
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
return false;
});
Internals: Remove unused vertices immediately in DfgPeephole
2026-03-23 16:06:07 +01:00
// Original vertex is now unused, so delete it
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
deleteVertex(m_vtxp);
Internals: Remove unused vertices immediately in DfgPeephole
2026-03-23 16:06:07 +01:00
// Add all sources to the work list
addSourcesToWorkList(resp);
// Add replacement to the work list
addToWorkList(resp);
// Add sinks of replaced vertex to the work list
addSinksToWorkList(resp);
DfgPeephole: Use a work list driven algorithm for speed Replace the 'run to fixed point' algorithm with a work list driven approach. Instead of marking the graph as changed, we explicitly add vertices to the work list, to be visited, when a vertex is changed. This improves both memory locality (as the work list is processed in last in first out order), and removed unnecessary visitations when only a few nodes changes.
2022-11-04 16:50:20 +01:00
}
Make DfgConst hold V3Number directly Remove intermediary AstConst. No functional change intended.
2022-10-07 16:44:14 +02:00
// Create a 32-bit DfgConst vertex
DfgConst* makeI32(FileLine* flp, uint32_t val) { return new DfgConst{m_dfg, flp, 32, val}; }
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
Make DfgConst hold V3Number directly Remove intermediary AstConst. No functional change intended.
2022-10-07 16:44:14 +02:00
// Create a DfgConst vertex with the given width and value zero
Internals: Improve DFG implementation details (#6355) Large scale refactoring to simplify some of the more obtuse internals of DFG. Remove multiple redundant internal APIs, simplify representation of variables, fix potential unsoundness in circular decomposition. No functional change intended.
2025-09-02 17:50:40 +02:00
DfgConst* makeZero(FileLine* flp, uint32_t width) {
return new DfgConst{m_dfg, flp, width, 0};
}
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
DfgPeephole: Use a work list driven algorithm for speed Replace the 'run to fixed point' algorithm with a work list driven approach. Instead of marking the graph as changed, we explicitly add vertices to the work list, to be visited, when a vertex is changed. This improves both memory locality (as the work list is processed in last in first out order), and removed unnecessary visitations when only a few nodes changes.
2022-11-04 16:50:20 +01:00
// Create a new vertex of the given type
Avoid creating redundant vertices in V3DfgPeephole (#4944) Add a new data-structure V3DfgCache, which can be used to retrieve existing vertices with some given inputs vertices. Use this in V3DfgPeephole to eliminate the creation of redundant vertices. Overall this is performance neutral, but is in prep for some future work.
2024-03-06 19:01:52 +01:00
template <typename Vertex, typename... Operands>
Improve Dfg type system (#6390) Added a mini type system for Dfg using DfgDataType to replace Dfg's use of AstNodeDType. This is much more restricted and represents only the types Dfg can handle in a canonical form. This will be needed when adding more support for unpacked arrays and maybe unpacked structs one day. Also added an internal type checker for DfgGraphs which encodes all the assumptions the code makes about type relationships in the graph. Run this in a few places with --debug-check. Fix resulting fallout.
2025-09-07 21:38:50 +02:00
Vertex* make(FileLine* flp, const DfgDataType& dtype, Operands... operands) {
Avoid creating redundant vertices in V3DfgPeephole (#4944) Add a new data-structure V3DfgCache, which can be used to retrieve existing vertices with some given inputs vertices. Use this in V3DfgPeephole to eliminate the creation of redundant vertices. Overall this is performance neutral, but is in prep for some future work.
2024-03-06 19:01:52 +01:00
// Find or create an equivalent vertex
Improve Dfg type system (#6390) Added a mini type system for Dfg using DfgDataType to replace Dfg's use of AstNodeDType. This is much more restricted and represents only the types Dfg can handle in a canonical form. This will be needed when adding more support for unpacked arrays and maybe unpacked structs one day. Also added an internal type checker for DfgGraphs which encodes all the assumptions the code makes about type relationships in the graph. Run this in a few places with --debug-check. Fix resulting fallout.
2025-09-07 21:38:50 +02:00
Vertex* const vtxp = m_cache.getOrCreate<Vertex, Operands...>(flp, dtype, operands...);
Internals: Type check vertices made in DfgPeephole
2026-03-21 11:15:18 +01:00
// Sanity check
UASSERT_OBJ(vtxp->dtype() == dtype, vtxp, "Vertex dtype mismatch");
if (VL_UNLIKELY(v3Global.opt.debugCheck())) vtxp->typeCheck(m_dfg);
Internals: Manage DfgVertex user data via a typed map (#6356) Add DfgUserMap as a handle around the one pointer worth of algorithm specific 'user' storage in each DfgVertex. This reduces verbosity, improves type safety and correctness. Also enables us to remove one pointer from DfgVertex to reduce memory use. No functional change.
2025-09-02 23:21:24 +02:00
// Add to work list
addToWorkList(vtxp);
DfgPeephole: Use a work list driven algorithm for speed Replace the 'run to fixed point' algorithm with a work list driven approach. Instead of marking the graph as changed, we explicitly add vertices to the work list, to be visited, when a vertex is changed. This improves both memory locality (as the work list is processed in last in first out order), and removed unnecessary visitations when only a few nodes changes.
2022-11-04 16:50:20 +01:00
// Return new node
return vtxp;
}
Avoid creating redundant vertices in V3DfgPeephole (#4944) Add a new data-structure V3DfgCache, which can be used to retrieve existing vertices with some given inputs vertices. Use this in V3DfgPeephole to eliminate the creation of redundant vertices. Overall this is performance neutral, but is in prep for some future work.
2024-03-06 19:01:52 +01:00
// Same as above, but 'flp' and 'dtypep' are taken from the given example vertex
template <typename Vertex, typename... Operands>
Internals: Cleanup cppcheck warnings (#6317) `make cppcheck-4` is now clean.
2025-08-20 19:21:24 +02:00
Vertex* make(const DfgVertex* examplep, Operands... operands) {
Improve Dfg type system (#6390) Added a mini type system for Dfg using DfgDataType to replace Dfg's use of AstNodeDType. This is much more restricted and represents only the types Dfg can handle in a canonical form. This will be needed when adding more support for unpacked arrays and maybe unpacked structs one day. Also added an internal type checker for DfgGraphs which encodes all the assumptions the code makes about type relationships in the graph. Run this in a few places with --debug-check. Fix resulting fallout.
2025-09-07 21:38:50 +02:00
return make<Vertex>(examplep->fileline(), examplep->dtype(), operands...);
Avoid creating redundant vertices in V3DfgPeephole (#4944) Add a new data-structure V3DfgCache, which can be used to retrieve existing vertices with some given inputs vertices. Use this in V3DfgPeephole to eliminate the creation of redundant vertices. Overall this is performance neutral, but is in prep for some future work.
2024-03-06 19:01:52 +01:00
}
Add DfgPeephole patterns (#6149)
2025-07-07 17:25:29 +02:00
// Check two vertex are the same, or the same constant value
static bool isSame(const DfgVertex* ap, const DfgVertex* bp) {
if (ap == bp) return true;
const DfgConst* const aConstp = ap->cast<DfgConst>();
if (!aConstp) return false;
const DfgConst* const bConstp = bp->cast<DfgConst>();
if (!bConstp) return false;
return aConstp->num().isCaseEq(bConstp->num());
}
Internals: Improve DFG implementation details (#6355) Large scale refactoring to simplify some of the more obtuse internals of DFG. Remove multiple redundant internal APIs, simplify representation of variables, fix potential unsoundness in circular decomposition. No functional change intended.
2025-09-02 17:50:40 +02:00
static bool isZero(const DfgVertex* vtxp) {
if (const DfgConst* const constp = vtxp->cast<DfgConst>()) return constp->isZero();
return false;
}
static bool isOnes(const DfgVertex* vtxp) {
if (const DfgConst* const constp = vtxp->cast<DfgConst>()) return constp->isOnes();
return false;
}
Avoid creating redundant vertices in V3DfgPeephole (#4944) Add a new data-structure V3DfgCache, which can be used to retrieve existing vertices with some given inputs vertices. Use this in V3DfgPeephole to eliminate the creation of redundant vertices. Overall this is performance neutral, but is in prep for some future work.
2024-03-06 19:01:52 +01:00
// Note: If any of the following transformers return true, then the vertex was replaced and the
// caller must not do any further changes, so the caller must check the return value, otherwise
// there will be hard to debug issues.
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
// Constant fold unary vertex, return true if folded
template <typename Vertex>
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
VL_ATTR_WARN_UNUSED_RESULT bool foldUnary(Vertex* const vtxp) {
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
static_assert(std::is_base_of<DfgVertexUnary, Vertex>::value, "Must invoke on unary");
static_assert(std::is_final<Vertex>::value, "Must invoke on final class");
if (DfgConst* const srcp = vtxp->srcp()->template cast<DfgConst>()) {
APPLYING(FOLD_UNARY) {
DfgConst* const resultp = makeZero(vtxp->fileline(), vtxp->width());
foldOp<Vertex>(resultp->num(), srcp->num());
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(resultp);
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
return true;
}
}
return false;
}
// Constant fold binary vertex, return true if folded
template <typename Vertex>
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
VL_ATTR_WARN_UNUSED_RESULT bool foldBinary(Vertex* const vtxp) {
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
static_assert(std::is_base_of<DfgVertexBinary, Vertex>::value, "Must invoke on binary");
static_assert(std::is_final<Vertex>::value, "Must invoke on final class");
Internals: Improve DFG implementation details (#6355) Large scale refactoring to simplify some of the more obtuse internals of DFG. Remove multiple redundant internal APIs, simplify representation of variables, fix potential unsoundness in circular decomposition. No functional change intended.
2025-09-02 17:50:40 +02:00
if (DfgConst* const lhsp = vtxp->inputp(0)->template cast<DfgConst>()) {
if (DfgConst* const rhsp = vtxp->inputp(1)->template cast<DfgConst>()) {
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
APPLYING(FOLD_BINARY) {
DfgConst* const resultp = makeZero(vtxp->fileline(), vtxp->width());
foldOp<Vertex>(resultp->num(), lhsp->num(), rhsp->num());
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(resultp);
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
return true;
}
}
}
return false;
}
DfgPeephole: Rework folding of associative operations Allow constant folding through adjacent nodes of all associative operations, for example '((a & 2) & 3)' or '(3 & (2 & a))' can now be folded into '(a & 2)' and '(2 & a)' respectively. Also improve speed of making associative expression trees right leaning by using rotation of the existing vertices whenever instead of allocation of new nodes.
2022-10-05 11:36:28 +02:00
// Transformations that apply to all associative binary vertices.
// Returns true if vtxp was replaced.
template <typename Vertex>
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
VL_ATTR_WARN_UNUSED_RESULT bool associativeBinary(Vertex* const vtxp) {
DfgPeephole: Rework folding of associative operations Allow constant folding through adjacent nodes of all associative operations, for example '((a & 2) & 3)' or '(3 & (2 & a))' can now be folded into '(a & 2)' and '(2 & a)' respectively. Also improve speed of making associative expression trees right leaning by using rotation of the existing vertices whenever instead of allocation of new nodes.
2022-10-05 11:36:28 +02:00
static_assert(std::is_base_of<DfgVertexBinary, Vertex>::value, "Must invoke on binary");
static_assert(std::is_final<Vertex>::value, "Must invoke on final class");
DfgVertex* const lhsp = vtxp->lhsp();
DfgVertex* const rhsp = vtxp->rhsp();
FileLine* const flp = vtxp->fileline();
DfgConst* const lConstp = lhsp->cast<DfgConst>();
DfgConst* const rConstp = rhsp->cast<DfgConst>();
if (lConstp && rConstp) {
APPLYING(FOLD_ASSOC_BINARY) {
DfgConst* const resultp = makeZero(flp, vtxp->width());
foldOp<Vertex>(resultp->num(), lConstp->num(), rConstp->num());
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(resultp);
DfgPeephole: Rework folding of associative operations Allow constant folding through adjacent nodes of all associative operations, for example '((a & 2) & 3)' or '(3 & (2 & a))' can now be folded into '(a & 2)' and '(2 & a)' respectively. Also improve speed of making associative expression trees right leaning by using rotation of the existing vertices whenever instead of allocation of new nodes.
2022-10-05 11:36:28 +02:00
return true;
}
}
if (lConstp) {
if (Vertex* const rVtxp = rhsp->cast<Vertex>()) {
if (DfgConst* const rlConstp = rVtxp->lhsp()->template cast<DfgConst>()) {
APPLYING(FOLD_ASSOC_BINARY_LHS_OF_RHS) {
// Fold constants
const uint32_t width = std::is_same<DfgConcat, Vertex>::value
? lConstp->width() + rlConstp->width()
: vtxp->width();
DfgConst* const constp = makeZero(flp, width);
foldOp<Vertex>(constp->num(), lConstp->num(), rlConstp->num());
// Replace vertex
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(make<Vertex>(vtxp, constp, rVtxp->rhsp()));
Avoid creating redundant vertices in V3DfgPeephole (#4944) Add a new data-structure V3DfgCache, which can be used to retrieve existing vertices with some given inputs vertices. Use this in V3DfgPeephole to eliminate the creation of redundant vertices. Overall this is performance neutral, but is in prep for some future work.
2024-03-06 19:01:52 +01:00
return true;
DfgPeephole: Rework folding of associative operations Allow constant folding through adjacent nodes of all associative operations, for example '((a & 2) & 3)' or '(3 & (2 & a))' can now be folded into '(a & 2)' and '(2 & a)' respectively. Also improve speed of making associative expression trees right leaning by using rotation of the existing vertices whenever instead of allocation of new nodes.
2022-10-05 11:36:28 +02:00
}
}
}
}
if (rConstp) {
if (Vertex* const lVtxp = lhsp->cast<Vertex>()) {
if (DfgConst* const lrConstp = lVtxp->rhsp()->template cast<DfgConst>()) {
APPLYING(FOLD_ASSOC_BINARY_RHS_OF_LHS) {
// Fold constants
const uint32_t width = std::is_same<DfgConcat, Vertex>::value
? lrConstp->width() + rConstp->width()
: vtxp->width();
DfgConst* const constp = makeZero(flp, width);
foldOp<Vertex>(constp->num(), lrConstp->num(), rConstp->num());
// Replace vertex
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(make<Vertex>(vtxp, lVtxp->lhsp(), constp));
Avoid creating redundant vertices in V3DfgPeephole (#4944) Add a new data-structure V3DfgCache, which can be used to retrieve existing vertices with some given inputs vertices. Use this in V3DfgPeephole to eliminate the creation of redundant vertices. Overall this is performance neutral, but is in prep for some future work.
2024-03-06 19:01:52 +01:00
return true;
DfgPeephole: Rework folding of associative operations Allow constant folding through adjacent nodes of all associative operations, for example '((a & 2) & 3)' or '(3 & (2 & a))' can now be folded into '(a & 2)' and '(2 & a)' respectively. Also improve speed of making associative expression trees right leaning by using rotation of the existing vertices whenever instead of allocation of new nodes.
2022-10-05 11:36:28 +02:00
}
}
}
}
// Make associative trees right leaning to reduce pattern variations, and for better CSE
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
if (Vertex* const alhsp = vtxp->lhsp()->template cast<Vertex>()) {
if (!alhsp->hasMultipleSinks()) {
APPLYING(RIGHT_LEANING_ASSOC) {
// Rotate the expression tree rooted at 'vtxp' to the right, producing a
// right-leaning tree
DfgVertex* const ap = alhsp->lhsp();
DfgVertex* const bp = alhsp->rhsp();
DfgVertex* const cp = vtxp->rhsp();
// Concatenation dtypes need adjusting, other assoc vertices preserve types
const DfgDataType& childDType
= std::is_same<Vertex, DfgConcat>::value
? DfgDataType::packed(bp->width() + cp->width())
: vtxp->dtype();
Vertex* const bcp = make<Vertex>(vtxp->fileline(), childDType, bp, cp);
replace(make<Vertex>(alhsp->fileline(), vtxp->dtype(), ap, bcp));
return true;
}
DfgPeephole: Rework folding of associative operations Allow constant folding through adjacent nodes of all associative operations, for example '((a & 2) & 3)' or '(3 & (2 & a))' can now be folded into '(a & 2)' and '(2 & a)' respectively. Also improve speed of making associative expression trees right leaning by using rotation of the existing vertices whenever instead of allocation of new nodes.
2022-10-05 11:36:28 +02:00
}
}
Optimize reuse of existing associative terms in DfgPeephole Enable V3DfgCache to look up vertices without creating one. Reuse existing terms in associative expression trees if they already exist somewhere in the graph.
2026-03-17 09:20:00 +01:00
// if (a OP (b OP c)), check if (a OP b) exists and if so replace with (a OP b) OP c
if (Vertex* rVtxp = vtxp->rhsp()->template cast<Vertex>()) {
const DfgDataType& dtype
= std::is_same<Vertex, DfgConcat>::value
? DfgDataType::packed(lhsp->width() + rVtxp->lhsp()->width())
: vtxp->dtype();
if (Vertex* const cVtxp = m_cache.get<Vertex>(dtype, lhsp, rVtxp->lhsp())) {
if (cVtxp->hasSinks() && cVtxp != rhsp) {
APPLYING(REUSE_ASSOC_BINARY) {
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(make<Vertex>(vtxp, cVtxp, rVtxp->rhsp()));
Optimize reuse of existing associative terms in DfgPeephole Enable V3DfgCache to look up vertices without creating one. Reuse existing terms in associative expression trees if they already exist somewhere in the graph.
2026-03-17 09:20:00 +01:00
return true;
}
}
}
}
DfgPeephole: Rework folding of associative operations Allow constant folding through adjacent nodes of all associative operations, for example '((a & 2) & 3)' or '(3 & (2 & a))' can now be folded into '(a & 2)' and '(2 & a)' respectively. Also improve speed of making associative expression trees right leaning by using rotation of the existing vertices whenever instead of allocation of new nodes.
2022-10-05 11:36:28 +02:00
Optimize reuse of existing associative terms in DfgPeephole Enable V3DfgCache to look up vertices without creating one. Reuse existing terms in associative expression trees if they already exist somewhere in the graph.
2026-03-17 09:20:00 +01:00
return false;
DfgPeephole: Rework folding of associative operations Allow constant folding through adjacent nodes of all associative operations, for example '((a & 2) & 3)' or '(3 & (2 & a))' can now be folded into '(a & 2)' and '(2 & a)' respectively. Also improve speed of making associative expression trees right leaning by using rotation of the existing vertices whenever instead of allocation of new nodes.
2022-10-05 11:36:28 +02:00
}
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
// Transformations that apply to all commutative binary vertices
Avoid creating redundant vertices in V3DfgPeephole (#4944) Add a new data-structure V3DfgCache, which can be used to retrieve existing vertices with some given inputs vertices. Use this in V3DfgPeephole to eliminate the creation of redundant vertices. Overall this is performance neutral, but is in prep for some future work.
2024-03-06 19:01:52 +01:00
template <typename Vertex>
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
VL_ATTR_WARN_UNUSED_RESULT bool commutativeBinary(Vertex* const vtxp) {
Avoid creating redundant vertices in V3DfgPeephole (#4944) Add a new data-structure V3DfgCache, which can be used to retrieve existing vertices with some given inputs vertices. Use this in V3DfgPeephole to eliminate the creation of redundant vertices. Overall this is performance neutral, but is in prep for some future work.
2024-03-06 19:01:52 +01:00
static_assert(std::is_base_of<DfgVertexBinary, Vertex>::value, "Must invoke on binary");
static_assert(std::is_final<Vertex>::value, "Must invoke on final class");
DfgVertex* const lhsp = vtxp->lhsp();
DfgVertex* const rhsp = vtxp->rhsp();
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
// Ensure Const is on left-hand side to simplify other patterns
Avoid creating redundant vertices in V3DfgPeephole (#4944) Add a new data-structure V3DfgCache, which can be used to retrieve existing vertices with some given inputs vertices. Use this in V3DfgPeephole to eliminate the creation of redundant vertices. Overall this is performance neutral, but is in prep for some future work.
2024-03-06 19:01:52 +01:00
if (lhsp->is<DfgConst>()) return false;
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
if (rhsp->is<DfgConst>()) {
APPLYING(SWAP_CONST_IN_COMMUTATIVE_BINARY) {
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(make<Vertex>(vtxp, rhsp, lhsp));
Avoid creating redundant vertices in V3DfgPeephole (#4944) Add a new data-structure V3DfgCache, which can be used to retrieve existing vertices with some given inputs vertices. Use this in V3DfgPeephole to eliminate the creation of redundant vertices. Overall this is performance neutral, but is in prep for some future work.
2024-03-06 19:01:52 +01:00
return true;
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
}
}
// Ensure Not is on the left-hand side to simplify other patterns
Avoid creating redundant vertices in V3DfgPeephole (#4944) Add a new data-structure V3DfgCache, which can be used to retrieve existing vertices with some given inputs vertices. Use this in V3DfgPeephole to eliminate the creation of redundant vertices. Overall this is performance neutral, but is in prep for some future work.
2024-03-06 19:01:52 +01:00
if (lhsp->is<DfgNot>()) return false;
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
if (rhsp->is<DfgNot>()) {
APPLYING(SWAP_NOT_IN_COMMUTATIVE_BINARY) {
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(make<Vertex>(vtxp, rhsp, lhsp));
Avoid creating redundant vertices in V3DfgPeephole (#4944) Add a new data-structure V3DfgCache, which can be used to retrieve existing vertices with some given inputs vertices. Use this in V3DfgPeephole to eliminate the creation of redundant vertices. Overall this is performance neutral, but is in prep for some future work.
2024-03-06 19:01:52 +01:00
return true;
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
}
}
Avoid creating redundant vertices in V3DfgPeephole (#4944) Add a new data-structure V3DfgCache, which can be used to retrieve existing vertices with some given inputs vertices. Use this in V3DfgPeephole to eliminate the creation of redundant vertices. Overall this is performance neutral, but is in prep for some future work.
2024-03-06 19:01:52 +01:00
// If both sides are variable references, order the side in some defined way. This
// allows CSE to later merge 'a op b' with 'b op a'.
DFG: Make implementation more similar to AST Use the same style, and reuse the bulk of astgen to generate DfgVertex related code. In particular allow for easier definition of custom DfgVertex sub-types that do not directly correspond to an AstNode sub-type. Also introduces specific names for the fixed arity vertices. No functional change intended.
2022-10-04 12:03:41 +02:00
if (lhsp->is<DfgVertexVar>() && rhsp->is<DfgVertexVar>()) {
Internals: Cleanup cppcheck warnings (#6317) `make cppcheck-4` is now clean.
2025-08-20 19:21:24 +02:00
const AstNode* const lVarp = lhsp->as<DfgVertexVar>()->nodep();
const AstNode* const rVarp = rhsp->as<DfgVertexVar>()->nodep();
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
if (lVarp->name() > rVarp->name()) {
APPLYING(SWAP_VAR_IN_COMMUTATIVE_BINARY) {
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(make<Vertex>(vtxp, rhsp, lhsp));
Avoid creating redundant vertices in V3DfgPeephole (#4944) Add a new data-structure V3DfgCache, which can be used to retrieve existing vertices with some given inputs vertices. Use this in V3DfgPeephole to eliminate the creation of redundant vertices. Overall this is performance neutral, but is in prep for some future work.
2024-03-06 19:01:52 +01:00
return true;
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
}
}
}
Avoid creating redundant vertices in V3DfgPeephole (#4944) Add a new data-structure V3DfgCache, which can be used to retrieve existing vertices with some given inputs vertices. Use this in V3DfgPeephole to eliminate the creation of redundant vertices. Overall this is performance neutral, but is in prep for some future work.
2024-03-06 19:01:52 +01:00
return false;
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
}
Add DfgPeephole patterns (#6149)
2025-07-07 17:25:29 +02:00
// Transformations that apply to all distributive and associative binary
// vertices 'Other' is the type that is distributive over 'Vertex',
// that is: a Other (b Vertex c) == (a Other b) Vertex (a Other c)
template <typename Other, typename Vertex>
VL_ATTR_WARN_UNUSED_RESULT bool distributiveAndAssociativeBinary(Vertex* vtxp) {
DfgVertex* const lhsp = vtxp->lhsp();
DfgVertex* const rhsp = vtxp->rhsp();
if (!lhsp->hasMultipleSinks() && !rhsp->hasMultipleSinks()) {
// Convert '(a Other b) Vertex (a Other c)' and associative
// variations to 'a Other (b Vertex c)'
if (Other* const lp = lhsp->cast<Other>()) {
if (Other* const rp = rhsp->cast<Other>()) {
DfgVertex* const llp = lp->lhsp();
DfgVertex* const lrp = lp->rhsp();
DfgVertex* const rlp = rp->lhsp();
DfgVertex* const rrp = rp->rhsp();
DfgVertex* ap = nullptr;
DfgVertex* bp = nullptr;
DfgVertex* cp = nullptr;
if (llp == rlp) {
ap = llp;
bp = lrp;
cp = rrp;
} else if (llp == rrp) {
ap = llp;
bp = lrp;
cp = rlp;
} else if (lrp == rlp) {
ap = lrp;
bp = llp;
cp = rrp;
} else if (lrp == rrp) {
ap = lrp;
bp = llp;
cp = rlp;
}
if (ap) {
APPLYING(REPLACE_DISTRIBUTIVE_BINARY) {
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(make<Other>(vtxp, ap, make<Vertex>(lhsp, bp, cp)));
Add DfgPeephole patterns (#6149)
2025-07-07 17:25:29 +02:00
return true;
}
}
}
}
}
return false;
}
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
// Bitwise operation with one side Const, and the other side a Concat
template <typename Vertex>
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
VL_ATTR_WARN_UNUSED_RESULT bool
tryPushBitwiseOpThroughConcat(Vertex* const vtxp, DfgConst* constp, DfgConcat* concatp) {
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
FileLine* const flp = vtxp->fileline();
// If at least one of the sides of the Concat constant, or width 1 (i.e.: can be
// further simplified), then push the Vertex past the Concat
if (concatp->lhsp()->is<DfgConst>() || concatp->rhsp()->is<DfgConst>() //
Improve Dfg type system (#6390) Added a mini type system for Dfg using DfgDataType to replace Dfg's use of AstNodeDType. This is much more restricted and represents only the types Dfg can handle in a canonical form. This will be needed when adding more support for unpacked arrays and maybe unpacked structs one day. Also added an internal type checker for DfgGraphs which encodes all the assumptions the code makes about type relationships in the graph. Run this in a few places with --debug-check. Fix resulting fallout.
2025-09-07 21:38:50 +02:00
|| concatp->lhsp()->dtype() == m_bitDType || concatp->rhsp()->dtype() == m_bitDType) {
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
APPLYING(PUSH_BITWISE_OP_THROUGH_CONCAT) {
const uint32_t width = concatp->width();
Improve Dfg type system (#6390) Added a mini type system for Dfg using DfgDataType to replace Dfg's use of AstNodeDType. This is much more restricted and represents only the types Dfg can handle in a canonical form. This will be needed when adding more support for unpacked arrays and maybe unpacked structs one day. Also added an internal type checker for DfgGraphs which encodes all the assumptions the code makes about type relationships in the graph. Run this in a few places with --debug-check. Fix resulting fallout.
2025-09-07 21:38:50 +02:00
const DfgDataType& lDtype = concatp->lhsp()->dtype();
const DfgDataType& rDtype = concatp->rhsp()->dtype();
const uint32_t lWidth = lDtype.size();
const uint32_t rWidth = rDtype.size();
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
// The new Lhs vertex
DfgConst* const newLhsConstp = makeZero(constp->fileline(), lWidth);
newLhsConstp->num().opSel(constp->num(), width - 1, rWidth);
Improve Dfg type system (#6390) Added a mini type system for Dfg using DfgDataType to replace Dfg's use of AstNodeDType. This is much more restricted and represents only the types Dfg can handle in a canonical form. This will be needed when adding more support for unpacked arrays and maybe unpacked structs one day. Also added an internal type checker for DfgGraphs which encodes all the assumptions the code makes about type relationships in the graph. Run this in a few places with --debug-check. Fix resulting fallout.
2025-09-07 21:38:50 +02:00
Vertex* const newLhsp = make<Vertex>(flp, lDtype, newLhsConstp, concatp->lhsp());
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
// The new Rhs vertex
DfgConst* const newRhsConstp = makeZero(constp->fileline(), rWidth);
newRhsConstp->num().opSel(constp->num(), rWidth - 1, 0);
Improve Dfg type system (#6390) Added a mini type system for Dfg using DfgDataType to replace Dfg's use of AstNodeDType. This is much more restricted and represents only the types Dfg can handle in a canonical form. This will be needed when adding more support for unpacked arrays and maybe unpacked structs one day. Also added an internal type checker for DfgGraphs which encodes all the assumptions the code makes about type relationships in the graph. Run this in a few places with --debug-check. Fix resulting fallout.
2025-09-07 21:38:50 +02:00
Vertex* const newRhsp = make<Vertex>(flp, rDtype, newRhsConstp, concatp->rhsp());
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
// Replace this vertex
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(make<DfgConcat>(concatp, newLhsp, newRhsp));
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
return true;
}
}
return false;
}
template <typename Vertex>
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
VL_ATTR_WARN_UNUSED_RESULT bool
tryPushCompareOpThroughConcat(Vertex* const vtxp, DfgConst* constp, DfgConcat* concatp) {
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
FileLine* const flp = vtxp->fileline();
Avoid creating redundant vertices in V3DfgPeephole (#4944) Add a new data-structure V3DfgCache, which can be used to retrieve existing vertices with some given inputs vertices. Use this in V3DfgPeephole to eliminate the creation of redundant vertices. Overall this is performance neutral, but is in prep for some future work.
2024-03-06 19:01:52 +01:00
// If at least one of the sides of the Concat is constant, then push the Vertex past
// the Concat
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
if (concatp->lhsp()->is<DfgConst>() || concatp->rhsp()->is<DfgConst>()) {
APPLYING(PUSH_COMPARE_OP_THROUGH_CONCAT) {
const uint32_t width = concatp->width();
const uint32_t lWidth = concatp->lhsp()->width();
const uint32_t rWidth = concatp->rhsp()->width();
// The new Lhs vertex
DfgConst* const newLhsConstp = makeZero(constp->fileline(), lWidth);
newLhsConstp->num().opSel(constp->num(), width - 1, rWidth);
Avoid creating redundant vertices in V3DfgPeephole (#4944) Add a new data-structure V3DfgCache, which can be used to retrieve existing vertices with some given inputs vertices. Use this in V3DfgPeephole to eliminate the creation of redundant vertices. Overall this is performance neutral, but is in prep for some future work.
2024-03-06 19:01:52 +01:00
Vertex* const newLhsp
= make<Vertex>(flp, m_bitDType, newLhsConstp, concatp->lhsp());
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
// The new Rhs vertex
DfgConst* const newRhsConstp = makeZero(constp->fileline(), rWidth);
newRhsConstp->num().opSel(constp->num(), rWidth - 1, 0);
Avoid creating redundant vertices in V3DfgPeephole (#4944) Add a new data-structure V3DfgCache, which can be used to retrieve existing vertices with some given inputs vertices. Use this in V3DfgPeephole to eliminate the creation of redundant vertices. Overall this is performance neutral, but is in prep for some future work.
2024-03-06 19:01:52 +01:00
Vertex* const newRhsp
= make<Vertex>(flp, m_bitDType, newRhsConstp, concatp->rhsp());
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
// The replacement Vertex
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
DfgVertexBinary* const resp
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
= std::is_same<Vertex, DfgEq>::value
Avoid creating redundant vertices in V3DfgPeephole (#4944) Add a new data-structure V3DfgCache, which can be used to retrieve existing vertices with some given inputs vertices. Use this in V3DfgPeephole to eliminate the creation of redundant vertices. Overall this is performance neutral, but is in prep for some future work.
2024-03-06 19:01:52 +01:00
? make<DfgAnd>(concatp->fileline(), m_bitDType, newLhsp, newRhsp)
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
: nullptr;
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
UASSERT_OBJ(resp, vtxp,
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
"Unhandled vertex type in 'tryPushCompareOpThroughConcat': "
<< vtxp->typeName());
// Replace this vertex
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(resp);
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
return true;
}
}
return false;
}
DFG: Add more peephole patterns
2022-09-30 17:19:53 +02:00
template <typename Bitwise>
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
VL_ATTR_WARN_UNUSED_RESULT bool tryPushBitwiseOpThroughReductions(Bitwise* const vtxp) {
DFG: Add more peephole patterns
2022-09-30 17:19:53 +02:00
using Reduction = BitwiseToReduction<Bitwise>;
if (Reduction* const lRedp = vtxp->lhsp()->template cast<Reduction>()) {
if (Reduction* const rRedp = vtxp->rhsp()->template cast<Reduction>()) {
DfgVertex* const lSrcp = lRedp->srcp();
DfgVertex* const rSrcp = rRedp->srcp();
Improve Dfg type system (#6390) Added a mini type system for Dfg using DfgDataType to replace Dfg's use of AstNodeDType. This is much more restricted and represents only the types Dfg can handle in a canonical form. This will be needed when adding more support for unpacked arrays and maybe unpacked structs one day. Also added an internal type checker for DfgGraphs which encodes all the assumptions the code makes about type relationships in the graph. Run this in a few places with --debug-check. Fix resulting fallout.
2025-09-07 21:38:50 +02:00
if (lSrcp->dtype() == rSrcp->dtype() && lSrcp->width() <= 64
Try to ensure DFG peephole patterns don't grow the graph Some optimizations are only a net win if they help us remove a graph node (or at least ensure they don't grow the graph), or yields otherwise special logic, so try to apply them only in these cases.
2022-10-04 19:26:53 +02:00
&& !lSrcp->hasMultipleSinks() && !rSrcp->hasMultipleSinks()) {
DFG: Add more peephole patterns
2022-09-30 17:19:53 +02:00
APPLYING(PUSH_BITWISE_THROUGH_REDUCTION) {
FileLine* const flp = vtxp->fileline();
Improve Dfg type system (#6390) Added a mini type system for Dfg using DfgDataType to replace Dfg's use of AstNodeDType. This is much more restricted and represents only the types Dfg can handle in a canonical form. This will be needed when adding more support for unpacked arrays and maybe unpacked structs one day. Also added an internal type checker for DfgGraphs which encodes all the assumptions the code makes about type relationships in the graph. Run this in a few places with --debug-check. Fix resulting fallout.
2025-09-07 21:38:50 +02:00
Bitwise* const bwp = make<Bitwise>(flp, lSrcp->dtype(), lSrcp, rSrcp);
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(make<Reduction>(flp, m_bitDType, bwp));
DFG: Add more peephole patterns
2022-09-30 17:19:53 +02:00
return true;
}
}
}
}
return false;
}
template <typename Reduction>
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
VL_ATTR_WARN_UNUSED_RESULT bool optimizeReduction(Reduction* const vtxp) {
DFG: Add more peephole patterns
2022-09-30 17:19:53 +02:00
using Bitwise = ReductionToBitwise<Reduction>;
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
Avoid creating redundant vertices in V3DfgPeephole (#4944) Add a new data-structure V3DfgCache, which can be used to retrieve existing vertices with some given inputs vertices. Use this in V3DfgPeephole to eliminate the creation of redundant vertices. Overall this is performance neutral, but is in prep for some future work.
2024-03-06 19:01:52 +01:00
if (foldUnary(vtxp)) return true;
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
DfgVertex* const srcp = vtxp->srcp();
DFG: Add more peephole patterns
2022-09-30 17:19:53 +02:00
FileLine* const flp = vtxp->fileline();
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
DFG: Add more peephole patterns
2022-09-30 17:19:53 +02:00
// Reduction of 1-bit value
Improve Dfg type system (#6390) Added a mini type system for Dfg using DfgDataType to replace Dfg's use of AstNodeDType. This is much more restricted and represents only the types Dfg can handle in a canonical form. This will be needed when adding more support for unpacked arrays and maybe unpacked structs one day. Also added an internal type checker for DfgGraphs which encodes all the assumptions the code makes about type relationships in the graph. Run this in a few places with --debug-check. Fix resulting fallout.
2025-09-07 21:38:50 +02:00
if (srcp->dtype() == m_bitDType) {
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
APPLYING(REMOVE_WIDTH_ONE_REDUCTION) {
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(srcp);
Avoid creating redundant vertices in V3DfgPeephole (#4944) Add a new data-structure V3DfgCache, which can be used to retrieve existing vertices with some given inputs vertices. Use this in V3DfgPeephole to eliminate the creation of redundant vertices. Overall this is performance neutral, but is in prep for some future work.
2024-03-06 19:01:52 +01:00
return true;
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
}
DFG: Add more peephole patterns
2022-09-30 17:19:53 +02:00
}
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
if (DfgCond* const condp = srcp->cast<DfgCond>()) {
if (condp->thenp()->is<DfgConst>() || condp->elsep()->is<DfgConst>()) {
APPLYING(PUSH_REDUCTION_THROUGH_COND_WITH_CONST_BRANCH) {
// The new 'then' vertex
Avoid creating redundant vertices in V3DfgPeephole (#4944) Add a new data-structure V3DfgCache, which can be used to retrieve existing vertices with some given inputs vertices. Use this in V3DfgPeephole to eliminate the creation of redundant vertices. Overall this is performance neutral, but is in prep for some future work.
2024-03-06 19:01:52 +01:00
Reduction* const newThenp = make<Reduction>(flp, m_bitDType, condp->thenp());
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
// The new 'else' vertex
Avoid creating redundant vertices in V3DfgPeephole (#4944) Add a new data-structure V3DfgCache, which can be used to retrieve existing vertices with some given inputs vertices. Use this in V3DfgPeephole to eliminate the creation of redundant vertices. Overall this is performance neutral, but is in prep for some future work.
2024-03-06 19:01:52 +01:00
Reduction* const newElsep = make<Reduction>(flp, m_bitDType, condp->elsep());
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
// The replacement Cond vertex
Avoid creating redundant vertices in V3DfgPeephole (#4944) Add a new data-structure V3DfgCache, which can be used to retrieve existing vertices with some given inputs vertices. Use this in V3DfgPeephole to eliminate the creation of redundant vertices. Overall this is performance neutral, but is in prep for some future work.
2024-03-06 19:01:52 +01:00
DfgCond* const newCondp = make<DfgCond>(condp->fileline(), m_bitDType,
condp->condp(), newThenp, newElsep);
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
// Replace this vertex
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(newCondp);
Avoid creating redundant vertices in V3DfgPeephole (#4944) Add a new data-structure V3DfgCache, which can be used to retrieve existing vertices with some given inputs vertices. Use this in V3DfgPeephole to eliminate the creation of redundant vertices. Overall this is performance neutral, but is in prep for some future work.
2024-03-06 19:01:52 +01:00
return true;
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
}
}
}
DFG: Add more peephole patterns
2022-09-30 17:19:53 +02:00
if (DfgConcat* const concatp = srcp->cast<DfgConcat>()) {
DfgPeephole: Further restrict PUSH_REDUCTION_THROUGH_CONCAT Only apply when there is guaranteed to be a subsequent constant folding and elimination of some of the expression, otherwise this sometimes interferes with the simplification of concatenations and harms overall performance.
2022-10-05 23:54:36 +02:00
if (concatp->lhsp()->is<DfgConst>() || concatp->rhsp()->is<DfgConst>()) {
Try to ensure DFG peephole patterns don't grow the graph Some optimizations are only a net win if they help us remove a graph node (or at least ensure they don't grow the graph), or yields otherwise special logic, so try to apply them only in these cases.
2022-10-04 19:26:53 +02:00
APPLYING(PUSH_REDUCTION_THROUGH_CONCAT) {
// Reduce the parts of the concatenation
Avoid creating redundant vertices in V3DfgPeephole (#4944) Add a new data-structure V3DfgCache, which can be used to retrieve existing vertices with some given inputs vertices. Use this in V3DfgPeephole to eliminate the creation of redundant vertices. Overall this is performance neutral, but is in prep for some future work.
2024-03-06 19:01:52 +01:00
Reduction* const lRedp
= make<Reduction>(concatp->fileline(), m_bitDType, concatp->lhsp());
Reduction* const rRedp
= make<Reduction>(concatp->fileline(), m_bitDType, concatp->rhsp());
Try to ensure DFG peephole patterns don't grow the graph Some optimizations are only a net win if they help us remove a graph node (or at least ensure they don't grow the graph), or yields otherwise special logic, so try to apply them only in these cases.
2022-10-04 19:26:53 +02:00
// Bitwise reduce the results
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(make<Bitwise>(flp, m_bitDType, lRedp, rRedp));
Avoid creating redundant vertices in V3DfgPeephole (#4944) Add a new data-structure V3DfgCache, which can be used to retrieve existing vertices with some given inputs vertices. Use this in V3DfgPeephole to eliminate the creation of redundant vertices. Overall this is performance neutral, but is in prep for some future work.
2024-03-06 19:01:52 +01:00
return true;
Try to ensure DFG peephole patterns don't grow the graph Some optimizations are only a net win if they help us remove a graph node (or at least ensure they don't grow the graph), or yields otherwise special logic, so try to apply them only in these cases.
2022-10-04 19:26:53 +02:00
}
DFG: Add more peephole patterns
2022-09-30 17:19:53 +02:00
}
}
Avoid creating redundant vertices in V3DfgPeephole (#4944) Add a new data-structure V3DfgCache, which can be used to retrieve existing vertices with some given inputs vertices. Use this in V3DfgPeephole to eliminate the creation of redundant vertices. Overall this is performance neutral, but is in prep for some future work.
2024-03-06 19:01:52 +01:00
return false;
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
}
Avoid creating redundant vertices in V3DfgPeephole (#4944) Add a new data-structure V3DfgCache, which can be used to retrieve existing vertices with some given inputs vertices. Use this in V3DfgPeephole to eliminate the creation of redundant vertices. Overall this is performance neutral, but is in prep for some future work.
2024-03-06 19:01:52 +01:00
template <typename Shift>
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
VL_ATTR_WARN_UNUSED_RESULT bool optimizeShiftRHS(Shift* const vtxp) {
Avoid creating redundant vertices in V3DfgPeephole (#4944) Add a new data-structure V3DfgCache, which can be used to retrieve existing vertices with some given inputs vertices. Use this in V3DfgPeephole to eliminate the creation of redundant vertices. Overall this is performance neutral, but is in prep for some future work.
2024-03-06 19:01:52 +01:00
static_assert(std::is_base_of<DfgVertexBinary, Shift>::value, "Must invoke on binary");
static_assert(std::is_final<Shift>::value, "Must invoke on final class");
if (const DfgConcat* const concatp = vtxp->rhsp()->template cast<DfgConcat>()) {
Internals: Improve DFG implementation details (#6355) Large scale refactoring to simplify some of the more obtuse internals of DFG. Remove multiple redundant internal APIs, simplify representation of variables, fix potential unsoundness in circular decomposition. No functional change intended.
2025-09-02 17:50:40 +02:00
if (isZero(concatp->lhsp())) { // Drop redundant zero extension
DfgPeephole: Use a work list driven algorithm for speed Replace the 'run to fixed point' algorithm with a work list driven approach. Instead of marking the graph as changed, we explicitly add vertices to the work list, to be visited, when a vertex is changed. This improves both memory locality (as the work list is processed in last in first out order), and removed unnecessary visitations when only a few nodes changes.
2022-11-04 16:50:20 +01:00
APPLYING(REMOVE_REDUNDANT_ZEXT_ON_RHS_OF_SHIFT) {
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(make<Shift>(vtxp, vtxp->lhsp(), concatp->rhsp()));
Avoid creating redundant vertices in V3DfgPeephole (#4944) Add a new data-structure V3DfgCache, which can be used to retrieve existing vertices with some given inputs vertices. Use this in V3DfgPeephole to eliminate the creation of redundant vertices. Overall this is performance neutral, but is in prep for some future work.
2024-03-06 19:01:52 +01:00
return true;
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
}
}
}
Avoid creating redundant vertices in V3DfgPeephole (#4944) Add a new data-structure V3DfgCache, which can be used to retrieve existing vertices with some given inputs vertices. Use this in V3DfgPeephole to eliminate the creation of redundant vertices. Overall this is performance neutral, but is in prep for some future work.
2024-03-06 19:01:52 +01:00
return false;
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
}
// VISIT methods
void visit(DfgVertex*) override {}
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
//=========================================================================
// DfgVertexUnary
//=========================================================================
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
void visit(DfgExtend* const vtxp) override {
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
if (foldUnary(vtxp)) return;
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
Avoid creating redundant vertices in V3DfgPeephole (#4944) Add a new data-structure V3DfgCache, which can be used to retrieve existing vertices with some given inputs vertices. Use this in V3DfgPeephole to eliminate the creation of redundant vertices. Overall this is performance neutral, but is in prep for some future work.
2024-03-06 19:01:52 +01:00
// Convert all Extend into Concat with zeros. This simplifies other patterns as they
// only need to handle Concat, which is more generic, and don't need special cases for
DFG: Add more peephole patterns
2022-09-30 17:19:53 +02:00
// Extend.
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
APPLYING(REPLACE_EXTEND) {
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
DfgVertex* const zerop
= makeZero(vtxp->fileline(), vtxp->width() - vtxp->srcp()->width());
replace(make<DfgConcat>(vtxp, zerop, vtxp->srcp()));
DfgPeephole: Use a work list driven algorithm for speed Replace the 'run to fixed point' algorithm with a work list driven approach. Instead of marking the graph as changed, we explicitly add vertices to the work list, to be visited, when a vertex is changed. This improves both memory locality (as the work list is processed in last in first out order), and removed unnecessary visitations when only a few nodes changes.
2022-11-04 16:50:20 +01:00
return;
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
}
}
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
void visit(DfgExtendS* const vtxp) override {
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
if (foldUnary(vtxp)) return;
}
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
void visit(DfgLogNot* const vtxp) override {
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
if (foldUnary(vtxp)) return;
}
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
void visit(DfgNegate* const vtxp) override {
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
if (foldUnary(vtxp)) return;
}
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
void visit(DfgNot* const vtxp) override {
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
if (foldUnary(vtxp)) return;
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
// Not of Cond
if (DfgCond* const condp = vtxp->srcp()->cast<DfgCond>()) {
// If at least one of the branches are a constant, push the Not past the Cond
if (condp->thenp()->is<DfgConst>() || condp->elsep()->is<DfgConst>()) {
APPLYING(PUSH_NOT_THROUGH_COND) {
// The new 'then' vertex
Avoid creating redundant vertices in V3DfgPeephole (#4944) Add a new data-structure V3DfgCache, which can be used to retrieve existing vertices with some given inputs vertices. Use this in V3DfgPeephole to eliminate the creation of redundant vertices. Overall this is performance neutral, but is in prep for some future work.
2024-03-06 19:01:52 +01:00
DfgNot* const newThenp = make<DfgNot>(vtxp, condp->thenp());
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
// The new 'else' vertex
Avoid creating redundant vertices in V3DfgPeephole (#4944) Add a new data-structure V3DfgCache, which can be used to retrieve existing vertices with some given inputs vertices. Use this in V3DfgPeephole to eliminate the creation of redundant vertices. Overall this is performance neutral, but is in prep for some future work.
2024-03-06 19:01:52 +01:00
DfgNot* const newElsep = make<DfgNot>(vtxp, condp->elsep());
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
// The replacement Cond vertex
Improve Dfg type system (#6390) Added a mini type system for Dfg using DfgDataType to replace Dfg's use of AstNodeDType. This is much more restricted and represents only the types Dfg can handle in a canonical form. This will be needed when adding more support for unpacked arrays and maybe unpacked structs one day. Also added an internal type checker for DfgGraphs which encodes all the assumptions the code makes about type relationships in the graph. Run this in a few places with --debug-check. Fix resulting fallout.
2025-09-07 21:38:50 +02:00
DfgCond* const newCondp = make<DfgCond>(condp->fileline(), vtxp->dtype(),
Avoid creating redundant vertices in V3DfgPeephole (#4944) Add a new data-structure V3DfgCache, which can be used to retrieve existing vertices with some given inputs vertices. Use this in V3DfgPeephole to eliminate the creation of redundant vertices. Overall this is performance neutral, but is in prep for some future work.
2024-03-06 19:01:52 +01:00
condp->condp(), newThenp, newElsep);
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
// Replace this vertex
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(newCondp);
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
return;
}
}
}
// Not of Not
if (DfgNot* const notp = vtxp->srcp()->cast<DfgNot>()) {
APPLYING(REMOVE_NOT_NOT) {
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(notp->srcp());
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
return;
}
}
Try to ensure DFG peephole patterns don't grow the graph Some optimizations are only a net win if they help us remove a graph node (or at least ensure they don't grow the graph), or yields otherwise special logic, so try to apply them only in these cases.
2022-10-04 19:26:53 +02:00
if (!vtxp->srcp()->hasMultipleSinks()) {
// Not of Eq
if (DfgEq* const eqp = vtxp->srcp()->cast<DfgEq>()) {
APPLYING(REPLACE_NOT_EQ) {
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(
make<DfgNeq>(eqp->fileline(), vtxp->dtype(), eqp->lhsp(), eqp->rhsp()));
Try to ensure DFG peephole patterns don't grow the graph Some optimizations are only a net win if they help us remove a graph node (or at least ensure they don't grow the graph), or yields otherwise special logic, so try to apply them only in these cases.
2022-10-04 19:26:53 +02:00
return;
}
DFG: Add more peephole patterns
2022-09-30 17:19:53 +02:00
}
Try to ensure DFG peephole patterns don't grow the graph Some optimizations are only a net win if they help us remove a graph node (or at least ensure they don't grow the graph), or yields otherwise special logic, so try to apply them only in these cases.
2022-10-04 19:26:53 +02:00
// Not of Neq
if (DfgNeq* const neqp = vtxp->srcp()->cast<DfgNeq>()) {
APPLYING(REPLACE_NOT_NEQ) {
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(
make<DfgEq>(neqp->fileline(), vtxp->dtype(), neqp->lhsp(), neqp->rhsp()));
Try to ensure DFG peephole patterns don't grow the graph Some optimizations are only a net win if they help us remove a graph node (or at least ensure they don't grow the graph), or yields otherwise special logic, so try to apply them only in these cases.
2022-10-04 19:26:53 +02:00
return;
}
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
}
}
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
}
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
void visit(DfgRedOr* const vtxp) override {
Avoid creating redundant vertices in V3DfgPeephole (#4944) Add a new data-structure V3DfgCache, which can be used to retrieve existing vertices with some given inputs vertices. Use this in V3DfgPeephole to eliminate the creation of redundant vertices. Overall this is performance neutral, but is in prep for some future work.
2024-03-06 19:01:52 +01:00
if (optimizeReduction(vtxp)) return;
}
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
void visit(DfgRedAnd* const vtxp) override {
Avoid creating redundant vertices in V3DfgPeephole (#4944) Add a new data-structure V3DfgCache, which can be used to retrieve existing vertices with some given inputs vertices. Use this in V3DfgPeephole to eliminate the creation of redundant vertices. Overall this is performance neutral, but is in prep for some future work.
2024-03-06 19:01:52 +01:00
if (optimizeReduction(vtxp)) return;
}
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
void visit(DfgRedXor* const vtxp) override {
Avoid creating redundant vertices in V3DfgPeephole (#4944) Add a new data-structure V3DfgCache, which can be used to retrieve existing vertices with some given inputs vertices. Use this in V3DfgPeephole to eliminate the creation of redundant vertices. Overall this is performance neutral, but is in prep for some future work.
2024-03-06 19:01:52 +01:00
if (optimizeReduction(vtxp)) return;
}
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
void visit(DfgSel* const vtxp) override {
DFG: Special case representation of AstSel AstSel is a ternary node, but the 'widthp' is always constant and is hence redundant, and 'lsbp' is very often constant. As AstSel is fairly common, we special case as a DfgSel for the constant 'lsbp', and as 'DfgMux` for the non-constant 'lsbp'.
2022-10-06 19:34:18 +02:00
DfgVertex* const fromp = vtxp->fromp();
FileLine* const flp = vtxp->fileline();
const uint32_t lsb = vtxp->lsb();
const uint32_t width = vtxp->width();
const uint32_t msb = lsb + width - 1;
if (DfgConst* const constp = fromp->cast<DfgConst>()) {
APPLYING(FOLD_SEL) {
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
DfgConst* const resp = makeZero(flp, width);
resp->num().opSel(constp->num(), msb, lsb);
replace(resp);
DFG: Special case representation of AstSel AstSel is a ternary node, but the 'widthp' is always constant and is hence redundant, and 'lsbp' is very often constant. As AstSel is fairly common, we special case as a DfgSel for the constant 'lsbp', and as 'DfgMux` for the non-constant 'lsbp'.
2022-10-06 19:34:18 +02:00
return;
}
}
// Full width select, replace with the source.
if (fromp->width() == width) {
Remove out of range Sel in V3Unknown (#5443) Fixes #5393
2024-09-09 15:09:29 +02:00
UASSERT_OBJ(lsb == 0, fromp, "Out of range select should have been fixed up earlier");
DFG: Special case representation of AstSel AstSel is a ternary node, but the 'widthp' is always constant and is hence redundant, and 'lsbp' is very often constant. As AstSel is fairly common, we special case as a DfgSel for the constant 'lsbp', and as 'DfgMux` for the non-constant 'lsbp'.
2022-10-06 19:34:18 +02:00
APPLYING(REMOVE_FULL_WIDTH_SEL) {
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(fromp);
DFG: Special case representation of AstSel AstSel is a ternary node, but the 'widthp' is always constant and is hence redundant, and 'lsbp' is very often constant. As AstSel is fairly common, we special case as a DfgSel for the constant 'lsbp', and as 'DfgMux` for the non-constant 'lsbp'.
2022-10-06 19:34:18 +02:00
return;
}
}
// Sel from Concat
if (DfgConcat* const concatp = fromp->cast<DfgConcat>()) {
DfgVertex* const lhsp = concatp->lhsp();
DfgVertex* const rhsp = concatp->rhsp();
if (msb < rhsp->width()) {
// If the select is entirely from rhs, then replace with sel from rhs
APPLYING(REMOVE_SEL_FROM_RHS_OF_CONCAT) { //
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(make<DfgSel>(vtxp, rhsp, vtxp->lsb()));
return;
DFG: Special case representation of AstSel AstSel is a ternary node, but the 'widthp' is always constant and is hence redundant, and 'lsbp' is very often constant. As AstSel is fairly common, we special case as a DfgSel for the constant 'lsbp', and as 'DfgMux` for the non-constant 'lsbp'.
2022-10-06 19:34:18 +02:00
}
} else if (lsb >= rhsp->width()) {
// If the select is entirely from the lhs, then replace with sel from lhs
APPLYING(REMOVE_SEL_FROM_LHS_OF_CONCAT) {
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(make<DfgSel>(vtxp, lhsp, lsb - rhsp->width()));
return;
DFG: Special case representation of AstSel AstSel is a ternary node, but the 'widthp' is always constant and is hence redundant, and 'lsbp' is very often constant. As AstSel is fairly common, we special case as a DfgSel for the constant 'lsbp', and as 'DfgMux` for the non-constant 'lsbp'.
2022-10-06 19:34:18 +02:00
}
}
}
if (DfgReplicate* const repp = fromp->cast<DfgReplicate>()) {
// If the Sel is wholly into the source of the Replicate, push the Sel through the
// Replicate and apply it directly to the source of the Replicate.
const uint32_t srcWidth = repp->srcp()->width();
if (width <= srcWidth) {
const uint32_t newLsb = lsb % srcWidth;
if (newLsb + width <= srcWidth) {
APPLYING(PUSH_SEL_THROUGH_REPLICATE) {
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(make<DfgSel>(vtxp, repp->srcp(), newLsb));
return;
DFG: Special case representation of AstSel AstSel is a ternary node, but the 'widthp' is always constant and is hence redundant, and 'lsbp' is very often constant. As AstSel is fairly common, we special case as a DfgSel for the constant 'lsbp', and as 'DfgMux` for the non-constant 'lsbp'.
2022-10-06 19:34:18 +02:00
}
}
}
}
// Sel from Not
if (DfgNot* const notp = fromp->cast<DfgNot>()) {
// Replace "Sel from Not" with "Not of Sel"
if (!notp->hasMultipleSinks()) {
APPLYING(PUSH_SEL_THROUGH_NOT) {
// Make Sel select from source of Not
Avoid creating redundant vertices in V3DfgPeephole (#4944) Add a new data-structure V3DfgCache, which can be used to retrieve existing vertices with some given inputs vertices. Use this in V3DfgPeephole to eliminate the creation of redundant vertices. Overall this is performance neutral, but is in prep for some future work.
2024-03-06 19:01:52 +01:00
DfgSel* const newSelp = make<DfgSel>(vtxp, notp->srcp(), vtxp->lsb());
DFG: Special case representation of AstSel AstSel is a ternary node, but the 'widthp' is always constant and is hence redundant, and 'lsbp' is very often constant. As AstSel is fairly common, we special case as a DfgSel for the constant 'lsbp', and as 'DfgMux` for the non-constant 'lsbp'.
2022-10-06 19:34:18 +02:00
// Add Not after Sel
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(make<DfgNot>(notp->fileline(), vtxp->dtype(), newSelp));
return;
DFG: Special case representation of AstSel AstSel is a ternary node, but the 'widthp' is always constant and is hence redundant, and 'lsbp' is very often constant. As AstSel is fairly common, we special case as a DfgSel for the constant 'lsbp', and as 'DfgMux` for the non-constant 'lsbp'.
2022-10-06 19:34:18 +02:00
}
}
}
// Sel from Sel
if (DfgSel* const selp = fromp->cast<DfgSel>()) {
APPLYING(REPLACE_SEL_FROM_SEL) {
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
// Select from the source of the source Sel with adjusted LSB
replace(make<DfgSel>(vtxp, selp->fromp(), lsb + selp->lsb()));
return;
DFG: Special case representation of AstSel AstSel is a ternary node, but the 'widthp' is always constant and is hence redundant, and 'lsbp' is very often constant. As AstSel is fairly common, we special case as a DfgSel for the constant 'lsbp', and as 'DfgMux` for the non-constant 'lsbp'.
2022-10-06 19:34:18 +02:00
}
}
// Sel from Cond
if (DfgCond* const condp = fromp->cast<DfgCond>()) {
// If at least one of the branches are a constant, push the select past the cond
DFG peephole: Only push SEL through COND when SEL is the only sink. (#6071) This avoids potential O(n) logic duplication where 'n' is the fanout of the COND node. Fixes #6064.
2025-06-08 12:12:39 +02:00
if (!condp->hasMultipleSinks()
&& (condp->thenp()->is<DfgConst>() || condp->elsep()->is<DfgConst>())) {
DFG: Special case representation of AstSel AstSel is a ternary node, but the 'widthp' is always constant and is hence redundant, and 'lsbp' is very often constant. As AstSel is fairly common, we special case as a DfgSel for the constant 'lsbp', and as 'DfgMux` for the non-constant 'lsbp'.
2022-10-06 19:34:18 +02:00
APPLYING(PUSH_SEL_THROUGH_COND) {
// The new 'then' vertex
Avoid creating redundant vertices in V3DfgPeephole (#4944) Add a new data-structure V3DfgCache, which can be used to retrieve existing vertices with some given inputs vertices. Use this in V3DfgPeephole to eliminate the creation of redundant vertices. Overall this is performance neutral, but is in prep for some future work.
2024-03-06 19:01:52 +01:00
DfgSel* const newThenp = make<DfgSel>(vtxp, condp->thenp(), lsb);
DFG: Special case representation of AstSel AstSel is a ternary node, but the 'widthp' is always constant and is hence redundant, and 'lsbp' is very often constant. As AstSel is fairly common, we special case as a DfgSel for the constant 'lsbp', and as 'DfgMux` for the non-constant 'lsbp'.
2022-10-06 19:34:18 +02:00
// The new 'else' vertex
Avoid creating redundant vertices in V3DfgPeephole (#4944) Add a new data-structure V3DfgCache, which can be used to retrieve existing vertices with some given inputs vertices. Use this in V3DfgPeephole to eliminate the creation of redundant vertices. Overall this is performance neutral, but is in prep for some future work.
2024-03-06 19:01:52 +01:00
DfgSel* const newElsep = make<DfgSel>(vtxp, condp->elsep(), lsb);
DFG: Special case representation of AstSel AstSel is a ternary node, but the 'widthp' is always constant and is hence redundant, and 'lsbp' is very often constant. As AstSel is fairly common, we special case as a DfgSel for the constant 'lsbp', and as 'DfgMux` for the non-constant 'lsbp'.
2022-10-06 19:34:18 +02:00
// The replacement Cond vertex
Improve Dfg type system (#6390) Added a mini type system for Dfg using DfgDataType to replace Dfg's use of AstNodeDType. This is much more restricted and represents only the types Dfg can handle in a canonical form. This will be needed when adding more support for unpacked arrays and maybe unpacked structs one day. Also added an internal type checker for DfgGraphs which encodes all the assumptions the code makes about type relationships in the graph. Run this in a few places with --debug-check. Fix resulting fallout.
2025-09-07 21:38:50 +02:00
DfgCond* const newCondp = make<DfgCond>(condp->fileline(), vtxp->dtype(),
Avoid creating redundant vertices in V3DfgPeephole (#4944) Add a new data-structure V3DfgCache, which can be used to retrieve existing vertices with some given inputs vertices. Use this in V3DfgPeephole to eliminate the creation of redundant vertices. Overall this is performance neutral, but is in prep for some future work.
2024-03-06 19:01:52 +01:00
condp->condp(), newThenp, newElsep);
DFG: Special case representation of AstSel AstSel is a ternary node, but the 'widthp' is always constant and is hence redundant, and 'lsbp' is very often constant. As AstSel is fairly common, we special case as a DfgSel for the constant 'lsbp', and as 'DfgMux` for the non-constant 'lsbp'.
2022-10-06 19:34:18 +02:00
// Replace this vertex
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(newCondp);
DFG: Special case representation of AstSel AstSel is a ternary node, but the 'widthp' is always constant and is hence redundant, and 'lsbp' is very often constant. As AstSel is fairly common, we special case as a DfgSel for the constant 'lsbp', and as 'DfgMux` for the non-constant 'lsbp'.
2022-10-06 19:34:18 +02:00
return;
}
}
}
// Sel from ShiftL
if (DfgShiftL* const shiftLp = fromp->cast<DfgShiftL>()) {
// If selecting bottom bits of left shift, push the Sel before the shift
if (lsb == 0) {
UASSERT_OBJ(shiftLp->lhsp()->width() >= width, vtxp, "input of shift narrow");
APPLYING(PUSH_SEL_THROUGH_SHIFTL) {
Avoid creating redundant vertices in V3DfgPeephole (#4944) Add a new data-structure V3DfgCache, which can be used to retrieve existing vertices with some given inputs vertices. Use this in V3DfgPeephole to eliminate the creation of redundant vertices. Overall this is performance neutral, but is in prep for some future work.
2024-03-06 19:01:52 +01:00
DfgSel* const newSelp = make<DfgSel>(vtxp, shiftLp->lhsp(), vtxp->lsb());
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(make<DfgShiftL>(vtxp, newSelp, shiftLp->rhsp()));
return;
DFG: Special case representation of AstSel AstSel is a ternary node, but the 'widthp' is always constant and is hence redundant, and 'lsbp' is very often constant. As AstSel is fairly common, we special case as a DfgSel for the constant 'lsbp', and as 'DfgMux` for the non-constant 'lsbp'.
2022-10-06 19:34:18 +02:00
}
}
}
Optimize complex combinational logic in DFG (#6298) This patch adds DfgLogic, which is a vertex that represents a whole, arbitrarily complex combinational AstAlways or AstAssignW in the DfgGraph. Implementing this requires computing the variables live at entry to the AstAlways (variables read by the block), so there is a new ControlFlowGraph data structure and a classical data-flow analysis based live variable analysis to do that at the variable level (as opposed to bit/element level). The actual CFG construction and live variable analysis is best effort, and might fail for currently unhandled constructs or data types. This can be extended later. V3DfgAstToDfg is changed to convert the Ast into an initial DfgGraph containing only DfgLogic, DfgVertexSplice and DfgVertexVar vertices. The DfgLogic are then subsequently synthesized into primitive operations by the new V3DfgSynthesize pass, which is a combination of the old V3DfgAstToDfg conversion and new code to handle AstAlways blocks with complex flow control. V3DfgSynthesize by default will synthesize roughly the same constructs as V3DfgAstToDfg used to handle before, plus any logic that is part of a combinational cycle within the DfgGraph. This enables breaking up these cycles, for which there are extensions to V3DfgBreakCycles in this patch as well. V3DfgSynthesize will then delete all non synthesized or non synthesizable DfgLogic vertices and the rest of the Dfg pipeline is identical, with minor changes to adjust for the changed representation. Because with this change we can now eliminate many more UNOPTFLAT, DFG has been disabled in all the tests that specifically target testing the scheduling and reporting of circular combinational logic.
2025-08-19 16:06:38 +02:00
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
// Sel from a partial variable or narrowed vertex
{
DfgSplicePacked* splicep = fromp->cast<DfgSplicePacked>();
if (DfgVarPacked* const varp = fromp->cast<DfgVarPacked>()) {
Optimize complex combinational logic in DFG (#6298) This patch adds DfgLogic, which is a vertex that represents a whole, arbitrarily complex combinational AstAlways or AstAssignW in the DfgGraph. Implementing this requires computing the variables live at entry to the AstAlways (variables read by the block), so there is a new ControlFlowGraph data structure and a classical data-flow analysis based live variable analysis to do that at the variable level (as opposed to bit/element level). The actual CFG construction and live variable analysis is best effort, and might fail for currently unhandled constructs or data types. This can be extended later. V3DfgAstToDfg is changed to convert the Ast into an initial DfgGraph containing only DfgLogic, DfgVertexSplice and DfgVertexVar vertices. The DfgLogic are then subsequently synthesized into primitive operations by the new V3DfgSynthesize pass, which is a combination of the old V3DfgAstToDfg conversion and new code to handle AstAlways blocks with complex flow control. V3DfgSynthesize by default will synthesize roughly the same constructs as V3DfgAstToDfg used to handle before, plus any logic that is part of a combinational cycle within the DfgGraph. This enables breaking up these cycles, for which there are extensions to V3DfgBreakCycles in this patch as well. V3DfgSynthesize will then delete all non synthesized or non synthesizable DfgLogic vertices and the rest of the Dfg pipeline is identical, with minor changes to adjust for the changed representation. Because with this change we can now eliminate many more UNOPTFLAT, DFG has been disabled in all the tests that specifically target testing the scheduling and reporting of circular combinational logic.
2025-08-19 16:06:38 +02:00
// Must be a splice, otherwise it would have been inlined
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
if (varp->tmpForp() && varp->srcp()) splicep = varp->srcp()->as<DfgSplicePacked>();
}
if (splicep) {
Internals: Remove unused vertices immediately in DfgPeephole
2026-03-23 16:06:07 +01:00
DfgVertex* driverp = nullptr;
uint32_t driverLsb = 0;
Internals: Improve DFG implementation details (#6355) Large scale refactoring to simplify some of the more obtuse internals of DFG. Remove multiple redundant internal APIs, simplify representation of variables, fix potential unsoundness in circular decomposition. No functional change intended.
2025-09-02 17:50:40 +02:00
splicep->foreachDriver([&](DfgVertex& src, const uint32_t dLsb) {
const uint32_t dMsb = dLsb + src.width() - 1;
Optimize complex combinational logic in DFG (#6298) This patch adds DfgLogic, which is a vertex that represents a whole, arbitrarily complex combinational AstAlways or AstAssignW in the DfgGraph. Implementing this requires computing the variables live at entry to the AstAlways (variables read by the block), so there is a new ControlFlowGraph data structure and a classical data-flow analysis based live variable analysis to do that at the variable level (as opposed to bit/element level). The actual CFG construction and live variable analysis is best effort, and might fail for currently unhandled constructs or data types. This can be extended later. V3DfgAstToDfg is changed to convert the Ast into an initial DfgGraph containing only DfgLogic, DfgVertexSplice and DfgVertexVar vertices. The DfgLogic are then subsequently synthesized into primitive operations by the new V3DfgSynthesize pass, which is a combination of the old V3DfgAstToDfg conversion and new code to handle AstAlways blocks with complex flow control. V3DfgSynthesize by default will synthesize roughly the same constructs as V3DfgAstToDfg used to handle before, plus any logic that is part of a combinational cycle within the DfgGraph. This enables breaking up these cycles, for which there are extensions to V3DfgBreakCycles in this patch as well. V3DfgSynthesize will then delete all non synthesized or non synthesizable DfgLogic vertices and the rest of the Dfg pipeline is identical, with minor changes to adjust for the changed representation. Because with this change we can now eliminate many more UNOPTFLAT, DFG has been disabled in all the tests that specifically target testing the scheduling and reporting of circular combinational logic.
2025-08-19 16:06:38 +02:00
// If it does not cover the whole searched bit range, move on
Internals: Improve DFG implementation details (#6355) Large scale refactoring to simplify some of the more obtuse internals of DFG. Remove multiple redundant internal APIs, simplify representation of variables, fix potential unsoundness in circular decomposition. No functional change intended.
2025-09-02 17:50:40 +02:00
if (lsb < dLsb || dMsb < msb) return false;
Internals: Remove unused vertices immediately in DfgPeephole
2026-03-23 16:06:07 +01:00
// Save the driver
driverp = &src;
driverLsb = dLsb;
Internals: Improve DFG implementation details (#6355) Large scale refactoring to simplify some of the more obtuse internals of DFG. Remove multiple redundant internal APIs, simplify representation of variables, fix potential unsoundness in circular decomposition. No functional change intended.
2025-09-02 17:50:40 +02:00
return true;
});
Internals: Remove unused vertices immediately in DfgPeephole
2026-03-23 16:06:07 +01:00
if (driverp) {
Optimize complex combinational logic in DFG (#6298) This patch adds DfgLogic, which is a vertex that represents a whole, arbitrarily complex combinational AstAlways or AstAssignW in the DfgGraph. Implementing this requires computing the variables live at entry to the AstAlways (variables read by the block), so there is a new ControlFlowGraph data structure and a classical data-flow analysis based live variable analysis to do that at the variable level (as opposed to bit/element level). The actual CFG construction and live variable analysis is best effort, and might fail for currently unhandled constructs or data types. This can be extended later. V3DfgAstToDfg is changed to convert the Ast into an initial DfgGraph containing only DfgLogic, DfgVertexSplice and DfgVertexVar vertices. The DfgLogic are then subsequently synthesized into primitive operations by the new V3DfgSynthesize pass, which is a combination of the old V3DfgAstToDfg conversion and new code to handle AstAlways blocks with complex flow control. V3DfgSynthesize by default will synthesize roughly the same constructs as V3DfgAstToDfg used to handle before, plus any logic that is part of a combinational cycle within the DfgGraph. This enables breaking up these cycles, for which there are extensions to V3DfgBreakCycles in this patch as well. V3DfgSynthesize will then delete all non synthesized or non synthesizable DfgLogic vertices and the rest of the Dfg pipeline is identical, with minor changes to adjust for the changed representation. Because with this change we can now eliminate many more UNOPTFLAT, DFG has been disabled in all the tests that specifically target testing the scheduling and reporting of circular combinational logic.
2025-08-19 16:06:38 +02:00
APPLYING(PUSH_SEL_THROUGH_SPLICE) {
Internals: Remove unused vertices immediately in DfgPeephole
2026-03-23 16:06:07 +01:00
replace(make<DfgSel>(vtxp, driverp, lsb - driverLsb));
Optimize: Remove redundant variables during Dfg Peephole pass (#7076) Many rules in the Dfg Peephole pass check if a node has more than one sinks. Redundant variables that will ultimately be removed can prevent these from matching. Remove such variables during the Peeophole pass itself to enable more matches.
2026-02-14 13:33:20 +01:00
return;
Optimize complex combinational logic in DFG (#6298) This patch adds DfgLogic, which is a vertex that represents a whole, arbitrarily complex combinational AstAlways or AstAssignW in the DfgGraph. Implementing this requires computing the variables live at entry to the AstAlways (variables read by the block), so there is a new ControlFlowGraph data structure and a classical data-flow analysis based live variable analysis to do that at the variable level (as opposed to bit/element level). The actual CFG construction and live variable analysis is best effort, and might fail for currently unhandled constructs or data types. This can be extended later. V3DfgAstToDfg is changed to convert the Ast into an initial DfgGraph containing only DfgLogic, DfgVertexSplice and DfgVertexVar vertices. The DfgLogic are then subsequently synthesized into primitive operations by the new V3DfgSynthesize pass, which is a combination of the old V3DfgAstToDfg conversion and new code to handle AstAlways blocks with complex flow control. V3DfgSynthesize by default will synthesize roughly the same constructs as V3DfgAstToDfg used to handle before, plus any logic that is part of a combinational cycle within the DfgGraph. This enables breaking up these cycles, for which there are extensions to V3DfgBreakCycles in this patch as well. V3DfgSynthesize will then delete all non synthesized or non synthesizable DfgLogic vertices and the rest of the Dfg pipeline is identical, with minor changes to adjust for the changed representation. Because with this change we can now eliminate many more UNOPTFLAT, DFG has been disabled in all the tests that specifically target testing the scheduling and reporting of circular combinational logic.
2025-08-19 16:06:38 +02:00
}
}
}
}
DFG: Special case representation of AstSel AstSel is a ternary node, but the 'widthp' is always constant and is hence redundant, and 'lsbp' is very often constant. As AstSel is fairly common, we special case as a DfgSel for the constant 'lsbp', and as 'DfgMux` for the non-constant 'lsbp'.
2022-10-06 19:34:18 +02:00
}
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
//=========================================================================
// DfgVertexBinary - bitwise
//=========================================================================
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
void visit(DfgAnd* const vtxp) override {
Add DfgPeephole patterns (#6149)
2025-07-07 17:25:29 +02:00
DfgVertex* const lhsp = vtxp->lhsp();
DfgVertex* const rhsp = vtxp->rhsp();
Improve Dfg type system (#6390) Added a mini type system for Dfg using DfgDataType to replace Dfg's use of AstNodeDType. This is much more restricted and represents only the types Dfg can handle in a canonical form. This will be needed when adding more support for unpacked arrays and maybe unpacked structs one day. Also added an internal type checker for DfgGraphs which encodes all the assumptions the code makes about type relationships in the graph. Run this in a few places with --debug-check. Fix resulting fallout.
2025-09-07 21:38:50 +02:00
if (isSame(lhsp, rhsp)) {
Add DfgPeephole patterns (#6149)
2025-07-07 17:25:29 +02:00
APPLYING(REMOVE_AND_WITH_SELF) {
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(lhsp);
Add DfgPeephole patterns (#6149)
2025-07-07 17:25:29 +02:00
return;
}
}
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
DfgPeephole: Rework folding of associative operations Allow constant folding through adjacent nodes of all associative operations, for example '((a & 2) & 3)' or '(3 & (2 & a))' can now be folded into '(a & 2)' and '(2 & a)' respectively. Also improve speed of making associative expression trees right leaning by using rotation of the existing vertices whenever instead of allocation of new nodes.
2022-10-05 11:36:28 +02:00
if (associativeBinary(vtxp)) return;
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
Avoid creating redundant vertices in V3DfgPeephole (#4944) Add a new data-structure V3DfgCache, which can be used to retrieve existing vertices with some given inputs vertices. Use this in V3DfgPeephole to eliminate the creation of redundant vertices. Overall this is performance neutral, but is in prep for some future work.
2024-03-06 19:01:52 +01:00
if (commutativeBinary(vtxp)) return;
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
FileLine* const flp = vtxp->fileline();
Optimize DFG De Morgan patterns (#6090) It's ok if the replaced vertex has multiple sinks, this pattern cannot increase the size of logic even then.
2025-06-15 17:00:11 +02:00
// Bubble pushing (De Morgan)
if (!lhsp->hasMultipleSinks() && !rhsp->hasMultipleSinks()) {
Try to ensure DFG peephole patterns don't grow the graph Some optimizations are only a net win if they help us remove a graph node (or at least ensure they don't grow the graph), or yields otherwise special logic, so try to apply them only in these cases.
2022-10-04 19:26:53 +02:00
if (DfgNot* const lhsNotp = lhsp->cast<DfgNot>()) {
if (DfgNot* const rhsNotp = rhsp->cast<DfgNot>()) {
APPLYING(REPLACE_AND_OF_NOT_AND_NOT) {
Avoid creating redundant vertices in V3DfgPeephole (#4944) Add a new data-structure V3DfgCache, which can be used to retrieve existing vertices with some given inputs vertices. Use this in V3DfgPeephole to eliminate the creation of redundant vertices. Overall this is performance neutral, but is in prep for some future work.
2024-03-06 19:01:52 +01:00
DfgOr* const orp = make<DfgOr>(vtxp, lhsNotp->srcp(), rhsNotp->srcp());
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(make<DfgNot>(vtxp, orp));
Try to ensure DFG peephole patterns don't grow the graph Some optimizations are only a net win if they help us remove a graph node (or at least ensure they don't grow the graph), or yields otherwise special logic, so try to apply them only in these cases.
2022-10-04 19:26:53 +02:00
return;
}
DFG: Add more peephole patterns
2022-09-30 17:19:53 +02:00
}
Try to ensure DFG peephole patterns don't grow the graph Some optimizations are only a net win if they help us remove a graph node (or at least ensure they don't grow the graph), or yields otherwise special logic, so try to apply them only in these cases.
2022-10-04 19:26:53 +02:00
if (DfgNeq* const rhsNeqp = rhsp->cast<DfgNeq>()) {
APPLYING(REPLACE_AND_OF_NOT_AND_NEQ) {
Avoid creating redundant vertices in V3DfgPeephole (#4944) Add a new data-structure V3DfgCache, which can be used to retrieve existing vertices with some given inputs vertices. Use this in V3DfgPeephole to eliminate the creation of redundant vertices. Overall this is performance neutral, but is in prep for some future work.
2024-03-06 19:01:52 +01:00
DfgEq* const newRhsp = make<DfgEq>(rhsp, rhsNeqp->lhsp(), rhsNeqp->rhsp());
DfgOr* const orp = make<DfgOr>(vtxp, lhsNotp->srcp(), newRhsp);
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(make<DfgNot>(vtxp, orp));
Try to ensure DFG peephole patterns don't grow the graph Some optimizations are only a net win if they help us remove a graph node (or at least ensure they don't grow the graph), or yields otherwise special logic, so try to apply them only in these cases.
2022-10-04 19:26:53 +02:00
return;
}
DFG: Add more peephole patterns
2022-09-30 17:19:53 +02:00
}
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
}
}
if (DfgConst* const lhsConstp = lhsp->cast<DfgConst>()) {
if (lhsConstp->isZero()) {
APPLYING(REPLACE_AND_WITH_ZERO) {
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(lhsConstp);
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
return;
}
}
if (lhsConstp->isOnes()) {
APPLYING(REMOVE_AND_WITH_ONES) {
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(rhsp);
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
return;
}
}
if (DfgConcat* const rhsConcatp = rhsp->cast<DfgConcat>()) {
if (tryPushBitwiseOpThroughConcat(vtxp, lhsConstp, rhsConcatp)) return;
}
}
Add DfgPeephole patterns (#6149)
2025-07-07 17:25:29 +02:00
if (distributiveAndAssociativeBinary<DfgOr, DfgAnd>(vtxp)) return;
DFG: Add more peephole patterns
2022-09-30 17:19:53 +02:00
if (tryPushBitwiseOpThroughReductions(vtxp)) return;
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
if (DfgNot* const lhsNotp = lhsp->cast<DfgNot>()) {
// ~A & A is all zeroes
if (lhsNotp->srcp() == rhsp) {
APPLYING(REPLACE_CONTRADICTORY_AND) {
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(makeZero(flp, vtxp->width()));
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
return;
}
}
Add DfgPeephole patterns (#6149)
2025-07-07 17:25:29 +02:00
// ~A & (A & _) or ~A & (_ & A) is all zeroes
if (DfgAnd* const rhsAndp = rhsp->cast<DfgAnd>()) {
if (lhsNotp->srcp() == rhsAndp->lhsp() || lhsNotp->srcp() == rhsAndp->rhsp()) {
APPLYING(REPLACE_CONTRADICTORY_AND_3) {
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(makeZero(flp, vtxp->width()));
Add DfgPeephole patterns (#6149)
2025-07-07 17:25:29 +02:00
return;
}
}
}
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
}
}
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
void visit(DfgOr* const vtxp) override {
Add DfgPeephole patterns (#6149)
2025-07-07 17:25:29 +02:00
DfgVertex* const lhsp = vtxp->lhsp();
DfgVertex* const rhsp = vtxp->rhsp();
Improve Dfg type system (#6390) Added a mini type system for Dfg using DfgDataType to replace Dfg's use of AstNodeDType. This is much more restricted and represents only the types Dfg can handle in a canonical form. This will be needed when adding more support for unpacked arrays and maybe unpacked structs one day. Also added an internal type checker for DfgGraphs which encodes all the assumptions the code makes about type relationships in the graph. Run this in a few places with --debug-check. Fix resulting fallout.
2025-09-07 21:38:50 +02:00
if (isSame(lhsp, rhsp)) {
Add DfgPeephole patterns (#6149)
2025-07-07 17:25:29 +02:00
APPLYING(REMOVE_OR_WITH_SELF) {
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(lhsp);
Add DfgPeephole patterns (#6149)
2025-07-07 17:25:29 +02:00
return;
}
}
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
DfgPeephole: Rework folding of associative operations Allow constant folding through adjacent nodes of all associative operations, for example '((a & 2) & 3)' or '(3 & (2 & a))' can now be folded into '(a & 2)' and '(2 & a)' respectively. Also improve speed of making associative expression trees right leaning by using rotation of the existing vertices whenever instead of allocation of new nodes.
2022-10-05 11:36:28 +02:00
if (associativeBinary(vtxp)) return;
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
Avoid creating redundant vertices in V3DfgPeephole (#4944) Add a new data-structure V3DfgCache, which can be used to retrieve existing vertices with some given inputs vertices. Use this in V3DfgPeephole to eliminate the creation of redundant vertices. Overall this is performance neutral, but is in prep for some future work.
2024-03-06 19:01:52 +01:00
if (commutativeBinary(vtxp)) return;
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
FileLine* const flp = vtxp->fileline();
Optimize DFG De Morgan patterns (#6090) It's ok if the replaced vertex has multiple sinks, this pattern cannot increase the size of logic even then.
2025-06-15 17:00:11 +02:00
// Bubble pushing (De Morgan)
if (!lhsp->hasMultipleSinks() && !rhsp->hasMultipleSinks()) {
Try to ensure DFG peephole patterns don't grow the graph Some optimizations are only a net win if they help us remove a graph node (or at least ensure they don't grow the graph), or yields otherwise special logic, so try to apply them only in these cases.
2022-10-04 19:26:53 +02:00
if (DfgNot* const lhsNotp = lhsp->cast<DfgNot>()) {
if (DfgNot* const rhsNotp = rhsp->cast<DfgNot>()) {
APPLYING(REPLACE_OR_OF_NOT_AND_NOT) {
Avoid creating redundant vertices in V3DfgPeephole (#4944) Add a new data-structure V3DfgCache, which can be used to retrieve existing vertices with some given inputs vertices. Use this in V3DfgPeephole to eliminate the creation of redundant vertices. Overall this is performance neutral, but is in prep for some future work.
2024-03-06 19:01:52 +01:00
DfgAnd* const andp = make<DfgAnd>(vtxp, lhsNotp->srcp(), rhsNotp->srcp());
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(make<DfgNot>(vtxp, andp));
Try to ensure DFG peephole patterns don't grow the graph Some optimizations are only a net win if they help us remove a graph node (or at least ensure they don't grow the graph), or yields otherwise special logic, so try to apply them only in these cases.
2022-10-04 19:26:53 +02:00
return;
}
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
}
Try to ensure DFG peephole patterns don't grow the graph Some optimizations are only a net win if they help us remove a graph node (or at least ensure they don't grow the graph), or yields otherwise special logic, so try to apply them only in these cases.
2022-10-04 19:26:53 +02:00
if (DfgNeq* const rhsNeqp = rhsp->cast<DfgNeq>()) {
APPLYING(REPLACE_OR_OF_NOT_AND_NEQ) {
Avoid creating redundant vertices in V3DfgPeephole (#4944) Add a new data-structure V3DfgCache, which can be used to retrieve existing vertices with some given inputs vertices. Use this in V3DfgPeephole to eliminate the creation of redundant vertices. Overall this is performance neutral, but is in prep for some future work.
2024-03-06 19:01:52 +01:00
DfgEq* const newRhsp = make<DfgEq>(rhsp, rhsNeqp->lhsp(), rhsNeqp->rhsp());
DfgAnd* const andp = make<DfgAnd>(vtxp, lhsNotp->srcp(), newRhsp);
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(make<DfgNot>(vtxp, andp));
Try to ensure DFG peephole patterns don't grow the graph Some optimizations are only a net win if they help us remove a graph node (or at least ensure they don't grow the graph), or yields otherwise special logic, so try to apply them only in these cases.
2022-10-04 19:26:53 +02:00
return;
}
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
}
}
}
if (DfgConcat* const lhsConcatp = lhsp->cast<DfgConcat>()) {
if (DfgConcat* const rhsConcatp = rhsp->cast<DfgConcat>()) {
Improve Dfg type system (#6390) Added a mini type system for Dfg using DfgDataType to replace Dfg's use of AstNodeDType. This is much more restricted and represents only the types Dfg can handle in a canonical form. This will be needed when adding more support for unpacked arrays and maybe unpacked structs one day. Also added an internal type checker for DfgGraphs which encodes all the assumptions the code makes about type relationships in the graph. Run this in a few places with --debug-check. Fix resulting fallout.
2025-09-07 21:38:50 +02:00
if (lhsConcatp->lhsp()->dtype() == rhsConcatp->lhsp()->dtype()) {
Internals: Improve DFG implementation details (#6355) Large scale refactoring to simplify some of the more obtuse internals of DFG. Remove multiple redundant internal APIs, simplify representation of variables, fix potential unsoundness in circular decomposition. No functional change intended.
2025-09-02 17:50:40 +02:00
if (isZero(lhsConcatp->lhsp()) && isZero(rhsConcatp->rhsp())) {
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
APPLYING(REPLACE_OR_OF_CONCAT_ZERO_LHS_AND_CONCAT_RHS_ZERO) {
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(make<DfgConcat>(vtxp, rhsConcatp->lhsp(), lhsConcatp->rhsp()));
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
return;
}
}
Internals: Improve DFG implementation details (#6355) Large scale refactoring to simplify some of the more obtuse internals of DFG. Remove multiple redundant internal APIs, simplify representation of variables, fix potential unsoundness in circular decomposition. No functional change intended.
2025-09-02 17:50:40 +02:00
if (isZero(lhsConcatp->rhsp()) && isZero(rhsConcatp->lhsp())) {
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
APPLYING(REPLACE_OR_OF_CONCAT_LHS_ZERO_AND_CONCAT_ZERO_RHS) {
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(make<DfgConcat>(vtxp, lhsConcatp->lhsp(), rhsConcatp->rhsp()));
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
return;
}
}
}
}
}
if (DfgConst* const lhsConstp = lhsp->cast<DfgConst>()) {
if (lhsConstp->isZero()) {
APPLYING(REMOVE_OR_WITH_ZERO) {
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(rhsp);
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
return;
}
}
if (lhsConstp->isOnes()) {
APPLYING(REPLACE_OR_WITH_ONES) {
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(lhsp);
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
return;
}
}
if (DfgConcat* const rhsConcatp = rhsp->cast<DfgConcat>()) {
if (tryPushBitwiseOpThroughConcat(vtxp, lhsConstp, rhsConcatp)) return;
}
}
Add DfgPeephole patterns (#6149)
2025-07-07 17:25:29 +02:00
if (distributiveAndAssociativeBinary<DfgAnd, DfgOr>(vtxp)) return;
DFG: Add more peephole patterns
2022-09-30 17:19:53 +02:00
if (tryPushBitwiseOpThroughReductions(vtxp)) return;
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
if (DfgNot* const lhsNotp = lhsp->cast<DfgNot>()) {
// ~A | A is all ones
if (lhsNotp->srcp() == rhsp) {
APPLYING(REPLACE_TAUTOLOGICAL_OR) {
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
DfgConst* const resp = makeZero(flp, vtxp->width());
resp->num().setAllBits1();
replace(resp);
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
return;
}
}
Add DfgPeephole patterns (#6149)
2025-07-07 17:25:29 +02:00
// ~A | (A | _) or ~A | (_ | A) is all ones
if (DfgOr* const rhsOrp = rhsp->cast<DfgOr>()) {
if (lhsNotp->srcp() == rhsOrp->lhsp() || lhsNotp->srcp() == rhsOrp->rhsp()) {
APPLYING(REPLACE_TAUTOLOGICAL_OR_3) {
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
DfgConst* const resp = makeZero(flp, vtxp->width());
resp->num().setAllBits1();
replace(resp);
Add DfgPeephole patterns (#6149)
2025-07-07 17:25:29 +02:00
return;
}
}
}
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
}
}
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
void visit(DfgXor* const vtxp) override {
Add DfgPeephole patterns (#6149)
2025-07-07 17:25:29 +02:00
DfgVertex* const lhsp = vtxp->lhsp();
DfgVertex* const rhsp = vtxp->rhsp();
Improve Dfg type system (#6390) Added a mini type system for Dfg using DfgDataType to replace Dfg's use of AstNodeDType. This is much more restricted and represents only the types Dfg can handle in a canonical form. This will be needed when adding more support for unpacked arrays and maybe unpacked structs one day. Also added an internal type checker for DfgGraphs which encodes all the assumptions the code makes about type relationships in the graph. Run this in a few places with --debug-check. Fix resulting fallout.
2025-09-07 21:38:50 +02:00
if (isSame(lhsp, rhsp)) {
Add DfgPeephole patterns (#6149)
2025-07-07 17:25:29 +02:00
APPLYING(REPLACE_XOR_WITH_SELF) {
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(makeZero(vtxp->fileline(), vtxp->width()));
Add DfgPeephole patterns (#6149)
2025-07-07 17:25:29 +02:00
return;
}
}
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
DfgPeephole: Rework folding of associative operations Allow constant folding through adjacent nodes of all associative operations, for example '((a & 2) & 3)' or '(3 & (2 & a))' can now be folded into '(a & 2)' and '(2 & a)' respectively. Also improve speed of making associative expression trees right leaning by using rotation of the existing vertices whenever instead of allocation of new nodes.
2022-10-05 11:36:28 +02:00
if (associativeBinary(vtxp)) return;
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
Avoid creating redundant vertices in V3DfgPeephole (#4944) Add a new data-structure V3DfgCache, which can be used to retrieve existing vertices with some given inputs vertices. Use this in V3DfgPeephole to eliminate the creation of redundant vertices. Overall this is performance neutral, but is in prep for some future work.
2024-03-06 19:01:52 +01:00
if (commutativeBinary(vtxp)) return;
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
DFG: Add more peephole patterns
2022-09-30 17:19:53 +02:00
if (DfgConst* const lConstp = lhsp->cast<DfgConst>()) {
if (lConstp->isZero()) {
APPLYING(REMOVE_XOR_WITH_ZERO) {
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(rhsp);
DFG: Add more peephole patterns
2022-09-30 17:19:53 +02:00
return;
}
}
if (lConstp->isOnes()) {
APPLYING(REPLACE_XOR_WITH_ONES) {
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(make<DfgNot>(vtxp, rhsp));
DFG: Add more peephole patterns
2022-09-30 17:19:53 +02:00
return;
}
}
if (DfgConcat* const rConcatp = rhsp->cast<DfgConcat>()) {
Avoid creating redundant vertices in V3DfgPeephole (#4944) Add a new data-structure V3DfgCache, which can be used to retrieve existing vertices with some given inputs vertices. Use this in V3DfgPeephole to eliminate the creation of redundant vertices. Overall this is performance neutral, but is in prep for some future work.
2024-03-06 19:01:52 +01:00
if (tryPushBitwiseOpThroughConcat(vtxp, lConstp, rConcatp)) return;
DFG: Add more peephole patterns
2022-09-30 17:19:53 +02:00
return;
}
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
}
DFG: Add more peephole patterns
2022-09-30 17:19:53 +02:00
if (tryPushBitwiseOpThroughReductions(vtxp)) return;
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
}
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
//=========================================================================
// DfgVertexBinary - other
//=========================================================================
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
void visit(DfgAdd* const vtxp) override {
DfgPeephole: Rework folding of associative operations Allow constant folding through adjacent nodes of all associative operations, for example '((a & 2) & 3)' or '(3 & (2 & a))' can now be folded into '(a & 2)' and '(2 & a)' respectively. Also improve speed of making associative expression trees right leaning by using rotation of the existing vertices whenever instead of allocation of new nodes.
2022-10-05 11:36:28 +02:00
if (associativeBinary(vtxp)) return;
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
Avoid creating redundant vertices in V3DfgPeephole (#4944) Add a new data-structure V3DfgCache, which can be used to retrieve existing vertices with some given inputs vertices. Use this in V3DfgPeephole to eliminate the creation of redundant vertices. Overall this is performance neutral, but is in prep for some future work.
2024-03-06 19:01:52 +01:00
if (commutativeBinary(vtxp)) return;
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
}
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
void visit(DfgArraySel* const vtxp) override {
Optimize complex combinational logic in DFG (#6298) This patch adds DfgLogic, which is a vertex that represents a whole, arbitrarily complex combinational AstAlways or AstAssignW in the DfgGraph. Implementing this requires computing the variables live at entry to the AstAlways (variables read by the block), so there is a new ControlFlowGraph data structure and a classical data-flow analysis based live variable analysis to do that at the variable level (as opposed to bit/element level). The actual CFG construction and live variable analysis is best effort, and might fail for currently unhandled constructs or data types. This can be extended later. V3DfgAstToDfg is changed to convert the Ast into an initial DfgGraph containing only DfgLogic, DfgVertexSplice and DfgVertexVar vertices. The DfgLogic are then subsequently synthesized into primitive operations by the new V3DfgSynthesize pass, which is a combination of the old V3DfgAstToDfg conversion and new code to handle AstAlways blocks with complex flow control. V3DfgSynthesize by default will synthesize roughly the same constructs as V3DfgAstToDfg used to handle before, plus any logic that is part of a combinational cycle within the DfgGraph. This enables breaking up these cycles, for which there are extensions to V3DfgBreakCycles in this patch as well. V3DfgSynthesize will then delete all non synthesized or non synthesizable DfgLogic vertices and the rest of the Dfg pipeline is identical, with minor changes to adjust for the changed representation. Because with this change we can now eliminate many more UNOPTFLAT, DFG has been disabled in all the tests that specifically target testing the scheduling and reporting of circular combinational logic.
2025-08-19 16:06:38 +02:00
DfgConst* const idxp = vtxp->bitp()->cast<DfgConst>();
if (!idxp) return;
DfgVarArray* const varp = vtxp->fromp()->cast<DfgVarArray>();
if (!varp) return;
if (varp->varp()->isForced()) return;
Fix corner case bugs in module and variable inlining (#6322) There were a couple corner case bugs in V3Inline, and one in Dfg when dealing with inlining of modules/variables. V3Inline: - Invalid code generated when inlining an input that also had an assignment to it (Throws an ASSIGNIN, but this is sometimes reasonable to do, e.g. hiererchical reference to an unonnected input port) - Inlining (aliasing) publicly writeable input port. - Inlining forcable port connected to constant. Dfg: - Inining publicly writeable variables The tests that cover these are the same and fixing one will trigger the other bug, so fixing them all in one go. Also cleanup V3Inline to be less out of order and rely less on unique APIs only used by V3Inine (will remove those in follow up patch). Small step towards #6280.
2025-08-22 22:43:49 +02:00
if (varp->varp()->isSigUserRWPublic()) return;
Optimize complex combinational logic in DFG (#6298) This patch adds DfgLogic, which is a vertex that represents a whole, arbitrarily complex combinational AstAlways or AstAssignW in the DfgGraph. Implementing this requires computing the variables live at entry to the AstAlways (variables read by the block), so there is a new ControlFlowGraph data structure and a classical data-flow analysis based live variable analysis to do that at the variable level (as opposed to bit/element level). The actual CFG construction and live variable analysis is best effort, and might fail for currently unhandled constructs or data types. This can be extended later. V3DfgAstToDfg is changed to convert the Ast into an initial DfgGraph containing only DfgLogic, DfgVertexSplice and DfgVertexVar vertices. The DfgLogic are then subsequently synthesized into primitive operations by the new V3DfgSynthesize pass, which is a combination of the old V3DfgAstToDfg conversion and new code to handle AstAlways blocks with complex flow control. V3DfgSynthesize by default will synthesize roughly the same constructs as V3DfgAstToDfg used to handle before, plus any logic that is part of a combinational cycle within the DfgGraph. This enables breaking up these cycles, for which there are extensions to V3DfgBreakCycles in this patch as well. V3DfgSynthesize will then delete all non synthesized or non synthesizable DfgLogic vertices and the rest of the Dfg pipeline is identical, with minor changes to adjust for the changed representation. Because with this change we can now eliminate many more UNOPTFLAT, DFG has been disabled in all the tests that specifically target testing the scheduling and reporting of circular combinational logic.
2025-08-19 16:06:38 +02:00
DfgVertex* const srcp = varp->srcp();
if (!srcp) return;
if (DfgSpliceArray* const splicep = srcp->cast<DfgSpliceArray>()) {
DfgVertex* const driverp = splicep->driverAt(idxp->toSizeT());
if (!driverp) return;
DfgUnitArray* const uap = driverp->cast<DfgUnitArray>();
if (!uap) return;
if (uap->srcp()->is<DfgVertexSplice>()) return;
// If driven by a variable that had a Driver in DFG, it is partial
if (DfgVertexVar* const dvarp = uap->srcp()->cast<DfgVertexVar>()) {
if (dvarp->srcp()) return;
}
APPLYING(INLINE_ARRAYSEL_SPLICE) {
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(uap->srcp());
Optimize complex combinational logic in DFG (#6298) This patch adds DfgLogic, which is a vertex that represents a whole, arbitrarily complex combinational AstAlways or AstAssignW in the DfgGraph. Implementing this requires computing the variables live at entry to the AstAlways (variables read by the block), so there is a new ControlFlowGraph data structure and a classical data-flow analysis based live variable analysis to do that at the variable level (as opposed to bit/element level). The actual CFG construction and live variable analysis is best effort, and might fail for currently unhandled constructs or data types. This can be extended later. V3DfgAstToDfg is changed to convert the Ast into an initial DfgGraph containing only DfgLogic, DfgVertexSplice and DfgVertexVar vertices. The DfgLogic are then subsequently synthesized into primitive operations by the new V3DfgSynthesize pass, which is a combination of the old V3DfgAstToDfg conversion and new code to handle AstAlways blocks with complex flow control. V3DfgSynthesize by default will synthesize roughly the same constructs as V3DfgAstToDfg used to handle before, plus any logic that is part of a combinational cycle within the DfgGraph. This enables breaking up these cycles, for which there are extensions to V3DfgBreakCycles in this patch as well. V3DfgSynthesize will then delete all non synthesized or non synthesizable DfgLogic vertices and the rest of the Dfg pipeline is identical, with minor changes to adjust for the changed representation. Because with this change we can now eliminate many more UNOPTFLAT, DFG has been disabled in all the tests that specifically target testing the scheduling and reporting of circular combinational logic.
2025-08-19 16:06:38 +02:00
return;
}
}
if (DfgUnitArray* const uap = srcp->cast<DfgUnitArray>()) {
UASSERT_OBJ(idxp->toSizeT() == 0, vtxp, "Array index out of range");
if (uap->srcp()->is<DfgSplicePacked>()) return;
// If driven by a variable that had a Driver in DFG, it is partial
if (DfgVertexVar* const dvarp = uap->srcp()->cast<DfgVertexVar>()) {
if (dvarp->srcp()) return;
}
APPLYING(INLINE_ARRAYSEL_UNIT) {
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(uap->srcp());
Optimize complex combinational logic in DFG (#6298) This patch adds DfgLogic, which is a vertex that represents a whole, arbitrarily complex combinational AstAlways or AstAssignW in the DfgGraph. Implementing this requires computing the variables live at entry to the AstAlways (variables read by the block), so there is a new ControlFlowGraph data structure and a classical data-flow analysis based live variable analysis to do that at the variable level (as opposed to bit/element level). The actual CFG construction and live variable analysis is best effort, and might fail for currently unhandled constructs or data types. This can be extended later. V3DfgAstToDfg is changed to convert the Ast into an initial DfgGraph containing only DfgLogic, DfgVertexSplice and DfgVertexVar vertices. The DfgLogic are then subsequently synthesized into primitive operations by the new V3DfgSynthesize pass, which is a combination of the old V3DfgAstToDfg conversion and new code to handle AstAlways blocks with complex flow control. V3DfgSynthesize by default will synthesize roughly the same constructs as V3DfgAstToDfg used to handle before, plus any logic that is part of a combinational cycle within the DfgGraph. This enables breaking up these cycles, for which there are extensions to V3DfgBreakCycles in this patch as well. V3DfgSynthesize will then delete all non synthesized or non synthesizable DfgLogic vertices and the rest of the Dfg pipeline is identical, with minor changes to adjust for the changed representation. Because with this change we can now eliminate many more UNOPTFLAT, DFG has been disabled in all the tests that specifically target testing the scheduling and reporting of circular combinational logic.
2025-08-19 16:06:38 +02:00
return;
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
}
}
}
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
void visit(DfgConcat* const vtxp) override {
Speed up DfgPeephole in various ways
2022-10-06 13:02:46 +02:00
if (associativeBinary(vtxp)) return;
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
DfgVertex* const lhsp = vtxp->lhsp();
DfgVertex* const rhsp = vtxp->rhsp();
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
FileLine* const flp = vtxp->fileline();
Internals: Improve DFG implementation details (#6355) Large scale refactoring to simplify some of the more obtuse internals of DFG. Remove multiple redundant internal APIs, simplify representation of variables, fix potential unsoundness in circular decomposition. No functional change intended.
2025-09-02 17:50:40 +02:00
if (isZero(lhsp)) {
DfgPeephole: Rework folding of associative operations Allow constant folding through adjacent nodes of all associative operations, for example '((a & 2) & 3)' or '(3 & (2 & a))' can now be folded into '(a & 2)' and '(2 & a)' respectively. Also improve speed of making associative expression trees right leaning by using rotation of the existing vertices whenever instead of allocation of new nodes.
2022-10-05 11:36:28 +02:00
DfgConst* const lConstp = lhsp->as<DfgConst>();
if (DfgSel* const rSelp = rhsp->cast<DfgSel>()) {
Improve Dfg type system (#6390) Added a mini type system for Dfg using DfgDataType to replace Dfg's use of AstNodeDType. This is much more restricted and represents only the types Dfg can handle in a canonical form. This will be needed when adding more support for unpacked arrays and maybe unpacked structs one day. Also added an internal type checker for DfgGraphs which encodes all the assumptions the code makes about type relationships in the graph. Run this in a few places with --debug-check. Fix resulting fallout.
2025-09-07 21:38:50 +02:00
if (vtxp->dtype() == rSelp->fromp()->dtype() && rSelp->lsb() == lConstp->width()) {
DFG: Special case representation of AstSel AstSel is a ternary node, but the 'widthp' is always constant and is hence redundant, and 'lsbp' is very often constant. As AstSel is fairly common, we special case as a DfgSel for the constant 'lsbp', and as 'DfgMux` for the non-constant 'lsbp'.
2022-10-06 19:34:18 +02:00
APPLYING(REPLACE_CONCAT_ZERO_AND_SEL_TOP_WITH_SHIFTR) {
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(
make<DfgShiftR>(vtxp, rSelp->fromp(), makeI32(flp, lConstp->width())));
DFG: Special case representation of AstSel AstSel is a ternary node, but the 'widthp' is always constant and is hence redundant, and 'lsbp' is very often constant. As AstSel is fairly common, we special case as a DfgSel for the constant 'lsbp', and as 'DfgMux` for the non-constant 'lsbp'.
2022-10-06 19:34:18 +02:00
return;
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
}
}
}
DfgPeephole: Rework folding of associative operations Allow constant folding through adjacent nodes of all associative operations, for example '((a & 2) & 3)' or '(3 & (2 & a))' can now be folded into '(a & 2)' and '(2 & a)' respectively. Also improve speed of making associative expression trees right leaning by using rotation of the existing vertices whenever instead of allocation of new nodes.
2022-10-05 11:36:28 +02:00
}
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
Internals: Improve DFG implementation details (#6355) Large scale refactoring to simplify some of the more obtuse internals of DFG. Remove multiple redundant internal APIs, simplify representation of variables, fix potential unsoundness in circular decomposition. No functional change intended.
2025-09-02 17:50:40 +02:00
if (isZero(rhsp)) {
DfgPeephole: Rework folding of associative operations Allow constant folding through adjacent nodes of all associative operations, for example '((a & 2) & 3)' or '(3 & (2 & a))' can now be folded into '(a & 2)' and '(2 & a)' respectively. Also improve speed of making associative expression trees right leaning by using rotation of the existing vertices whenever instead of allocation of new nodes.
2022-10-05 11:36:28 +02:00
DfgConst* const rConstp = rhsp->as<DfgConst>();
if (DfgSel* const lSelp = lhsp->cast<DfgSel>()) {
Improve Dfg type system (#6390) Added a mini type system for Dfg using DfgDataType to replace Dfg's use of AstNodeDType. This is much more restricted and represents only the types Dfg can handle in a canonical form. This will be needed when adding more support for unpacked arrays and maybe unpacked structs one day. Also added an internal type checker for DfgGraphs which encodes all the assumptions the code makes about type relationships in the graph. Run this in a few places with --debug-check. Fix resulting fallout.
2025-09-07 21:38:50 +02:00
if (vtxp->dtype() == lSelp->fromp()->dtype() && lSelp->lsb() == 0) {
DFG: Special case representation of AstSel AstSel is a ternary node, but the 'widthp' is always constant and is hence redundant, and 'lsbp' is very often constant. As AstSel is fairly common, we special case as a DfgSel for the constant 'lsbp', and as 'DfgMux` for the non-constant 'lsbp'.
2022-10-06 19:34:18 +02:00
APPLYING(REPLACE_CONCAT_SEL_BOTTOM_AND_ZERO_WITH_SHIFTL) {
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(
make<DfgShiftL>(vtxp, lSelp->fromp(), makeI32(flp, rConstp->width())));
DFG: Special case representation of AstSel AstSel is a ternary node, but the 'widthp' is always constant and is hence redundant, and 'lsbp' is very often constant. As AstSel is fairly common, we special case as a DfgSel for the constant 'lsbp', and as 'DfgMux` for the non-constant 'lsbp'.
2022-10-06 19:34:18 +02:00
return;
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
}
}
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
}
}
DfgPeephole: Rework folding of associative operations Allow constant folding through adjacent nodes of all associative operations, for example '((a & 2) & 3)' or '(3 & (2 & a))' can now be folded into '(a & 2)' and '(2 & a)' respectively. Also improve speed of making associative expression trees right leaning by using rotation of the existing vertices whenever instead of allocation of new nodes.
2022-10-05 11:36:28 +02:00
if (DfgNot* const lNot = lhsp->cast<DfgNot>()) {
if (DfgNot* const rNot = rhsp->cast<DfgNot>()) {
if (!lNot->hasMultipleSinks() && !rNot->hasMultipleSinks()) {
APPLYING(PUSH_CONCAT_THROUGH_NOTS) {
Avoid creating redundant vertices in V3DfgPeephole (#4944) Add a new data-structure V3DfgCache, which can be used to retrieve existing vertices with some given inputs vertices. Use this in V3DfgPeephole to eliminate the creation of redundant vertices. Overall this is performance neutral, but is in prep for some future work.
2024-03-06 19:01:52 +01:00
DfgConcat* const newCatp
= make<DfgConcat>(vtxp, lNot->srcp(), rNot->srcp());
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(make<DfgNot>(vtxp, newCatp));
DfgPeephole: Rework folding of associative operations Allow constant folding through adjacent nodes of all associative operations, for example '((a & 2) & 3)' or '(3 & (2 & a))' can now be folded into '(a & 2)' and '(2 & a)' respectively. Also improve speed of making associative expression trees right leaning by using rotation of the existing vertices whenever instead of allocation of new nodes.
2022-10-05 11:36:28 +02:00
return;
}
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
}
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
}
}
Internals: Remove unused vertices immediately in DfgPeephole
2026-03-23 16:06:07 +01:00
if (DfgSel* const lSelp = lhsp->cast<DfgSel>()) {
if (DfgSel* const rSelp = rhsp->cast<DfgSel>()) {
Internals: Remove use of DfgVetex::equals from V3DfgPeephole This is both unnecessary and slower than the existing `isSame`.
2025-09-03 11:26:47 +02:00
if (isSame(lSelp->fromp(), rSelp->fromp())) {
DFG: Special case representation of AstSel AstSel is a ternary node, but the 'widthp' is always constant and is hence redundant, and 'lsbp' is very often constant. As AstSel is fairly common, we special case as a DfgSel for the constant 'lsbp', and as 'DfgMux` for the non-constant 'lsbp'.
2022-10-06 19:34:18 +02:00
if (lSelp->lsb() == rSelp->lsb() + rSelp->width()) {
Internals: Remove unused vertices immediately in DfgPeephole
2026-03-23 16:06:07 +01:00
APPLYING(REMOVE_CONCAT_OF_ADJOINING_SELS) {
replace(make<DfgSel>(vtxp, rSelp->fromp(), rSelp->lsb()));
return;
}
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
}
}
}
Internals: Remove unused vertices immediately in DfgPeephole
2026-03-23 16:06:07 +01:00
if (DfgConcat* const rConcatp = rhsp->cast<DfgConcat>()) {
if (DfgSel* const rlSelp = rConcatp->lhsp()->cast<DfgSel>()) {
if (isSame(lSelp->fromp(), rlSelp->fromp())) {
if (lSelp->lsb() == rlSelp->lsb() + rlSelp->width()) {
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
APPLYING(REPLACE_NESTED_CONCAT_OF_ADJOINING_SELS_ON_LHS) {
Internals: Remove unused vertices immediately in DfgPeephole
2026-03-23 16:06:07 +01:00
const uint32_t width = lSelp->width() + rlSelp->width();
const DfgDataType& dtype = DfgDataType::packed(width);
DfgSel* const selp
= make<DfgSel>(flp, dtype, rlSelp->fromp(), rlSelp->lsb());
replace(make<DfgConcat>(vtxp, selp, rConcatp->rhsp()));
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
return;
}
}
}
}
}
Internals: Remove unused vertices immediately in DfgPeephole
2026-03-23 16:06:07 +01:00
}
if (DfgSel* const rSelp = rhsp->cast<DfgSel>()) {
if (DfgConcat* const lConcatp = lhsp->cast<DfgConcat>()) {
if (DfgSel* const lrSelp = lConcatp->rhsp()->cast<DfgSel>()) {
if (isSame(lrSelp->fromp(), rSelp->fromp())) {
if (lrSelp->lsb() == rSelp->lsb() + rSelp->width()) {
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
APPLYING(REPLACE_NESTED_CONCAT_OF_ADJOINING_SELS_ON_RHS) {
Internals: Remove unused vertices immediately in DfgPeephole
2026-03-23 16:06:07 +01:00
const uint32_t width = lrSelp->width() + rSelp->width();
const DfgDataType& dtype = DfgDataType::packed(width);
DfgSel* const selp
= make<DfgSel>(flp, dtype, rSelp->fromp(), rSelp->lsb());
replace(make<DfgConcat>(vtxp, lConcatp->lhsp(), selp));
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
return;
}
}
}
}
}
}
Optimize concatenations that produce unused bits in DFG (#6971) Concatenations that are only used by Sel expressions that do not consume some bits on the edges can be narrowed to not compute the unused bits. E.g.: `{a[4:0], b[4:0]}[5:4]` -> `{a[0], b[4]}[1:0]` This is a superset or the PUSH_SEL_THROUGH_CONCAT DFG pattern, which is removed.
2026-01-31 18:28:14 +01:00
Optimize additional DFG peephole Shift and Concat patterns (#7077)
2026-02-14 13:25:17 +01:00
if (DfgConst* const lConstp = lhsp->cast<DfgConst>()) {
if (DfgCond* const rCondp = rhsp->cast<DfgCond>()) {
if (!rCondp->hasMultipleSinks()) {
DfgVertex* const rtVtxp = rCondp->thenp();
DfgVertex* const reVtxp = rCondp->elsep();
APPLYING(PUSH_CONCAT_THROUGH_COND_LHS) {
DfgConcat* const thenp
= make<DfgConcat>(rtVtxp->fileline(), vtxp->dtype(), lConstp, rtVtxp);
DfgConcat* const elsep
= make<DfgConcat>(reVtxp->fileline(), vtxp->dtype(), lConstp, reVtxp);
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(make<DfgCond>(vtxp, rCondp->condp(), thenp, elsep));
Optimize additional DFG peephole Shift and Concat patterns (#7077)
2026-02-14 13:25:17 +01:00
return;
}
}
}
}
if (DfgConst* const rConstp = rhsp->cast<DfgConst>()) {
if (DfgCond* const lCondp = lhsp->cast<DfgCond>()) {
if (!lCondp->hasMultipleSinks()) {
DfgVertex* const ltVtxp = lCondp->thenp();
DfgVertex* const leVtxp = lCondp->elsep();
APPLYING(PUSH_CONCAT_THROUGH_COND_RHS) {
DfgConcat* const thenp
= make<DfgConcat>(ltVtxp->fileline(), vtxp->dtype(), ltVtxp, rConstp);
DfgConcat* const elsep
= make<DfgConcat>(leVtxp->fileline(), vtxp->dtype(), leVtxp, rConstp);
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(make<DfgCond>(vtxp, lCondp->condp(), thenp, elsep));
Optimize additional DFG peephole Shift and Concat patterns (#7077)
2026-02-14 13:25:17 +01:00
return;
}
}
}
}
Optimize concatenations that produce unused bits in DFG (#6971) Concatenations that are only used by Sel expressions that do not consume some bits on the edges can be narrowed to not compute the unused bits. E.g.: `{a[4:0], b[4:0]}[5:4]` -> `{a[0], b[4]}[1:0]` This is a superset or the PUSH_SEL_THROUGH_CONCAT DFG pattern, which is removed.
2026-01-31 18:28:14 +01:00
// Attempt to narrow a concatenation that produces unused bits on the edges
{
const uint32_t vMsb = vtxp->width() - 1; // MSB of the concatenation
const uint32_t lLsb = vtxp->rhsp()->width(); // LSB of the LHS
const uint32_t rMsb = lLsb - 1; // MSB of the RHS
// Check each sink, and record the range of bits used by them
uint32_t lsb = vMsb; // LSB used by a sink
uint32_t msb = 0; // MSB used by a sink
bool hasCrossSink = false; // True if some sinks use bits from both sides
vtxp->foreachSink([&](DfgVertex& sink) {
// Record bits used by DfgSel sinks
if (const DfgSel* const selp = sink.cast<DfgSel>()) {
const uint32_t selLsb = selp->lsb();
const uint32_t selMsb = selLsb + selp->width() - 1;
lsb = std::min(lsb, selLsb);
msb = std::max(msb, selMsb);
hasCrossSink |= selMsb >= lLsb && rMsb >= selLsb;
return false;
}
Commentary: Fix non-grammar
2026-03-12 00:53:23 +01:00
// Ignore non-observable variable sinks. These will be eliminated.
Optimize concatenations that produce unused bits in DFG (#6971) Concatenations that are only used by Sel expressions that do not consume some bits on the edges can be narrowed to not compute the unused bits. E.g.: `{a[4:0], b[4:0]}[5:4]` -> `{a[0], b[4]}[1:0]` This is a superset or the PUSH_SEL_THROUGH_CONCAT DFG pattern, which is removed.
2026-01-31 18:28:14 +01:00
if (const DfgVarPacked* const varp = sink.cast<DfgVarPacked>()) {
if (!varp->hasSinks() && !varp->isObserved()) return false;
}
// Otherwise the whole value is used
lsb = 0;
msb = vMsb;
return true;
});
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
if (hasCrossSink && (vMsb > msb || lsb > 0)) {
Optimize concatenations that produce unused bits in DFG (#6971) Concatenations that are only used by Sel expressions that do not consume some bits on the edges can be narrowed to not compute the unused bits. E.g.: `{a[4:0], b[4:0]}[5:4]` -> `{a[0], b[4]}[1:0]` This is a superset or the PUSH_SEL_THROUGH_CONCAT DFG pattern, which is removed.
2026-01-31 18:28:14 +01:00
APPLYING(NARROW_CONCAT) {
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
// Narrowed RHS
DfgVertex* nRhsp = rhsp;
if (lsb != 0)
nRhsp = make<DfgSel>(flp, DfgDataType::packed(rMsb - lsb + 1), rhsp, lsb);
// Narrowed LHS
DfgVertex* nLhsp = lhsp;
if (msb != vMsb)
nLhsp = make<DfgSel>(flp, DfgDataType::packed(msb - lLsb + 1), lhsp, 0U);
// Narrowed concatenation
DfgVertex* const catp
= make<DfgConcat>(flp, DfgDataType::packed(msb - lsb + 1), nLhsp, nRhsp);
// Need to insert via a partial splice to avoid infinite matching,
// this splice will be eliminated on later visits to its sinks.
DfgVertex::ScopeCache scopeCache;
DfgSplicePacked* const sp = new DfgSplicePacked{m_dfg, flp, vtxp->dtype()};
sp->addDriver(catp, lsb, flp);
replace(sp);
Optimize concatenations that produce unused bits in DFG (#6971) Concatenations that are only used by Sel expressions that do not consume some bits on the edges can be narrowed to not compute the unused bits. E.g.: `{a[4:0], b[4:0]}[5:4]` -> `{a[0], b[4]}[1:0]` This is a superset or the PUSH_SEL_THROUGH_CONCAT DFG pattern, which is removed.
2026-01-31 18:28:14 +01:00
return;
}
}
}
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
}
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
void visit(DfgDiv* const vtxp) override {
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
if (foldBinary(vtxp)) return;
}
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
void visit(DfgDivS* const vtxp) override {
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
if (foldBinary(vtxp)) return;
}
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
void visit(DfgEq* const vtxp) override {
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
if (foldBinary(vtxp)) return;
Avoid creating redundant vertices in V3DfgPeephole (#4944) Add a new data-structure V3DfgCache, which can be used to retrieve existing vertices with some given inputs vertices. Use this in V3DfgPeephole to eliminate the creation of redundant vertices. Overall this is performance neutral, but is in prep for some future work.
2024-03-06 19:01:52 +01:00
if (commutativeBinary(vtxp)) return;
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
DfgVertex* const lhsp = vtxp->lhsp();
DfgVertex* const rhsp = vtxp->rhsp();
if (DfgConst* const lhsConstp = lhsp->cast<DfgConst>()) {
if (DfgConcat* const rhsConcatp = rhsp->cast<DfgConcat>()) {
if (tryPushCompareOpThroughConcat(vtxp, lhsConstp, rhsConcatp)) return;
}
}
}
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
void visit(DfgGt* const vtxp) override {
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
if (foldBinary(vtxp)) return;
}
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
void visit(DfgGtS* const vtxp) override {
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
if (foldBinary(vtxp)) return;
}
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
void visit(DfgGte* const vtxp) override {
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
if (foldBinary(vtxp)) return;
}
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
void visit(DfgGteS* const vtxp) override {
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
if (foldBinary(vtxp)) return;
}
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
void visit(DfgLogAnd* const vtxp) override {
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
if (foldBinary(vtxp)) return;
Add DfgPeephole patterns (#6149)
2025-07-07 17:25:29 +02:00
DfgVertex* const lhsp = vtxp->lhsp();
DfgVertex* const rhsp = vtxp->rhsp();
if (lhsp->width() == 1 && rhsp->width() == 1) {
APPLYING(REPLACE_LOGAND_WITH_AND) {
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(make<DfgAnd>(vtxp, lhsp, rhsp));
Add DfgPeephole patterns (#6149)
2025-07-07 17:25:29 +02:00
return;
}
}
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
}
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
void visit(DfgLogEq* const vtxp) override {
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
if (foldBinary(vtxp)) return;
}
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
void visit(DfgLogIf* const vtxp) override {
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
if (foldBinary(vtxp)) return;
}
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
void visit(DfgLogOr* const vtxp) override {
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
if (foldBinary(vtxp)) return;
Add DfgPeephole patterns (#6149)
2025-07-07 17:25:29 +02:00
DfgVertex* const lhsp = vtxp->lhsp();
DfgVertex* const rhsp = vtxp->rhsp();
if (lhsp->width() == 1 && rhsp->width() == 1) {
APPLYING(REPLACE_LOGOR_WITH_OR) {
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(make<DfgOr>(vtxp, lhsp, rhsp));
Add DfgPeephole patterns (#6149)
2025-07-07 17:25:29 +02:00
return;
}
}
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
}
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
void visit(DfgLt* const vtxp) override {
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
if (foldBinary(vtxp)) return;
}
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
void visit(DfgLtS* const vtxp) override {
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
if (foldBinary(vtxp)) return;
}
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
void visit(DfgLte* const vtxp) override {
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
if (foldBinary(vtxp)) return;
}
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
void visit(DfgLteS* const vtxp) override {
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
if (foldBinary(vtxp)) return;
}
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
void visit(DfgModDiv* const vtxp) override {
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
if (foldBinary(vtxp)) return;
}
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
void visit(DfgModDivS* const vtxp) override {
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
if (foldBinary(vtxp)) return;
}
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
void visit(DfgMul* const vtxp) override {
DfgPeephole: Rework folding of associative operations Allow constant folding through adjacent nodes of all associative operations, for example '((a & 2) & 3)' or '(3 & (2 & a))' can now be folded into '(a & 2)' and '(2 & a)' respectively. Also improve speed of making associative expression trees right leaning by using rotation of the existing vertices whenever instead of allocation of new nodes.
2022-10-05 11:36:28 +02:00
if (associativeBinary(vtxp)) return;
Avoid creating redundant vertices in V3DfgPeephole (#4944) Add a new data-structure V3DfgCache, which can be used to retrieve existing vertices with some given inputs vertices. Use this in V3DfgPeephole to eliminate the creation of redundant vertices. Overall this is performance neutral, but is in prep for some future work.
2024-03-06 19:01:52 +01:00
if (commutativeBinary(vtxp)) return;
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
}
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
void visit(DfgMulS* const vtxp) override {
DfgPeephole: Rework folding of associative operations Allow constant folding through adjacent nodes of all associative operations, for example '((a & 2) & 3)' or '(3 & (2 & a))' can now be folded into '(a & 2)' and '(2 & a)' respectively. Also improve speed of making associative expression trees right leaning by using rotation of the existing vertices whenever instead of allocation of new nodes.
2022-10-05 11:36:28 +02:00
if (associativeBinary(vtxp)) return;
Avoid creating redundant vertices in V3DfgPeephole (#4944) Add a new data-structure V3DfgCache, which can be used to retrieve existing vertices with some given inputs vertices. Use this in V3DfgPeephole to eliminate the creation of redundant vertices. Overall this is performance neutral, but is in prep for some future work.
2024-03-06 19:01:52 +01:00
if (commutativeBinary(vtxp)) return;
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
}
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
void visit(DfgNeq* const vtxp) override {
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
if (foldBinary(vtxp)) return;
}
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
void visit(DfgPow* const vtxp) override {
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
if (foldBinary(vtxp)) return;
}
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
void visit(DfgPowSS* const vtxp) override {
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
if (foldBinary(vtxp)) return;
}
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
void visit(DfgPowSU* const vtxp) override {
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
if (foldBinary(vtxp)) return;
}
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
void visit(DfgPowUS* const vtxp) override {
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
if (foldBinary(vtxp)) return;
}
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
void visit(DfgReplicate* const vtxp) override {
Improve Dfg type system (#6390) Added a mini type system for Dfg using DfgDataType to replace Dfg's use of AstNodeDType. This is much more restricted and represents only the types Dfg can handle in a canonical form. This will be needed when adding more support for unpacked arrays and maybe unpacked structs one day. Also added an internal type checker for DfgGraphs which encodes all the assumptions the code makes about type relationships in the graph. Run this in a few places with --debug-check. Fix resulting fallout.
2025-09-07 21:38:50 +02:00
if (vtxp->dtype() == vtxp->srcp()->dtype()) {
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
APPLYING(REMOVE_REPLICATE_ONCE) {
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(vtxp->srcp());
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
return;
}
}
if (foldBinary(vtxp)) return;
}
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
void visit(DfgShiftL* const vtxp) override {
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
if (foldBinary(vtxp)) return;
Avoid creating redundant vertices in V3DfgPeephole (#4944) Add a new data-structure V3DfgCache, which can be used to retrieve existing vertices with some given inputs vertices. Use this in V3DfgPeephole to eliminate the creation of redundant vertices. Overall this is performance neutral, but is in prep for some future work.
2024-03-06 19:01:52 +01:00
if (optimizeShiftRHS(vtxp)) return;
Optimize additional DFG peephole Shift and Concat patterns (#7077)
2026-02-14 13:25:17 +01:00
DfgVertex* const lhsp = vtxp->lhsp();
DfgVertex* const rhsp = vtxp->rhsp();
if (DfgConst* const rConstp = rhsp->cast<DfgConst>()) {
if (DfgConcat* const lConcatp = lhsp->cast<DfgConcat>()) {
if (!lConcatp->hasMultipleSinks()
&& lConcatp->lhsp()->width() == rConstp->toU32()) {
APPLYING(REPLACE_SHIFTL_CAT) {
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
DfgVertex* const zerop
= makeZero(lConcatp->fileline(), lConcatp->lhsp()->width());
replace(make<DfgConcat>(vtxp, lConcatp->rhsp(), zerop));
Optimize additional DFG peephole Shift and Concat patterns (#7077)
2026-02-14 13:25:17 +01:00
return;
}
}
}
if (DfgShiftR* const lShiftRp = lhsp->cast<DfgShiftR>()) {
if (!lShiftRp->hasMultipleSinks() && isSame(rConstp, lShiftRp->rhsp())) {
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
if (DfgConcat* const llCatp = lShiftRp->lhsp()->cast<DfgConcat>()) {
Optimize additional DFG peephole Shift and Concat patterns (#7077)
2026-02-14 13:25:17 +01:00
const uint32_t shiftAmount = rConstp->toU32();
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
if (!llCatp->hasMultipleSinks()
&& llCatp->rhsp()->width() == shiftAmount) {
Optimize additional DFG peephole Shift and Concat patterns (#7077)
2026-02-14 13:25:17 +01:00
APPLYING(REPLACE_SHIFTRL_CAT) {
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
DfgConst* const zerop = makeZero(llCatp->fileline(), shiftAmount);
replace(make<DfgConcat>(vtxp, llCatp->lhsp(), zerop));
Optimize additional DFG peephole Shift and Concat patterns (#7077)
2026-02-14 13:25:17 +01:00
return;
}
}
}
}
}
}
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
}
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
void visit(DfgShiftR* const vtxp) override {
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
if (foldBinary(vtxp)) return;
Avoid creating redundant vertices in V3DfgPeephole (#4944) Add a new data-structure V3DfgCache, which can be used to retrieve existing vertices with some given inputs vertices. Use this in V3DfgPeephole to eliminate the creation of redundant vertices. Overall this is performance neutral, but is in prep for some future work.
2024-03-06 19:01:52 +01:00
if (optimizeShiftRHS(vtxp)) return;
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
}
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
void visit(DfgShiftRS* const vtxp) override {
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
if (foldBinary(vtxp)) return;
Avoid creating redundant vertices in V3DfgPeephole (#4944) Add a new data-structure V3DfgCache, which can be used to retrieve existing vertices with some given inputs vertices. Use this in V3DfgPeephole to eliminate the creation of redundant vertices. Overall this is performance neutral, but is in prep for some future work.
2024-03-06 19:01:52 +01:00
if (optimizeShiftRHS(vtxp)) return;
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
}
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
void visit(DfgSub* const vtxp) override {
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
if (foldBinary(vtxp)) return;
DfgVertex* const lhsp = vtxp->lhsp();
DfgVertex* const rhsp = vtxp->rhsp();
if (DfgConst* const rConstp = rhsp->cast<DfgConst>()) {
if (rConstp->isZero()) {
APPLYING(REMOVE_SUB_ZERO) {
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(lhsp);
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
return;
}
}
Improve Dfg type system (#6390) Added a mini type system for Dfg using DfgDataType to replace Dfg's use of AstNodeDType. This is much more restricted and represents only the types Dfg can handle in a canonical form. This will be needed when adding more support for unpacked arrays and maybe unpacked structs one day. Also added an internal type checker for DfgGraphs which encodes all the assumptions the code makes about type relationships in the graph. Run this in a few places with --debug-check. Fix resulting fallout.
2025-09-07 21:38:50 +02:00
if (vtxp->dtype() == m_bitDType && rConstp->hasValue(1)) {
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
APPLYING(REPLACE_SUB_WITH_NOT) {
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(make<DfgNot>(vtxp->fileline(), m_bitDType, lhsp));
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
return;
}
}
}
}
//=========================================================================
// DfgVertexTernary
//=========================================================================
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
void visit(DfgCond* const vtxp) override {
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
DfgVertex* const condp = vtxp->condp();
DfgVertex* const thenp = vtxp->thenp();
DfgVertex* const elsep = vtxp->elsep();
DfgPeephole: add constant folding for all integer types Also added a testing only -fno-const-before-dfg option, as otherwise V3Const eats up a lot of the simple inputs. A lot of the things V3Const swallows in the simple cases can make it to DFG in complex cases, or DFG itself can create them during optimization. In any case to save complexity of testing DFG constant folding, we use this option to turn off V3Const prior to the DFG passes in the relevant test.
2022-10-04 22:18:39 +02:00
FileLine* const flp = vtxp->fileline();
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
Improve Dfg type system (#6390) Added a mini type system for Dfg using DfgDataType to replace Dfg's use of AstNodeDType. This is much more restricted and represents only the types Dfg can handle in a canonical form. This will be needed when adding more support for unpacked arrays and maybe unpacked structs one day. Also added an internal type checker for DfgGraphs which encodes all the assumptions the code makes about type relationships in the graph. Run this in a few places with --debug-check. Fix resulting fallout.
2025-09-07 21:38:50 +02:00
if (condp->dtype() != m_bitDType) return;
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
Internals: Improve DFG implementation details (#6355) Large scale refactoring to simplify some of the more obtuse internals of DFG. Remove multiple redundant internal APIs, simplify representation of variables, fix potential unsoundness in circular decomposition. No functional change intended.
2025-09-02 17:50:40 +02:00
if (isOnes(condp)) {
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
APPLYING(REMOVE_COND_WITH_TRUE_CONDITION) {
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(thenp);
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
return;
}
}
Internals: Improve DFG implementation details (#6355) Large scale refactoring to simplify some of the more obtuse internals of DFG. Remove multiple redundant internal APIs, simplify representation of variables, fix potential unsoundness in circular decomposition. No functional change intended.
2025-09-02 17:50:40 +02:00
if (isZero(condp)) {
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
APPLYING(REMOVE_COND_WITH_FALSE_CONDITION) {
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(elsep);
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
return;
}
}
Add DfgPeephole patterns (#6149)
2025-07-07 17:25:29 +02:00
if (isSame(thenp, elsep)) {
APPLYING(REMOVE_COND_WITH_BRANCHES_SAME) {
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(elsep);
Add DfgPeephole patterns (#6149)
2025-07-07 17:25:29 +02:00
return;
}
}
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
if (DfgNot* const condNotp = condp->cast<DfgNot>()) {
Try to ensure DFG peephole patterns don't grow the graph Some optimizations are only a net win if they help us remove a graph node (or at least ensure they don't grow the graph), or yields otherwise special logic, so try to apply them only in these cases.
2022-10-04 19:26:53 +02:00
if (!condp->hasMultipleSinks() || condNotp->hasMultipleSinks()) {
APPLYING(SWAP_COND_WITH_NOT_CONDITION) {
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(make<DfgCond>(vtxp, condNotp->srcp(), elsep, thenp));
Try to ensure DFG peephole patterns don't grow the graph Some optimizations are only a net win if they help us remove a graph node (or at least ensure they don't grow the graph), or yields otherwise special logic, so try to apply them only in these cases.
2022-10-04 19:26:53 +02:00
return;
}
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
}
}
if (DfgNeq* const condNeqp = condp->cast<DfgNeq>()) {
Try to ensure DFG peephole patterns don't grow the graph Some optimizations are only a net win if they help us remove a graph node (or at least ensure they don't grow the graph), or yields otherwise special logic, so try to apply them only in these cases.
2022-10-04 19:26:53 +02:00
if (!condp->hasMultipleSinks()) {
APPLYING(SWAP_COND_WITH_NEQ_CONDITION) {
Avoid creating redundant vertices in V3DfgPeephole (#4944) Add a new data-structure V3DfgCache, which can be used to retrieve existing vertices with some given inputs vertices. Use this in V3DfgPeephole to eliminate the creation of redundant vertices. Overall this is performance neutral, but is in prep for some future work.
2024-03-06 19:01:52 +01:00
DfgEq* const newCondp = make<DfgEq>(condp, condNeqp->lhsp(), condNeqp->rhsp());
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(make<DfgCond>(vtxp, newCondp, elsep, thenp));
Try to ensure DFG peephole patterns don't grow the graph Some optimizations are only a net win if they help us remove a graph node (or at least ensure they don't grow the graph), or yields otherwise special logic, so try to apply them only in these cases.
2022-10-04 19:26:53 +02:00
return;
}
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
}
}
if (DfgNot* const thenNotp = thenp->cast<DfgNot>()) {
if (DfgNot* const elseNotp = elsep->cast<DfgNot>()) {
DfgPeephole: Use a work list driven algorithm for speed Replace the 'run to fixed point' algorithm with a work list driven approach. Instead of marking the graph as changed, we explicitly add vertices to the work list, to be visited, when a vertex is changed. This improves both memory locality (as the work list is processed in last in first out order), and removed unnecessary visitations when only a few nodes changes.
2022-11-04 16:50:20 +01:00
if (!thenNotp->srcp()->is<DfgConst>() && !elseNotp->srcp()->is<DfgConst>()
Fix tautological predicate in V3DfgPeephole Fixes #3771
2022-11-22 14:48:49 +01:00
&& !thenNotp->hasMultipleSinks() && !elseNotp->hasMultipleSinks()) {
Try to ensure DFG peephole patterns don't grow the graph Some optimizations are only a net win if they help us remove a graph node (or at least ensure they don't grow the graph), or yields otherwise special logic, so try to apply them only in these cases.
2022-10-04 19:26:53 +02:00
APPLYING(PULL_NOTS_THROUGH_COND) {
Avoid creating redundant vertices in V3DfgPeephole (#4944) Add a new data-structure V3DfgCache, which can be used to retrieve existing vertices with some given inputs vertices. Use this in V3DfgPeephole to eliminate the creation of redundant vertices. Overall this is performance neutral, but is in prep for some future work.
2024-03-06 19:01:52 +01:00
DfgCond* const newCondp = make<DfgCond>(
vtxp, vtxp->condp(), thenNotp->srcp(), elseNotp->srcp());
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(make<DfgNot>(thenp->fileline(), vtxp->dtype(), newCondp));
Try to ensure DFG peephole patterns don't grow the graph Some optimizations are only a net win if they help us remove a graph node (or at least ensure they don't grow the graph), or yields otherwise special logic, so try to apply them only in these cases.
2022-10-04 19:26:53 +02:00
return;
}
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
}
}
}
Add DfgPeephole patterns (#6149)
2025-07-07 17:25:29 +02:00
if (DfgOr* const condOrp = condp->cast<DfgOr>()) {
if (DfgCond* const thenCondp = thenp->cast<DfgCond>()) {
if (!thenCondp->hasMultipleSinks()) {
if (condOrp->lhsp() == thenCondp->condp()) {
// '(a | b) ? (a ? x : y) : z' -> 'a ? x : b ? y : z'
APPLYING(REPLACE_COND_OR_THEN_COND_LHS) {
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
DfgCond* const ep = make<DfgCond>(thenCondp, condOrp->rhsp(),
thenCondp->elsep(), elsep);
replace(make<DfgCond>(vtxp, condOrp->lhsp(), thenCondp->thenp(), ep));
Add DfgPeephole patterns (#6149)
2025-07-07 17:25:29 +02:00
return;
}
}
if (condOrp->rhsp() == thenCondp->condp()) {
// '(a | b) ? (a ? x : y) : z' -> 'a ? x : b ? y : z'
APPLYING(REPLACE_COND_OR_THEN_COND_RHS) {
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
DfgCond* const ep = make<DfgCond>(thenCondp, condOrp->lhsp(),
thenCondp->elsep(), elsep);
replace(make<DfgCond>(vtxp, condOrp->rhsp(), thenCondp->thenp(), ep));
Add DfgPeephole patterns (#6149)
2025-07-07 17:25:29 +02:00
return;
}
}
}
}
}
DFG: Add more peephole patterns
2022-09-30 17:19:53 +02:00
if (vtxp->width() > 1) {
// 'cond ? a + 1 : a' -> 'a + cond'
if (DfgAdd* const thenAddp = thenp->cast<DfgAdd>()) {
if (DfgConst* const constp = thenAddp->lhsp()->cast<DfgConst>()) {
DFG: Fix attempted evaluation of constants wider than 32 bits Fixes #3718
2022-10-28 18:00:32 +02:00
if (constp->hasValue(1)) {
DFG: Add more peephole patterns
2022-09-30 17:19:53 +02:00
if (thenAddp->rhsp() == elsep) {
APPLYING(REPLACE_COND_INC) {
Avoid creating redundant vertices in V3DfgPeephole (#4944) Add a new data-structure V3DfgCache, which can be used to retrieve existing vertices with some given inputs vertices. Use this in V3DfgPeephole to eliminate the creation of redundant vertices. Overall this is performance neutral, but is in prep for some future work.
2024-03-06 19:01:52 +01:00
DfgConcat* const extp = make<DfgConcat>(
vtxp, makeZero(flp, vtxp->width() - 1), condp);
DFG: Add more peephole patterns
2022-09-30 17:19:53 +02:00
FileLine* const thenFlp = thenAddp->fileline();
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(
make<DfgAdd>(thenFlp, vtxp->dtype(), thenAddp->rhsp(), extp));
DFG: Add more peephole patterns
2022-09-30 17:19:53 +02:00
return;
}
}
}
}
}
// 'cond ? a - 1 : a' -> 'a - cond'
if (DfgSub* const thenSubp = thenp->cast<DfgSub>()) {
if (DfgConst* const constp = thenSubp->rhsp()->cast<DfgConst>()) {
DFG: Fix attempted evaluation of constants wider than 32 bits Fixes #3718
2022-10-28 18:00:32 +02:00
if (constp->hasValue(1)) {
DFG: Add more peephole patterns
2022-09-30 17:19:53 +02:00
if (thenSubp->lhsp() == elsep) {
APPLYING(REPLACE_COND_DEC) {
Avoid creating redundant vertices in V3DfgPeephole (#4944) Add a new data-structure V3DfgCache, which can be used to retrieve existing vertices with some given inputs vertices. Use this in V3DfgPeephole to eliminate the creation of redundant vertices. Overall this is performance neutral, but is in prep for some future work.
2024-03-06 19:01:52 +01:00
DfgConcat* const extp = make<DfgConcat>(
vtxp, makeZero(flp, vtxp->width() - 1), condp);
DFG: Add more peephole patterns
2022-09-30 17:19:53 +02:00
FileLine* const thenFlp = thenSubp->fileline();
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(
make<DfgSub>(thenFlp, vtxp->dtype(), thenSubp->lhsp(), extp));
DFG: Add more peephole patterns
2022-09-30 17:19:53 +02:00
return;
}
}
}
}
}
}
Improve Dfg type system (#6390) Added a mini type system for Dfg using DfgDataType to replace Dfg's use of AstNodeDType. This is much more restricted and represents only the types Dfg can handle in a canonical form. This will be needed when adding more support for unpacked arrays and maybe unpacked structs one day. Also added an internal type checker for DfgGraphs which encodes all the assumptions the code makes about type relationships in the graph. Run this in a few places with --debug-check. Fix resulting fallout.
2025-09-07 21:38:50 +02:00
if (vtxp->dtype() == m_bitDType) {
Internals: Improve DFG implementation details (#6355) Large scale refactoring to simplify some of the more obtuse internals of DFG. Remove multiple redundant internal APIs, simplify representation of variables, fix potential unsoundness in circular decomposition. No functional change intended.
2025-09-02 17:50:40 +02:00
if (isZero(thenp)) { // a ? 0 : b becomes ~a & b
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
APPLYING(REPLACE_COND_WITH_THEN_BRANCH_ZERO) {
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(make<DfgAnd>(vtxp, make<DfgNot>(vtxp, condp), elsep));
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
return;
}
}
Add DfgPeephole patterns (#6149)
2025-07-07 17:25:29 +02:00
if (thenp == condp) { // a ? a : b becomes a | b
APPLYING(REPLACE_COND_WITH_THEN_BRANCH_COND) {
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(make<DfgOr>(vtxp, condp, elsep));
Add DfgPeephole patterns (#6149)
2025-07-07 17:25:29 +02:00
return;
}
}
Internals: Improve DFG implementation details (#6355) Large scale refactoring to simplify some of the more obtuse internals of DFG. Remove multiple redundant internal APIs, simplify representation of variables, fix potential unsoundness in circular decomposition. No functional change intended.
2025-09-02 17:50:40 +02:00
if (isOnes(thenp)) { // a ? 1 : b becomes a | b
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
APPLYING(REPLACE_COND_WITH_THEN_BRANCH_ONES) {
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(make<DfgOr>(vtxp, condp, elsep));
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
return;
}
}
Internals: Improve DFG implementation details (#6355) Large scale refactoring to simplify some of the more obtuse internals of DFG. Remove multiple redundant internal APIs, simplify representation of variables, fix potential unsoundness in circular decomposition. No functional change intended.
2025-09-02 17:50:40 +02:00
if (isZero(elsep)) { // a ? b : 0 becomes a & b
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
APPLYING(REPLACE_COND_WITH_ELSE_BRANCH_ZERO) {
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(make<DfgAnd>(vtxp, condp, thenp));
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
return;
}
}
Internals: Improve DFG implementation details (#6355) Large scale refactoring to simplify some of the more obtuse internals of DFG. Remove multiple redundant internal APIs, simplify representation of variables, fix potential unsoundness in circular decomposition. No functional change intended.
2025-09-02 17:50:40 +02:00
if (isOnes(elsep)) { // a ? b : 1 becomes ~a | b
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
APPLYING(REPLACE_COND_WITH_ELSE_BRANCH_ONES) {
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
replace(make<DfgOr>(vtxp, make<DfgNot>(vtxp, condp), thenp));
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
return;
}
}
}
}
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
void visit(DfgVertexVar* const vtxp) override {
Optimize: Remove redundant variables during Dfg Peephole pass (#7076) Many rules in the Dfg Peephole pass check if a node has more than one sinks. Redundant variables that will ultimately be removed can prevent these from matching. Remove such variables during the Peeophole pass itself to enable more matches.
2026-02-14 13:33:20 +01:00
if (vtxp->hasSinks()) return;
if (vtxp->isObserved()) return;
if (vtxp->defaultp()) return;
// If undriven, or driven from another var, it is completely redundant.
if (!vtxp->srcp() || vtxp->srcp()->is<DfgVertexVar>()) {
APPLYING(REMOVE_VAR) {
deleteVertex(vtxp);
return;
}
}
// Otherwise remove if there is only one sink that is not a removable variable
bool foundOne = false;
const bool keep = vtxp->srcp()->foreachSink([&](DfgVertex& sink) {
Commentary: Fix non-grammar
2026-03-12 00:53:23 +01:00
// Ignore non-observable variable sinks. These can be eliminated.
Optimize: Remove redundant variables during Dfg Peephole pass (#7076) Many rules in the Dfg Peephole pass check if a node has more than one sinks. Redundant variables that will ultimately be removed can prevent these from matching. Remove such variables during the Peeophole pass itself to enable more matches.
2026-02-14 13:33:20 +01:00
if (const DfgVertexVar* const varp = sink.cast<DfgVertexVar>()) {
if (!varp->hasSinks() && !varp->isObserved()) return false;
}
if (foundOne) return true;
foundOne = true;
return false;
});
if (!keep) {
APPLYING(REMOVE_VAR) {
deleteVertex(vtxp);
return;
}
}
}
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
V3DfgPeephole(DfgGraph& dfg, V3DfgPeepholeContext& ctx)
: m_dfg{dfg}
Speed up DfgPeephole in various ways
2022-10-06 13:02:46 +02:00
, m_ctx{ctx} {
Internals: Use a custom worklist in DfgPeephole
2026-03-23 15:41:36 +01:00
// Initialize the work list
m_wlist.reserve(m_dfg.size() * 4);
Internals: Remove unused vertices immediately in DfgPeephole
2026-03-23 16:06:07 +01:00
Internals: Use a custom worklist in DfgPeephole
2026-03-23 15:41:36 +01:00
// Need a nullptr at index 0 so VertexInfo::m_workListIndex can be used to check membership
m_wlist.push_back(nullptr);
Optimize: Remove redundant variables during Dfg Peephole pass (#7076) Many rules in the Dfg Peephole pass check if a node has more than one sinks. Redundant variables that will ultimately be removed can prevent these from matching. Remove such variables during the Peeophole pass itself to enable more matches.
2026-02-14 13:33:20 +01:00
// Add all variable vertices to the work list. Do this first so they are processed last.
// This order has a better chance of preserving original variables in case they are needed.
for (DfgVertexVar& vtx : m_dfg.varVertices()) addToWorkList(&vtx);
Internals: Use a custom worklist in DfgPeephole
2026-03-23 15:41:36 +01:00
// Add all operation vertices to the work list
Internals: Simplify V3DfgCache
2026-03-22 11:19:31 +01:00
for (DfgVertex& vtx : m_dfg.opVertices()) addToWorkList(&vtx);
DfgPeephole: Use a work list driven algorithm for speed Replace the 'run to fixed point' algorithm with a work list driven approach. Instead of marking the graph as changed, we explicitly add vertices to the work list, to be visited, when a vertex is changed. This improves both memory locality (as the work list is processed in last in first out order), and removed unnecessary visitations when only a few nodes changes.
2022-11-04 16:50:20 +01:00
// Process the work list
Internals: Use a custom worklist in DfgPeephole
2026-03-23 15:41:36 +01:00
while (!m_wlist.empty()) {
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
VL_RESTORER(m_vtxp);
Internals: Use a custom worklist in DfgPeephole
2026-03-23 15:41:36 +01:00
// Pop up head of work list
m_vtxp = m_wlist.back();
m_wlist.pop_back();
if (!m_vtxp) continue; // If removed from worklist, move on
m_vInfo[m_vtxp].m_workListIndex = 0; // No longer on work list
// Variables are special, just visit them, the visit might delete them
if (DfgVertexVar* const varp = m_vtxp->cast<DfgVertexVar>()) {
visit(varp);
continue;
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
}
Internals: Remove unused vertices immediately in DfgPeephole
2026-03-23 16:06:07 +01:00
// Unsued vertices should have been removed immediately
UASSERT_OBJ(m_vtxp->hasSinks(), m_vtxp, "Operation vertex should have sinks");
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
// Check if an equivalent vertex exists, if so replace this vertex with it
Internals: Use a custom worklist in DfgPeephole
2026-03-23 15:41:36 +01:00
if (DfgVertex* const sampep = m_cache.cache(m_vtxp)) {
Internals: Remove unused vertices immediately in DfgPeephole
2026-03-23 16:06:07 +01:00
APPLYING(REPLACE_WITH_EQUIVALENT) {
replace(sampep);
continue;
}
Internals: Simplify DfgPeephole replacement scheme Always only replace the currently iterated node. This ensures it is not on the work list, which simplifies upcoming work. No functional of performance change intended.
2026-03-23 10:09:47 +01:00
}
// Visit vertex, might get deleted in the process
Internals: Use a custom worklist in DfgPeephole
2026-03-23 15:41:36 +01:00
iterate(m_vtxp);
}
Speed up DfgPeephole in various ways
2022-10-06 13:02:46 +02:00
}
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
Internals: Remove unused vertices immediately in DfgPeephole
2026-03-23 16:06:07 +01:00
#undef APPLYING
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
public:
Speed up DfgPeephole in various ways
2022-10-06 13:02:46 +02:00
static void apply(DfgGraph& dfg, V3DfgPeepholeContext& ctx) { V3DfgPeephole{dfg, ctx}; }
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
};
Internals: Add utility to perform bisection search for debugging (#7294)
2026-03-21 11:13:27 +01:00
V3DebugBisect V3DfgPeephole::s_debugBisect{"DfgPeephole"};
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
void V3DfgPasses::peephole(DfgGraph& dfg, V3DfgPeepholeContext& ctx) {
Improve Dfg variable removal and temporary insertion (#6401) Combined Dfg variable elimination into the regularization pass that runs before converting back to Ast. This avoids introducing some unnecessary temporaries. Added replacing of variables with constants in the Ast if after the Dfg passes they are known to be constants. This is only done in final scoped Dfg application. Avoid introducing temporaries for common sub-expressions that are cheaper to re-compute than store in a temporary variable. Enable removal of redundant unpacked array variables. Also fixes #6394 as this patch involved changes to that code.
2025-09-10 13:38:49 +02:00
if (!v3Global.opt.fDfgPeephole()) return;
Introduce DFG based combinational logic optimizer (#3527) Added a new data-flow graph (DFG) based combinational logic optimizer. The capabilities of this covers a combination of V3Const and V3Gate, but is also more capable of transforming combinational logic into simplified forms and more. This entail adding a new internal representation, `DfgGraph`, and appropriate `astToDfg` and `dfgToAst` conversion functions. The graph represents some of the combinational equations (~continuous assignments) in a module, and for the duration of the DFG passes, it takes over the role of AstModule. A bulk of the Dfg vertices represent expressions. These vertex classes, and the corresponding conversions to/from AST are mostly auto-generated by astgen, together with a DfgVVisitor that can be used for dynamic dispatch based on vertex (operation) types. The resulting combinational logic graph (a `DfgGraph`) is then optimized in various ways. Currently we perform common sub-expression elimination, variable inlining, and some specific peephole optimizations, but there is scope for more optimizations in the future using the same representation. The optimizer is run directly before and after inlining. The pre inline pass can operate on smaller graphs and hence converges faster, but still has a chance of substantially reducing the size of the logic on some designs, making inlining both faster and less memory intensive. The post inline pass can then optimize across the inlined module boundaries. No optimization is performed across a module boundary. For debugging purposes, each peephole optimization can be disabled individually via the -fno-dfg-peepnole-<OPT> option, where <OPT> is one of the optimizations listed in V3DfgPeephole.h, for example -fno-dfg-peephole-remove-not-not. The peephole patterns currently implemented were mostly picked based on the design that inspired this work, and on that design the optimizations yields ~30% single threaded speedup, and ~50% speedup on 4 threads. As you can imagine not having to haul around redundant combinational networks in the rest of the compilation pipeline also helps with memory consumption, and up to 30% peak memory usage of Verilator was observed on the same design. Gains on other arbitrary designs are smaller (and can be improved by analyzing those designs). For example OpenTitan gains between 1-15% speedup depending on build type.
2022-09-23 17:46:22 +02:00
V3DfgPeephole::apply(dfg, ctx);
}