Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Complete Transition to Control Flow Regions #1676

Merged
merged 128 commits into from
Dec 12, 2024
Merged

Conversation

phschaad
Copy link
Collaborator

@phschaad phschaad commented Oct 8, 2024

This PR completes the transition to hierarchical control flow regions in DaCe. By nature of the significance in change that is brought through the transition to hierarchical control flow regions, this PR is rather substantial. An exhaustive listing of all adaptations is not feasible, but the most important changes and adaptations are listed below:

  • Change the default of the Python frontend to generate SDFGs using experimental CFG blocks. A subsequent PR will remove the option to not use experimental CFG blocks entirely, but this was left to a separate PR to avoid growing this one even more than it already has.
  • The option to write a pass or transformation that is not compatible with experimental blocks has been removed, forcing new transformations and passes to consider them in their design.
  • Simplifications to loop related transformations by adapting explicit loop regions.
  • Add a new pass base type, ControlFlowRegionPass: This pass works like StatePass or ScopePass, and can be extended to write a pass that applies recursively to each control flow region of an SDFG. An option can be set to either apply bottom-up or top-down.
  • A pass has been added to dead code elimination to prune empty or falsy conditional branches.
  • Include a control flow raising pass in the simplification pipeline, ensuring that even SDFGs generated without the explicit use of experimental blocks are raised to the new style SDFGs.
  • Adapt all passes and transformations currently in main DaCe to work with SDFGs containing experimental CFG blocks.
  • Almost all transformations and analyses now expect that experimental blocks are used for regular / reducible control flow, meaning some control flow analyses have been ditched to improve overall performance and reliability of DaCe, and remove redundancy.
  • Ensure all compiler backends correctly handle experimental blocks.
  • Adapt state propagation into a separate pass that has been made to use experimental blocks. Legacy state propagation has been left in for now, including tests that ensure it works as intended, to avoid making this PR even larger. However, it is planned to remove this in a subsequent PR soon.
  • A block fusion pass has been added to the simplification pipeline. This operates similar to StateFusion, but fuses no-op general control flow blocks (empty states or control flow blocks) with other control flow blocks. This reduces the number of nodes and edges in CFGs further.
  • Numerous bugfixes with respect to experimental blocks and analyses around them, thanks to the ability to now run the entire CI pipeline with them.

Note: The FV3 integration test fails and will continue to fail with this PR, since GT4Py cartesian, which is used by PyFV3, does not consider experimental blocks in their design. Since DaCe v1.0.0 will be released without this PR in it, my suggestion is to limit the application of the FV3 integration and regression tests to PRs which are made to a specific v1.0.0 maintenance branch, which is used for fixes to v1.0.0.

@phschaad phschaad added the no-ci Do not run any CI or actions for this PR label Oct 8, 2024
Copy link
Collaborator

@tbennun tbennun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good job, clearly a lot of work went into this! I think we can do better with future-proofing the code to other control flow regions with some methods that could be implemented, and I have some other open questions in the comments.

tests/codegen/data_instrumentation_test.py Show resolved Hide resolved
tests/inlining_test.py Show resolved Hide resolved
tests/passes/writeset_underapproximation_test.py Outdated Show resolved Hide resolved
tests/schedule_tree/schedule_test.py Outdated Show resolved Hide resolved
tests/sdfg/schedule_inference_test.py Show resolved Hide resolved
dace/transformation/passes/analysis/analysis.py Outdated Show resolved Hide resolved
dace/transformation/passes/dead_state_elimination.py Outdated Show resolved Hide resolved
@phschaad phschaad force-pushed the users/phschaad/adapt_passes branch from f1cb99f to 3a2b342 Compare December 9, 2024 15:47
@phschaad phschaad requested a review from tbennun December 11, 2024 15:30
Copy link
Collaborator

@tbennun tbennun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking much better now! Minor comments only. See unresolved comments for future PR tasks.

@@ -69,8 +69,7 @@ def program(f: F,
not depend on internal variables are constant.
This will hardcode their return values into the
resulting program.
:param use_experimental_cfg_blocks: If True, makes use of experimental CFG blocks susch as loop and conditional
regions.
:param use_explicit_cfl: If True, makes use of explicit control flow constructs.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
:param use_explicit_cfl: If True, makes use of explicit control flow constructs.
:param use_explicit_cf: If True, makes use of explicit control flow constructs.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed

@@ -2578,6 +2583,17 @@ def sdfg(self) -> 'SDFG':
@make_properties
class AbstractControlFlowRegion(OrderedDiGraph[ControlFlowBlock, 'dace.sdfg.InterstateEdge'], ControlGraphView,
ControlFlowBlock, abc.ABC):
"""
Abstract superclass to represent all kinds of control flow regions in an SDFG.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

docstrings should use double backticks (``) for inline code in Sphinx.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed

@@ -2711,6 +2753,12 @@ def inline(self) -> Tuple[bool, Any]:

return False, None

def new_symbols(self, symbols: dict) -> Dict[str, dtypes.typeclass]:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
def new_symbols(self, symbols: dict) -> Dict[str, dtypes.typeclass]:
def new_symbols(self, symbols: Dict[str, dtypes.typeclass]) -> Dict[str, dtypes.typeclass]:

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed

yield from node.sdfg.all_control_flow_regions(recursive=recursive, load_ext=load_ext,
parent_first=parent_first)
elif load_ext:
node.load_external(block)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remnant from another PR?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Must be due to a missed rebase, this is already in main: https://github.com/spcl/dace/blob/main/dace/sdfg/state.py#L2838

@@ -3042,6 +3098,11 @@ def start_block(self, block_id):

@make_properties
class ControlFlowRegion(AbstractControlFlowRegion):
"""
A `ControlFlowRegion` represents a control flow graph node that itself contains a control flow graph.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
A `ControlFlowRegion` represents a control flow graph node that itself contains a control flow graph.
A ``ControlFlowRegion`` represents a control flow graph node that itself contains a control flow graph.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed

@phschaad phschaad requested a review from tbennun December 12, 2024 08:02
@phschaad phschaad enabled auto-merge December 12, 2024 08:14
Copy link
Collaborator

@tbennun tbennun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@phschaad phschaad added this pull request to the merge queue Dec 12, 2024
Merged via the queue into main with commit 896a1e1 Dec 12, 2024
9 checks passed
@phschaad phschaad deleted the users/phschaad/adapt_passes branch December 12, 2024 16:47
@FlorianDeconinck
Copy link
Contributor

Congrats Phil, that's a push ;)

I logged the ci for Pace there for reference: #1826

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: ✅ Done
5 participants