Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Data Dependency Analysis for Control Flow Regions #1716

Draft
wants to merge 152 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
152 commits
Select commit Hold shift + click to select a range
036e407
Add data dependency analyses
phschaad Sep 16, 2024
4aa13ed
Fix type
phschaad Sep 16, 2024
a228f34
Fix types
phschaad Sep 16, 2024
10c3b6c
Update tests
phschaad Sep 16, 2024
77ca17f
Fixes
phschaad Sep 16, 2024
4e6035d
Fix tests
phschaad Sep 17, 2024
b61a283
Add tests
phschaad Sep 17, 2024
05b1c28
Add loop lifting capabilities
phschaad Sep 18, 2024
f08d95e
Adjust loop detection to LLVM canonical semantics
phschaad Sep 19, 2024
1d90346
Test fix
phschaad Sep 19, 2024
6b5ef0c
Remove unnecessary imports
phschaad Sep 19, 2024
ba5ccdf
Merge branch 'master' into cf_block_data_deps
phschaad Sep 30, 2024
23af038
Improved loop detection
phschaad Oct 2, 2024
3fbe26b
Loop detection and lifting fixes
phschaad Oct 2, 2024
2bd5d00
Work on conditional lifting
phschaad Oct 2, 2024
dc640a9
Improve conditional block interface
phschaad Oct 7, 2024
e14e7a4
Merge branch 'cf_block_data_deps' into users/phschaad/adapt_passes
phschaad Oct 7, 2024
02d53fc
Remove files from other PR
phschaad Oct 7, 2024
338db80
Merge branch 'cf_block_data_deps' into users/phschaad/adapt_passes
phschaad Oct 7, 2024
074a990
Add back missing file
phschaad Oct 7, 2024
cd8a258
Adapt dead state elimination
phschaad Oct 7, 2024
f77655f
Adapt DeadDataflowElimination
phschaad Oct 7, 2024
49475ec
Fixes
phschaad Oct 8, 2024
094c896
Merge branch 'cf_block_data_deps' into users/phschaad/adapt_passes
phschaad Oct 8, 2024
2a74901
Bugfix
phschaad Oct 8, 2024
35cfded
Merge branch 'cf_block_data_deps' into users/phschaad/adapt_passes
phschaad Oct 8, 2024
8127900
Adapt trivial loop elimination and state elimination
phschaad Oct 8, 2024
04f41c2
gdapt passes:
phschaad Oct 8, 2024
852719d
Adapt loop to map
phschaad Oct 9, 2024
352171a
Fixes in loop to map
phschaad Oct 9, 2024
bbddc88
Finished LoopToMap
phschaad Oct 9, 2024
5ef1f46
Adapt loop unrolling
phschaad Oct 9, 2024
2d26e0f
Adapt loop peeling and unrolling
phschaad Oct 10, 2024
145c0ea
Added tests for the `_read_and_write_sets()`.
philip-paul-mueller Oct 11, 2024
38e748b
Added the fix from my MapFusion PR.
philip-paul-mueller Oct 11, 2024
3748c03
Now made `read_and_write_sets()` fully adhere to their own definition.
philip-paul-mueller Oct 11, 2024
6d83976
Update refine nested access
phschaad Oct 11, 2024
bb4e9c8
Fixes
phschaad Oct 11, 2024
bd76961
Adjust SDFG nesting
phschaad Oct 11, 2024
3ab4bf3
Updated a test for the `PruneConnectors` transformation.
philip-paul-mueller Oct 11, 2024
26e4ff0
Adapt more passes and add conditional pruning pass
phschaad Oct 11, 2024
b4feddf
Added code to `test_more_than_a_map` to ensure that the transformatio…
philip-paul-mueller Oct 11, 2024
324fa34
More fixes
phschaad Oct 11, 2024
103b4e5
Merge remote-tracking branch 'origin/master' into users/phschaad/adap…
phschaad Oct 13, 2024
e1c25b2
Merge remote-tracking branch 'spcl/master' into read-write-sets
philip-paul-mueller Oct 14, 2024
70fa3db
Added the new memlet creation syntax.
philip-paul-mueller Oct 14, 2024
b187a82
Modified some comments to make them clearer.
philip-paul-mueller Oct 14, 2024
9c6cb6c
Modified the `tests/transformations/move_loop_into_map_test.py::test_…
philip-paul-mueller Oct 14, 2024
251833f
Adapt scalar fission and symbol ssa
phschaad Oct 15, 2024
5b7bdad
Fixes
phschaad Oct 15, 2024
99225a5
Adapt reference reduction pass
phschaad Oct 15, 2024
d6c7c8b
Adapt constant propagation
phschaad Oct 16, 2024
607b098
Fix pytest arguments
phschaad Oct 16, 2024
05b3e59
Fixes
phschaad Oct 16, 2024
f5b617c
Fix invalid graph manipulation in test
phschaad Oct 16, 2024
33c9287
Fixes
phschaad Oct 16, 2024
9185897
Adapt composite fusion
phschaad Oct 16, 2024
bf7d822
Fix StencilTiling
phschaad Oct 17, 2024
78803d5
Add some region inlining
phschaad Oct 17, 2024
26a7fbc
Fixes to deepcopy
phschaad Oct 17, 2024
1862ba4
More various fixes
phschaad Oct 17, 2024
422edb5
Yet more fixes
phschaad Oct 17, 2024
a66c610
More fixes
phschaad Oct 18, 2024
b507a4b
Fixes
phschaad Oct 18, 2024
4084dfe
More fixes
phschaad Oct 18, 2024
d010620
More fixes
phschaad Oct 18, 2024
a047d37
And yet more
phschaad Oct 18, 2024
39db909
Another one
phschaad Oct 18, 2024
b5fc16f
Merge branch 'master' into read-write-sets
philip-paul-mueller Oct 22, 2024
b7fe242
Added a test to highlights the error.
philip-paul-mueller Oct 22, 2024
b546b07
I now removed the filtering inside the read and write set.
philip-paul-mueller Oct 22, 2024
8f9e72f
And more
phschaad Oct 22, 2024
6067e3b
Merge remote-tracking branch 'origin/master' into users/phschaad/adap…
phschaad Oct 22, 2024
2c4c17b
Fix inline multistate
phschaad Oct 22, 2024
ae20590
Fixed `state_test.py::test_read_and_write_set_filter`.
philip-paul-mueller Oct 23, 2024
db211fa
Fixed the `state_test.py::test_read_write_set` test.
philip-paul-mueller Oct 23, 2024
570437b
Fixed the `state_test.py::test_read_write_set_y_formation` test.
philip-paul-mueller Oct 23, 2024
6806dc1
Fix cyclic dependency
phschaad Oct 23, 2024
ab255e8
Fixes to codegen and data instrumentation
phschaad Oct 23, 2024
e97f5bc
Fix subgraph nesting
phschaad Oct 23, 2024
8de1c1e
Fixes to GPU codegen
phschaad Oct 23, 2024
cb80f0b
Fixed `move_loop_into_map_test.py::MoveLoopIntoMapTest::test_more_tha…
philip-paul-mueller Oct 23, 2024
b704a43
Fixed `prune_connectors_test.py::test_read_write_*`.
philip-paul-mueller Oct 23, 2024
f74d6e8
General improvements to some tests.
philip-paul-mueller Oct 23, 2024
e103924
Updated `refine_nested_access_test.py::test_rna_read_and_write_sets_d…
philip-paul-mueller Oct 23, 2024
56e756d
More GPU fixes
phschaad Oct 23, 2024
6e14e6d
More fixes
phschaad Oct 23, 2024
1902e3d
Merge branch 'read-write-sets' into users/phschaad/adapt_passes
phschaad Oct 23, 2024
0c65359
More bugfixes
phschaad Oct 23, 2024
ac72cb1
Fixes
phschaad Oct 23, 2024
3c4cea0
Merge remote-tracking branch 'origin/master' into users/phschaad/adap…
phschaad Oct 24, 2024
cc9e5f4
FPGA fixes
phschaad Oct 24, 2024
cfa0299
Merge remote-tracking branch 'origin/main' into users/phschaad/adapt_…
phschaad Oct 25, 2024
bc9f61e
Adapt state propagation into a pass to adapt it
phschaad Oct 29, 2024
2a26d24
Merge remote-tracking branch 'origin/main' into users/phschaad/adapt_…
phschaad Oct 29, 2024
f63f75d
Fix w/d test inlining
phschaad Oct 29, 2024
48c7cb4
Fix to block fusion
phschaad Oct 29, 2024
a0e2c59
Derped a test..
phschaad Oct 29, 2024
ca00134
Merge branch 'main' into users/phschaad/adapt_passes
phschaad Oct 31, 2024
604a914
WIP
phschaad Oct 31, 2024
8d03dc5
Merge branch 'main' into users/phschaad/adapt_passes
phschaad Nov 5, 2024
0828fa1
Fix inlining
phschaad Nov 5, 2024
7477607
Merge branch 'users/phschaad/adapt_passes' into users/phschaad/cf_blo…
phschaad Nov 5, 2024
04336c3
First draft of propagation pass
phschaad Nov 7, 2024
94c6023
Refine propagation
phschaad Nov 7, 2024
07c822c
Merge branch 'main' into users/phschaad/adapt_passes
phschaad Nov 12, 2024
3094fa1
Update SDFV
phschaad Nov 12, 2024
38fcbaf
Endless loop in constant prop fix
phschaad Nov 12, 2024
567d307
Fix propagation
phschaad Nov 12, 2024
61ac6ee
More fixes
phschaad Nov 13, 2024
8c488de
Update gitignore
phschaad Nov 13, 2024
6fa4720
Merge branch 'users/phschaad/adapt_passes' into users/phschaad/cf_blo…
phschaad Nov 13, 2024
49106de
Fixes
phschaad Nov 13, 2024
7e4bc3d
Fix loop symbol type inference and loop to map
phschaad Nov 13, 2024
40d4a12
Fix traversal for defined symbols
phschaad Nov 13, 2024
f7b2543
Merge branch 'users/phschaad/adapt_passes' into users/phschaad/cf_blo…
phschaad Nov 13, 2024
682f960
Provide memlet propagation through loops
phschaad Nov 13, 2024
c61af96
Merge branch 'main' into users/phschaad/adapt_passes
phschaad Nov 25, 2024
1ea1d37
Merge branch 'users/phschaad/adapt_passes' into users/phschaad/cf_blo…
phschaad Nov 25, 2024
ead33cc
Add more tests to propagation
phschaad Nov 26, 2024
b14feff
Merge remote-tracking branch 'origin/main' into users/phschaad/cf_blo…
phschaad Dec 2, 2024
ac35c64
Minor fixes related to structs
phschaad Dec 3, 2024
ef595b4
Skip FV3 pipeline until adapted to V2
phschaad Dec 3, 2024
3af9a70
Merge branch 'main' into users/phschaad/adapt_passes
phschaad Dec 3, 2024
6b813d8
Merge branch 'users/phschaad/adapt_passes' into users/phschaad/cf_blo…
phschaad Dec 3, 2024
d27ec31
Fix bug introduced through merge
phschaad Dec 3, 2024
3a7634f
Merge branch 'users/phschaad/adapt_passes' into users/phschaad/cf_blo…
phschaad Dec 3, 2024
4d2b7b3
Loop dependency analysis
phschaad Dec 3, 2024
6ff0a6e
Add subset difference
phschaad Dec 3, 2024
20d0703
Subset improvements
phschaad Dec 4, 2024
786a07e
Merge branch 'main' into users/phschaad/adapt_passes
phschaad Dec 9, 2024
72508bd
Update dace/transformation/interstate/block_fusion.py
phschaad Dec 9, 2024
ce82732
Address review comments
phschaad Dec 9, 2024
83b26c1
Merge branch 'users/phschaad/adapt_passes' of github.com:spcl/dace in…
phschaad Dec 9, 2024
3c13369
More comments
phschaad Dec 9, 2024
c4e78d7
Add doc comments
phschaad Dec 9, 2024
3a2b342
Address more comments
phschaad Dec 9, 2024
2f7d6aa
Address more review comments
phschaad Dec 10, 2024
a469286
Inlining fix
phschaad Dec 10, 2024
a8546ab
Fixes to control flow raising and codegen
phschaad Dec 11, 2024
c9d6b51
Renamed experimental_cfg_blocks to explicit_control_flow
phschaad Dec 11, 2024
e2a6466
Added more extensible meta access replacement function
phschaad Dec 11, 2024
6f82fc6
Fixes
phschaad Dec 11, 2024
34a3247
Add more API methods
phschaad Dec 11, 2024
fad3424
Address comments
phschaad Dec 12, 2024
7c88c7f
Merge branch 'users/phschaad/adapt_passes' into users/phschaad/cf_blo…
phschaad Dec 12, 2024
71ac360
Fixes after merge
phschaad Dec 12, 2024
200e606
Let LiftStructViews lift interstate edge struct accesses
phschaad Dec 13, 2024
647ce87
Merge branch 'main' into users/phschaad/cf_block_data_deps
phschaad Dec 13, 2024
9ab0937
Merge branch 'users/phschaad/lift_interstate_edge_structs' into users…
phschaad Dec 13, 2024
fdce53a
Ensure array elimination correctly merges view nodes
phschaad Dec 13, 2024
d40f73b
Merge branch 'users/phschaad/lift_interstate_edge_structs' into users…
phschaad Dec 13, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions dace/codegen/tools/type_inference.py
Original file line number Diff line number Diff line change
Expand Up @@ -420,6 +420,8 @@ def _Attribute(t, symbols, inferred_symbols):
if (isinstance(inferred_type, dtypes.pointer) and isinstance(inferred_type.base_type, dtypes.struct) and
t.attr in inferred_type.base_type.fields):
return inferred_type.base_type.fields[t.attr]
elif isinstance(inferred_type, dtypes.struct) and t.attr in inferred_type.fields:
return inferred_type.fields[t.attr]
return inferred_type


Expand Down
16 changes: 14 additions & 2 deletions dace/memlet.py
Original file line number Diff line number Diff line change
Expand Up @@ -147,8 +147,20 @@ def __init__(self,

@staticmethod
def from_memlet(memlet: 'Memlet') -> 'Memlet':
sbs = subsets.Range(memlet.subset.ndrange()) if memlet.subset is not None else None
osbs = subsets.Range(memlet.other_subset.ndrange()) if memlet.other_subset is not None else None
if memlet.subset is not None:
if isinstance(memlet.subset, subsets.SubsetUnion):
sbs = subsets.SubsetUnion(memlet.subset.subset_list)
else:
sbs = subsets.Range(memlet.subset.ndrange())
else:
sbs = None
if memlet.other_subset is not None:
if isinstance(memlet.other_subset, subsets.SubsetUnion):
osbs = subsets.SubsetUnion(memlet.other_subset.subset_list)
else:
osbs = subsets.Range(memlet.other_subset.ndrange())
else:
osbs = None
result = Memlet(data=memlet.data,
subset=sbs,
other_subset=osbs,
Expand Down
60 changes: 42 additions & 18 deletions dace/sdfg/propagation.py
Original file line number Diff line number Diff line change
Expand Up @@ -747,7 +747,7 @@ def propagate_states(sdfg: 'SDFG', concretize_dynamic_unbounded: bool = False) -
if sdfg.using_explicit_control_flow:
# Avoid cyclic imports
from dace.transformation.pass_pipeline import Pipeline
from dace.transformation.passes.analysis import StatePropagation
from dace.transformation.passes.analysis.propagation import StatePropagation

state_prop_pipeline = Pipeline([StatePropagation()])
state_prop_pipeline.apply_pass(sdfg, {})
Expand Down Expand Up @@ -1456,24 +1456,48 @@ def propagate_subset(memlets: List[Memlet],
else:
subset = md.subset

for pclass in MemletPattern.extensions():
pattern = pclass()
if pattern.can_be_applied([subset], variable_context, rng, [md]):
tmp_subset = pattern.propagate(arr, [subset], rng)
break
if isinstance(subset, subsets.SubsetUnion):
tmp_subset_list = []
for sub in subset.subset_list:
for pclass in MemletPattern.extensions():
pattern = pclass()
if pattern.can_be_applied([sub], variable_context, rng, [md]):
sub_tmp_subset = pattern.propagate(arr, [sub], rng)
break
else:
# No patterns found. Emit a warning and propagate the entire
# array whenever symbols are used
warnings.warn('Cannot find appropriate memlet pattern to '
'propagate %s through %s' % (str(sub), str(rng)))
entire_array = subsets.Range.from_array(arr)
paramset = set(map(str, params))
# Fill in the entire array only if one of the parameters appears in the
# free symbols list of the subset dimension
sub_tmp_subset = subsets.Range([
ea if any(set(map(str, _freesyms(sd))) & paramset for sd in s) else s
for s, ea in zip(sub, entire_array)
])
tmp_subset_list.append(sub_tmp_subset)
tmp_subset = subsets.SubsetUnion(tmp_subset_list)
else:
# No patterns found. Emit a warning and propagate the entire
# array whenever symbols are used
warnings.warn('Cannot find appropriate memlet pattern to '
'propagate %s through %s' % (str(subset), str(rng)))
entire_array = subsets.Range.from_array(arr)
paramset = set(map(str, params))
# Fill in the entire array only if one of the parameters appears in the
# free symbols list of the subset dimension
tmp_subset = subsets.Range([
ea if any(set(map(str, _freesyms(sd))) & paramset for sd in s) else s
for s, ea in zip(subset, entire_array)
])
for pclass in MemletPattern.extensions():
pattern = pclass()
if pattern.can_be_applied([subset], variable_context, rng, [md]):
tmp_subset = pattern.propagate(arr, [subset], rng)
break
else:
# No patterns found. Emit a warning and propagate the entire
# array whenever symbols are used
warnings.warn('Cannot find appropriate memlet pattern to '
'propagate %s through %s' % (str(subset), str(rng)))
entire_array = subsets.Range.from_array(arr)
paramset = set(map(str, params))
# Fill in the entire array only if one of the parameters appears in the
# free symbols list of the subset dimension
tmp_subset = subsets.Range([
ea if any(set(map(str, _freesyms(sd))) & paramset for sd in s) else s
for s, ea in zip(subset, entire_array)
])

# Union edges as necessary
if new_subset is None:
Expand Down
9 changes: 9 additions & 0 deletions dace/sdfg/state.py
Original file line number Diff line number Diff line change
Expand Up @@ -1208,6 +1208,11 @@ class ControlFlowBlock(BlockGraphView, abc.ABC):
_sdfg: Optional['SDFG'] = None
_parent_graph: Optional['ControlFlowRegion'] = None

_certain_reads: Dict[str, mm.Memlet]
_possible_reads: Dict[str, mm.Memlet]
_certain_writes: Dict[str, mm.Memlet]
_possible_writes: Dict[str, mm.Memlet]

def __init__(self, label: str = '', sdfg: Optional['SDFG'] = None, parent: Optional['ControlFlowRegion'] = None):
super(ControlFlowBlock, self).__init__()
self._label = label
Expand All @@ -1218,6 +1223,10 @@ def __init__(self, label: str = '', sdfg: Optional['SDFG'] = None, parent: Optio
self.pre_conditions = {}
self.post_conditions = {}
self.invariant_conditions = {}
self._certain_reads = dict()
self._possible_reads = dict()
self._certain_writes = dict()
self._possible_writes = dict()

self.guid = generate_element_id(self)

Expand Down
6 changes: 3 additions & 3 deletions dace/sdfg/utils.py
Original file line number Diff line number Diff line change
Expand Up @@ -867,16 +867,16 @@ def get_view_edge(state: SDFGState, view: nd.AccessNode) -> gr.MultiConnectorEdg
return None

in_edge = in_edges[0]
out_edge = out_edges[0]
out_edge = out_edges[0] if len(out_edges) > 0 else None

# If there is one incoming and one outgoing edge, and one leads to a code
# node, the one that leads to an access node is the viewed data.
inmpath = state.memlet_path(in_edge)
outmpath = state.memlet_path(out_edge)
outmpath = state.memlet_path(out_edge) if out_edge else None
src_is_data, dst_is_data = False, False
if isinstance(inmpath[0].src, nd.AccessNode):
src_is_data = True
if isinstance(outmpath[-1].dst, nd.AccessNode):
if outmpath and isinstance(outmpath[-1].dst, nd.AccessNode):
dst_is_data = True

if src_is_data and not dst_is_data:
Expand Down
Loading