Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Supports over window on both limited and timed window in SparkProcessor #231

Open
yunfengzhou-hub opened this issue Sep 18, 2023 · 0 comments
Labels
type:feature New feature or request

Comments

@yunfengzhou-hub
Copy link
Collaborator

yunfengzhou-hub commented Sep 18, 2023

over_window_utils.py:

def _get_spark_window_spec(
    over_window_descriptor: "OverWindowDescriptor",
) -> WindowSpec:
    if (
        over_window_descriptor.limit is not None
        and over_window_descriptor.window_size is not None
    ):
        # TODO Supports aggregations on both limited and timed window
        raise FeathubException(
            "You cannot set window_size and limit of over window at the same time."
        )

test_spark_processor.py

    # TODO: Add back following test cases after SparkProcessor supports aggregations
    #  on both limited and timed window
    def test_over_window_on_join_field(self):
        pass
@yunfengzhou-hub yunfengzhou-hub added priority:major type:improvement improvement to existing functionality type:feature New feature or request and removed type:improvement improvement to existing functionality priority:major labels Sep 18, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type:feature New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant