Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create a dora-timer tool nodes #488

Open
haixuanTao opened this issue Apr 23, 2024 · 1 comment
Open

Create a dora-timer tool nodes #488

haixuanTao opened this issue Apr 23, 2024 · 1 comment

Comments

@haixuanTao
Copy link
Collaborator

We currrently have a very bespoke implementation of a timer within dora, that is if we see dora/timer/secs/1, we spawn a timer node inside of the graph.

The thing is that this pattern is not really explicit and we should probably rewrite this piece to have an explicit node that is spawned in a way that is clear for the users. We would be able to also make it more configurable such as adding a maximum duration of ticking.

This would also make it more explicit if there is multiple machine as each machine would not tick at the same time.

This would also reduce the complexity of the deserialization of our dataflow.

@phil-opp
Copy link
Collaborator

Providing the ticks directly by dora has some advantages:

  • no need to compile and distribute an extra node -> the timer is always available
  • we can generate the tick timer lazily based on the requirements
    • a separate timer node requires either manual modification to adjust the interval or sends lots of unneeded outputs
  • the ticks are always generated on the local machine, even when a dataflow is split across multiple machines
    • a separate timer node would require manual placement
    • remote timers would be less reliable and less deterministic

Also, we will probably need some sort of "special dora-provided inputs" in the future for features such as deadlines. I think that a special dora/ input namespace is a good way to implement this. Of course, we would need to properly document these special inputs.

(As a side node, maybe the input name 'timer' doesn't make it clear enough that it results in a regular tick, not just a single event. Maybe a name like "interval" would make this clearer.)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants