Skip to content

Releases: dora-rs/dora

v0.3.5

03 Jul 16:46
@haixuanTao haixuanTao
v0.3.5
110ce99
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Learn about vigilant mode.
Compare
Choose a tag to compare
v0.3.5

dora 0.3.5 Release

Better Error Logging and Graceful Stopping

Before

018ffe79-1c0c-7b68-8886-d8cff079c232
  2024-06-09T19:28:22.877741Z ERROR dora_daemon::spawn: 018ffe79-1c0c-7b68-8886-d8cff079c232/webcam: 
Traceback (most recent call last):
  File "/home/peter/Documents/work/dora/examples/python-dataflow/webcam.py", line 11, in <module>
    node = Node()
RuntimeError: failed to init event stream

Caused by:
    subscribe failed: Some nodes exited before subscribing to dora: {NodeId("plot")}

    This is typically happens when an initialization error occurs
                    in the node or operator. To check the output of the failed
                    nodes, run `dora logs 018ffe79-1c0c-7b68-8886-d8cff079c232 plot`.

Location:
    apis/rust/node/src/event_stream/mod.rs:90:17

    at binaries/daemon/src/spawn.rs:371


  2024-06-09T19:28:23.263843Z ERROR dora_daemon: 
    018ffe79-1c0c-7b68-8886-d8cff079c232/object_detection failed with exit code 1.

    Check logs using: dora logs 018ffe79-1c0c-7b68-8886-d8cff079c232 object_detection
                            
    at binaries/daemon/src/lib.rs:1077

  2024-06-09T19:28:23.264003Z ERROR dora_coordinator: some nodes failed:
                            
  - webcam: 
    018ffe79-1c0c-7b68-8886-d8cff079c232/webcam failed with exit code 1.

    Check logs using: dora logs 018ffe79-1c0c-7b68-8886-d8cff079c232 webcam
                            

Location:
    /home/peter/Documents/work/dora/binaries/coordinator/src/listener.rs:90:56
    at binaries/coordinator/src/lib.rs:279

dataflow failed: errors occurred in dataflow 018ffe79-1c0c-7b68-8886-d8cff079c232:
- machine ``:
    some nodes failed:
      - webcam: 
        018ffe79-1c0c-7b68-8886-d8cff079c232/webcam failed with exit code 1.
    
        Check logs using: dora logs 018ffe79-1c0c-7b68-8886-d8cff079c232 webcam
                                
    
    Location:
        /home/peter/Documents/work/dora/binaries/coordinator/src/listener.rs:90:56

After

01907807-9461-738c-a503-c92b82e0d4c2
attaching to dataflow (use `--detach` to run in background)
ERROR webcam: exited with code 1 with stderr output:
---------------------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/peter/Documents/work/dora/examples/python-dataflow/webcam.py", line 12, in <module>
    assert False, "could not start dataflow"
AssertionError: could not start dataflow
---------------------------------------------------------------------------------

INFO  plot: node finished successfully
INFO  object_detection: node finished successfully


[ERROR]
Dataflow 01907807-9461-738c-a503-c92b82e0d4c2 failed:

Node `webcam` failed: exited with code 1 with stderr output:
---------------------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/peter/Documents/work/dora/examples/python-dataflow/webcam.py", line 12, in <module>
    assert False, "could not start dataflow"
AssertionError: could not start dataflow
---------------------------------------------------------------------------------

Dynamic Node

We're introducing dynamic node, to run node in a native way, whether it is by calling python, jupyter or cargo. All you have to do is specify that the node is dynamic in the dataflow and specify the node id within the node code and you will be able to run the node natively without using dora start.

Warning

We have noticed some bugs on MacOS/Windows with dynamic node: ##575

Getting Started

You can try minimal example:

cd examples/python-dataflow
dora start dataflow_dynamic.yml 
# In another terminal
python plot_dynamic.py
  • Then starting the dataflow
dora start dataflow.yaml
  • Then using one of the following method to start the node:

Using interactive python:

$ python
>>> from dora import Node
>>> node = Node("node_0")
>>> event = node.next()
>>> event
{'id': 'tick', 'type': 'INPUT', 'value': <pyarrow.lib.NullArray object at 0x7bda86924460>
0 nulls, 'kind': 'dora', 'metadata': {'open_telemetry_context': ''}}

Using Python

$ python my_script.py # <-- node = Node("node_0")
# Log will appear in your terminal

Using Rust

$ cargo run my_node # <-- DoraNode::init_from_node_id(node_id);
# Log will appear in your terminal

Using a jupyter notebook

python

Made all dora python object representable in python

All dora-rs now implements: __str__, __repr__, __dir__, __dict__

Meaning that from Python, you will always be able to represent dora object as a Python Object.

>>> node
Node()
>>> event = node.next()
>>> event
{'id': 'tick', 'type': 'INPUT', 'value': <pyarrow.lib.NullArray object at 0x7bda86924460>
0 nulls, 'kind': 'dora', 'metadata': {'open_telemetry_context': ''}}
>>>
>>> from dora import Ros2NodeOptions, Ros2QosPolicies
>>>
>>> policies = Ros2QosPolicies()
>>> policies
Ros2QosPolicies(durability=Ros2Durability.Volatile, liveliness=Ros2Liveliness.Automatic, lease_duration=inf, reliable=false, max_blocking_time=0.0, keep_all=false, keep_last=1)
>>> policies.__dict__
{'keep_all': False, 'keep_last': 1, 'liveliness': Ros2Liveliness.Automatic, 'durability': Ros2Durability.Volatile, 'reliable': False, 'max_blocking_time': 0.0, 'lease_duration': inf}

Thanks a million to @EricLBuehler!

Simplified the dataflow

We removed one level of hierarchy by removing custom that was referencing custom node.

We also use the keyword path instead of source that might be a bit too confusing.

This change is backward compatible and you can keep the state of your previous dataflow.

Before:

  - id: idefics2
    custom:
      source: ../nodes/idefics2_node_demo.py
      inputs:
        text: whisper/text_llm
      outputs:
        - speak
        - control

After:

  - id: idefics2
    path: ../nodes/idefics2_node_demo.py
    inputs:
      text: whisper/text_llm
    outputs:
      - speak
      - control

Enabled dora on multiple machines

We can now experiment dora on different machine!

You can getting started with our multi-daemon example

A very minimalist setup would look like this

# On machine A
dora coordinator
dora daemon --machine-id A

# On machine B
dora daemon --machine-id B --coordinator-addr <COORDINATOR_IP>:53290

# On machine A or B
dora start ./examples/multiple-daemons/dataflow.yml
nodes:
  - id: rust-node
    _unstable_deploy:
      machine: A
    path: abs/path/to/target/debug/rust-node
    inputs:
      tick: dora/timer/millis/10
    outputs:
      - random

  - id: rust-sink
    _unstable_deploy:
      machine: B
    path: abs/path/to/target/debug/rust-sink
    inputs:
      message: rust-node/random

Thanks a million to @Gege-Wang and @XxChang!

I would also thanks @Michael-J-Ward and @LyonRust for their help on this release as well!!!

Full Changelog

Read more

Contributors

phil-opp, LyonRust, and 6 other contributors
Assets 17
Loading

v0.3.5-rc1

03 Jul 16:10
@haixuanTao haixuanTao
v0.3.5-rc1
4f9cf05
Compare
Choose a tag to compare
v0.3.5-rc1 Pre-release
Pre-release 
bumping to v0.3.5-rc1
Assets 17
Loading

v0.3.5-rc0

26 Jun 16:42
@haixuanTao haixuanTao
v0.3.5-rc0
257c889
Compare
Choose a tag to compare
v0.3.5-rc0 Pre-release
Pre-release 
Bump dora version to 0.3.5-rc0
Assets 14
Loading

v0.3.4

20 May 16:37
@haixuanTao haixuanTao
v0.3.4
dae290f
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Learn about vigilant mode.
Compare
Choose a tag to compare
v0.3.4

What's Changed

New Contributors

Full Changelog: v0.3.3...v0.3.4

Contributors

phil-opp, Michael-J-Ward, and haixuanTao
Assets 14
Loading

v0.3.4-rc2

17 May 14:11
@haixuanTao haixuanTao
v0.3.4-rc2
436fa99
Compare
Choose a tag to compare
v0.3.4-rc2 Pre-release
Pre-release 
Bumping dora to v0.3.4-rc2
Assets 14
Loading

v0.3.4-rc1

17 May 13:19
@haixuanTao haixuanTao
v0.3.4-rc1
bbbf129
Compare
Choose a tag to compare
v0.3.4-rc1 Pre-release
Pre-release

What's Changed

New Contributors

Full Changelog: v0.3.3...v0.3.4

Contributors

phil-opp, Michael-J-Ward, and haixuanTao
Assets 12
Loading

v0.3.3

11 Apr 13:45
@haixuanTao haixuanTao
v0.3.3
6754c9f
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Learn about vigilant mode.
Compare
Choose a tag to compare
v0.3.3

What's Changed

New Contributors

Full Changelog: v0.3.2...v0.3.3

Contributors

phil-opp, bobd988, and 4 other contributors
Assets 14
Loading

v0.3.3-rc

08 Apr 07:45
@haixuanTao haixuanTao
v0.3.3-rc
3551ab8
Compare
Choose a tag to compare
v0.3.3-rc Pre-release
Pre-release 
Bump to version 0.3.3-rc1
Assets 14
Loading

v0.3.3-rc1

08 Apr 07:30
@haixuanTao haixuanTao
v0.3.3-rc1
cafa0b6
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Learn about vigilant mode.
Compare
Choose a tag to compare
v0.3.3-rc1 Pre-release
Pre-release 
Merge pull request #451 from dora-rs/llm-example

Llm example
Assets 14
Loading

v0.3.2

29 Jan 13:34
@haixuanTao haixuanTao
v0.3.2
96074a5
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Learn about vigilant mode.
Compare
Choose a tag to compare
v0.3.2

Features

Minor

Full Changelog: v0.3.1...v0.3.2

Contributors

phil-opp, haixuanTao, and dependabot
Assets 14
Loading