CHANGELOG.md

Changelog

v1.2.1 (2024-05-07)

• 89ea33f1 Update manifests to v1.2.1
• 05610ad3 fix: height fixes to render pipeline view (#1721)

Contributors

• Vedant Gupta
• Yashash H L

v1.2.0 (2024-05-03)

• 636ef873 Update manifests to v1.2.0
• c9fc458b fix: summary bar overlay fix for plugin (#1710)
• 120244a3 chore(deps): bump ejs from 3.1.9 to 3.1.10 in /ui (#1711)

Contributors

• Vedant Gupta
• Yashash H L
• dependabot[bot]

v1.2.0-rc5 (2024-04-24)

• 2780c8e7 Update manifests to v1.2.0-rc5
• 59e4b453 doc: add "nav" for fallback-sink (#1694)
• df40e089 doc: reduce streaming (#1689)
• 8a6872e8 doc: Fallback Sink (#1691)
• 746ddb0a chore(deps): bump golang.org/x/net from 0.22.0 to 0.23.0 (#1692)
• e21ac91c doc: session doc (#1650)

Contributors

• Derek Wang
• Vigith Maurice
• Yashash H L
• dependabot[bot]

v1.2.0-rc4 (2024-04-18)

• 211bfaca Update manifests to v1.2.0-rc4
• 31e1a498 fix: version downgrade for monaco-editor (#1673)
• c8634256 feat: Fallback Sink (#1669)
• d68a3468 fix: routing fixes (#1671)
• 680e5d4b feat: controller change for fallback sink (#1664)
• 2faf759b Enable cors for numaflow api (#1631)
• 6910744d feat: expose controller leader election duration and renew opts (#1657)

Contributors

• Ali Ibrahim
• Derek Wang
• Vedant Gupta
• Yashash H L

v1.2.0-rc3 (2024-04-09)

• 4e172e80 Update manifests to v1.2.0-rc3
• 109582c8 fix: add headers to custom sinkrequest (#1653)
• 6e12f09e fix: pass headers to transfomer (#1651)
• 0b76352c fix: avoid publishing watermarks for duplicate messages. (#1649)
• 1717e511 fix: flaky TestDropOnFull (#1647)
• b4b21a5e fix: Dedup not working for multi-partitioned edge (#1639)
• 7a23eda0 fix: readonly view (#1640)

Contributors

• Naga
• Vedant Gupta
• Yashash H L

v1.2.0-rc2 (2024-04-03)

• 66cc4903 Update manifests to v1.2.0-rc2
• a4c1d480 feat: read only view for UI (#1628)
• 45032fc6 fix: race condition while publishing wm inside reduce (#1599)
• 3e6cd333 fix: bug in late message handling for sliding window (#1471)
• 3d82431b fix: numaflow package style fixes (#1622)
• 73e434a9 doc: add new user to the list (#1623)
• f6ed4bbe fix: watermark progression during pods creation/deletion (#1619)
• 941fc674 fix: allow pipeline to start with redis isbsvc (Fixes: #1513) (#1567)
• d1e10ffc fix: dedup in user defined source (#1613)
• e6b3d39c chore(deps): bump express from 4.18.2 to 4.19.2 in /ui (#1609)

Contributors

• Quentin FAIDIDE
• Vedant Gupta
• Yashash H L
• dependabot[bot]

v1.2.0-rc1 (2024-03-26)

• 0a1a2e8b Update manifests to v1.2.0-rc1
• aab37c6c fix: flaky e2e tests (#1590)
• 84bafd0a chore(deps): bump webpack-dev-middleware from 5.3.3 to 5.3.4 in /ui (#1595)
• 04ff0c6e fix: error message that cause Buttons overflow in the UI (#1591)
• 611bab70 feat: support headers for message (#1578)
• b394024f feat: numaflow package (#1579)
• ea55a92d chore(deps): bump follow-redirects from 1.15.4 to 1.15.6 in /ui (#1573)
• 4457ac88 feat: read Kafka header and propagate in the payload (#1565)
• c1bc119c fix(codegen): protobuf upgrade (#1558)
• 4b580b14 chore(deps): bump google.golang.org/protobuf from 1.31.0 to 1.33.0 (#1556)
• dd3cbfe3 Chore: go 1.21 and k8s 1.29 (#1555)
• dc69b29b feat: unaligned wal (#1511)
• 1844575f fix: initialize inflightAcks channel to not nil channel (#1548)
• c6e5fd55 chore(deps): bump github.com/go-jose/go-jose/v3 from 3.0.1 to 3.0.3 (#1549)
• cd05c47d fix: break from retry loop when key is not found (#1535)
• fd3f5e1a feat: noop persistence store for reduce (#1532)
• 8c2a160d fix(controller): vertex template metadata nil check (#1527)
• 5b31bac5 feat: add support for SASL SCRAM 256 and 512 for Kafka (#1518)
• 76266ef6 fix: incorrect json schema for tls config (#1520)
• 762e130c fix: podSpec incorrectly configured in case template exist (#1516)
• 37a9d5d7 add separate server info file paths for services on client side (#1494)
• f05ce9e2 fix: idle manager refactor for multi partitions (#1512)
• c58f9a1b fix: avoid panic when ctx is canceled (#1515)
• abf7baf9 fix: unknown for ISB details in pipeline card (#1497)
• 20cf66d9 feat: configure standardResources via controller configmap (#1490)
• 345e7ca7 fix: add idle handler offset nil check (#1489)
• 9954f847 feat: terminate reduce vertex pods when pausing pipeline (#1481)
• 0c53e8a0 fix(controller): incorrect cpu/mem resources calculation (#1477)
• 495d22bf feat: pipeline health status for UI (#1460)
• cd27ce60 fix: bug where dashed line is treated as permalink + spelling/formatting (#1467)
• 870d86bf feat: improve dex server (#1440)
• 85e76c7c fix: memory leak inside session windower (#1445)
• 43063576 fix: GetDownstreamEdges is not cycle safe (#1447)
• d1ad022e chore(deps): bump follow-redirects from 1.15.3 to 1.15.4 in /ui (#1448)
• 70c78e73 fix: UI Filter by status for pipelines doesn't work as expected (#1444)
• 795bef68 fix: Kafka source reads duplicated messages (#1438)
• 7fe3225c feat: enhance autoscaling peeking logic (#1432)
• 58e215e1 fix: server-secrets-init container restart (#1433)
• 9dc3bfd1 feat: update tcp client connections (#1429)
• 45c85942 feat: Session Window and Reduce Streaming (#1384)
• 38b44e69 doc: idle source (#1426)
• bca1b3b9 feat: health status implementation (#1406)
• 412bb210 chore(deps): bump golang.org/x/crypto from 0.14.0 to 0.17.0 (#1424)
• 5c734726 fix: configmap const name (#1423)

Contributors

• Abdullah Hadi
• Ali Ibrahim
• Antonino Fugazzotto
• Damien RAYMOND
• Derek Wang
• Dillen Padhiar
• Juanlu Yu
• Nishchith Shetty
• Sidhant Kohli
• Vedant Gupta
• Vigith Maurice
• Yashash H L
• akash khamkar
• dependabot[bot]

v1.1.7 (2024-03-15)

• d4e0bd28 Update manifests to v1.1.7
• e53f8584 fix: initialize inflightAcks channel to not nil channel (#1548)
• ae1ddaac chore(deps): bump github.com/go-jose/go-jose/v3 from 3.0.1 to 3.0.3 (#1549)

Contributors

• Antonino Fugazzotto
• Derek Wang
• dependabot[bot]

v1.1.6 (2024-02-27)

• 9613573e Update manifests to v1.1.6
• b8bb2336 fix(controller): vertex template metadata nil check (#1527)
• c0b3bdac feat: add support for SASL SCRAM 256 and 512 for Kafka (#1518)
• fe26f91a fix panic inside reduce after getting sigterm
• bdac5a75 fix: incorrect json schema for tls config (#1520)
• 53cfae97 fix: podSpec incorrectly configured in case template exist (#1516)
• 915c09aa fix: unknown for ISB details in pipeline card (#1497)
• c71b2aba feat: configure standardResources via controller configmap (#1490)
• ce0a3155 fix: add idle handler offset nil check (#1489)
• 8d16d499 feat: terminate reduce vertex pods when pausing pipeline (#1481)

Contributors

• Damien RAYMOND
• Derek Wang
• Dillen Padhiar
• Juanlu Yu
• Nishchith Shetty
• Vedant Gupta
• Yashash H L

v1.1.5 (2024-01-23)

• e5bcf32e Update manifests to v1.1.5
• 266cb227 fix(controller): incorrect cpu/mem resources calculation (#1477)

Contributors

• Derek Wang

v1.1.4 (2024-01-20)

• 7ffb521b Update manifests to v1.1.4
• de780b95 fix: bug in late message handling for sliding window (#1472)

Contributors

• Derek Wang
• Yashash H L

v1.1.3 (2024-01-14)

• 0b96acf9 Update manifests to v1.1.3
• 907949be fix: GetDownstreamEdges is not cycle safe (#1447)
• 1d83b51e chore(deps): bump follow-redirects from 1.15.3 to 1.15.4 in /ui (#1448)
• 855672dd fix: UI Filter by status for pipelines doesn't work as expected (#1444)
• c06de95e fix: Kafka source reads duplicated messages (#1438)
• 17c9c0e2 feat: enhance autoscaling peeking logic (#1432)

Contributors

• Derek Wang
• Juanlu Yu
• Nishchith Shetty
• akash khamkar
• dependabot[bot]

v1.1.2 (2024-01-01)

• ac716ec4 Update manifests to v1.1.2
• af17d8ce fix: server-secrets-init container restart (#1433)

Contributors

• Derek Wang
• Vedant Gupta

v1.1.1 (2023-12-21)

• 5ff77fe0 Update manifests to v1.1.1
• 5fd20ad9 chore(deps): bump golang.org/x/crypto from 0.14.0 to 0.17.0 (#1424)
• da32632c fix: configmap const name (#1423)

Contributors

• Derek Wang
• dependabot[bot]

v1.1.0 (2023-12-18)

• 07d46ca9 Update manifests to v1.1.0
• 41b8dffc feat: local user support for Numaflow (#1416)
• 818be4f2 fix: consider lastPublishedIdleWm when computed watermark is -1 (#1415)
• 263263b3 fix: access path for auth endpoints (#1403)
• 0db9cd19 feat: Generate Idle Watermark if the source is idling (#1385)
• 6eb25c25 fix: include dropped messages in source watermark calculation (#1404)
• 1eee1942 chore(deps): bump @adobe/css-tools from 4.3.1 to 4.3.2 in /ui (#1400)
• 0a2ff566 fix: updated access path config for root path (#1397)
• 024597d5 feat: improve numaflow k8s events (#1393)
• 83fb9068 fix: update numaflow-go version (#1387)
• 9b0fadb6 fix: access path for api/v1 route (#1388)
• 96cfa555 fix: dropped messages should not be considered for watermark propagation (#1386)
• 5bf63076 chore(deps): bump github.com/go-jose/go-jose/v3 from 3.0.0 to 3.0.1 (#1383)
• 0c82ee0f refactor: move udf forwarder to the right dir (#1381)
• 3a77ed42 fix: add pipeline update validation checks (#1379)
• 44a38f4a fix: disallow updating an existing isbsvc's persistence strategy (#1376)
• 82538b6a fix: add more checks to isbsvc validation (#1358)
• 118c309d fix: non-ack failed offsets (#1370)
• 3456f714 feat: validate patched data for pipelines (#1349)
• e75e9581 Add Atlan into USERS.md (#1351)
• 483c0185 Unit tests UI (#1348)
• d0ae1484 fix(SERVER): remove unknown filter (#1346)
• 7ac77521 Upstreammain (#1345)
• f1af1f02 fix: rc-4 bug bash bug fixes (#1343)
• f6edd5ae fix(UI): rc-0.4 fixes (#1342)
• ecbd489f fix(SERVER): fix styles for ISB cards
• d4836e9e fix(SERVER): fix styles for ISB cards
• 953feb67 fix(SERVER): fix for pagination issue
• d484393d fix(SERVER): namespace inout filtering space alignment
• e9360127 fix(SERVER): pipeline card style fix
• c24b91ce fix: block pipeline load post update (#1333)
• e928be34 fix: full isb spec in edit (#1331)
• d5c3b07a RC2.0 UI fixes (#1329)
• f4211fb1 feat: container for generator vertices (#1321)
• f4354af3 fix: create isb should move to isb tab (#1323)
• 24afc5e1 fix: max lag (#1319)
• 42f81df4 feat: cache daemon client for each pipeline (#1276)
• a7270426 fix: rc2 UI fixes (#1317)
• 01bf1854 feat: added filtering based on number, status and health of pipelines… (#1312)
• 91b0effa fix: user identity cookie max age (#1316)
• 5c999b63 feat: add scopes to authorization (#1288)
• e119a0ee fix: logout fix (#1310)
• 449dfd3a fix: Split cookie to meet the cookie length requirement (#1305)
• 87c4c1e1 chore(deps): bump github.com/nats-io/nats-server/v2 from 2.10.3 to 2.10.4 (#1307)
• ea577452 add tooltips (#1289)
• 93eec963 fix: fixed the timer not clearing issue (#1303)
• c7bdbda5 feat: k8s events filtering and cluster summary card fixes (#1297)
• a558b229 fix: graph overflow with large height (#1301)
• 73ffaa86 doc: need metrics server (#1296)
• 32416bf2 refactor: unified metrics names for forwarders (#1290)
• 17b7b313 feat: added separate colors for sideInput and dynamic legend (#1292)
• 6efab64e feat: add tabs to display pipelines and isb services (#1293)
• 86df4a84 fix: more sidebar testing (#1287)
• d7ae1d36 refactor: create interfaces for AuthN and AuthZ (#1286)
• 65aca23f fix: tests for utils (#1283)

Contributors

• Bradley Behnke
• Chandan Kumar
• Darshan Simha
• Derek Wang
• Dillen Padhiar
• Juanlu Yu
• Keran Yang
• Madusudanan.B.N
• Shakira M
• Sidhant Kohli
• Vedant Gupta
• Vigith Maurice
• Yashash H L
• dependabot[bot]
• mshakira

v1.0.0 (2023-11-03)

• 78134e8f Update manifests to v1.0.0
• 660ff501 fix: rc-4 bug bash bug fixes (#1343)
• ceed5def fix(UI): rc-0.4 fixes (#1342)

Contributors

• Darshan Simha
• Derek Wang
• mshakira

v1.0.0-rc4 (2023-11-03)

• e94b563f Update manifests to v1.0.0-rc4
• 14757afc fix(SERVER): fix styles for ISB cards
• fc90920b fix(SERVER): fix styles for ISB cards
• 8bf275bd fix(SERVER): fix for pagination issue
• 2e8cff7a fix(SERVER): namespace inout filtering space alignment
• c77fb0fc fix(SERVER): pipeline card style fix
• 52d9370f fix: block pipeline load post update (#1333)
• 2307660f fix: full isb spec in edit (#1331)
• a70c77b9 RC2.0 UI fixes (#1329)
• 02281f38 feat: container for generator vertices (#1321)
• feb4977a fix: create isb should move to isb tab (#1323)
• e05132ef fix: max lag (#1319)

Contributors

• Bradley Behnke
• Darshan Simha
• Derek Wang
• Juanlu Yu
• Shakira M
• Vedant Gupta
• mshakira

v1.0.0-rc3 (2023-11-01)

• 6ab96b18 Update manifests to v1.0.0-rc3
• 4ffda383 feat: cache daemon client for each pipeline (#1276)
• 5a7d739e fix: rc2 UI fixes (#1317)
• cbd810bc feat: added filtering based on number, status and health of pipelines… (#1312)
• 491c8786 fix: user identity cookie max age (#1316)

Contributors

• Darshan Simha
• Derek Wang
• Dillen Padhiar
• Juanlu Yu
• mshakira

v1.0.0-rc2 (2023-11-01)

• 8a7dc592 Update manifests to v1.0.0-rc2
• 8ed52b39 feat: add scopes to authorization (#1288)
• eebe623f fix: logout fix (#1310)
• 9c34c27f fix: Split cookie to meet the cookie length requirement (#1305)
• c6073b17 chore(deps): bump github.com/nats-io/nats-server/v2 from 2.10.3 to 2.10.4 (#1307)
• b1bb6575 add tooltips (#1289)
• 40ca1705 fix: fixed the timer not clearing issue (#1303)
• 2a91668d feat: k8s events filtering and cluster summary card fixes (#1297)
• 4929a1e4 fix: graph overflow with large height (#1301)
• 8cc04a77 doc: need metrics server (#1296)
• 05b19400 refactor: unified metrics names for forwarders (#1290)
• 5475e682 feat: added separate colors for sideInput and dynamic legend (#1292)
• 052201e0 feat: add tabs to display pipelines and isb services (#1293)
• 9e1ea4f0 fix: more sidebar testing (#1287)
• d018af50 refactor: create interfaces for AuthN and AuthZ (#1286)
• ada5ea4d fix: tests for utils (#1283)

Contributors

• Bradley Behnke
• Darshan Simha
• Derek Wang
• Juanlu Yu
• Keran Yang
• Sidhant Kohli
• Vedant Gupta
• Vigith Maurice
• dependabot[bot]
• mshakira

v1.0.0-rc1 (2023-10-26)

• 0ff1f58f Update manifests to v1.0.0-rc1
• 16c3fc3c fix: incorrect image version for namespaced numaflow-server (#1282)
• 18d62935 fix: ISBCreate test and fetch mock setup (#1279)
• 9325140c fix: update dex to work with basehref (#1278)
• 36610a5a feat: AuthN/AuthZ for Numaflow UI (#1234)
• 730552e8 fix(doc): hpa api version (#1274)
• c103427b fix: updating example.md (#1262)
• c1725b18 Feat/side input tests (#1257)
• 5554bd65 chore(deps): bump google.golang.org/grpc from 1.57.0 to 1.57.1 (#1268)
• b86b2254 fix(UI): pod selection fix (#1266)
• 07a7b8ea doc: Numaflow high level security (#1264)
• 308acf2f fix: fixed the ns-summary page to allow creation of pipeline when no … (#1263)
• ce737732 Unit tests graph page (#1250)
• 639a9364 feat: Add e2e test for map sideinput, Fixes #1192 (#1211)
• 7b32af34 feat: get current status of ISB service (#1199)
• 5c7fc90e Summary view fixes (#1253)
• 53aed68d chore(deps): bump github.com/nats-io/nats-server/v2 from 2.9.19 to 2.9.23 (#1232)
• 30382995 fix: updated div's with box and removed unwanted css (#1236)
• 8b50f36c chore(deps): bump @babel/traverse from 7.23.0 to 7.23.2 in /ui (#1221)
• 7a3ca76a Update kafka.md (#1218)
• e49a1811 Update generator.md (#1217)
• 9785eb07 Update map.md (#1219)
• a874478a feat: UI 1.0 CRUD (#1181)
• d34fcc47 fix: get isbsvc kind apiversion (#1220)
• 74f4d980 Update kafka.md (#1215)
• 7d5fe51f Update generator.md (#1214)
• b6adac15 Update overview.md (#1213)
• 0748449d feat: add udsource python e2e (#1204)
• fea29657 doc: roadmap (#1208)
• d7630553 Namespace card status bar changes 0.11 (#1206)
• 19a523aa feat: added unit tests for PipelineCard component (#1205)
• 6a50f130 pods component error fix (#1203)
• a07e78f7 fix: updated image styles to a class (#1202)
• 71c048eb feat: Changed the status bar component to an icon based component (#1198)
• 13264e3f fix(SERVER): restructure pod details component (#1189)
• 1bbdcad8 feat: updated the legend to a collapsible one on the top left (#1196)
• 910243bd chore(deps): bump golang.org/x/net from 0.12.0 to 0.17.0 (#1190)
• 57af20e5 feat: API Delete ISBSVC validation (#1182)
• 2e1fd701 added BCubed to the user list (#1184)
• b962f8e3 feat: add timeout for pausing pipeline. Fixes #992 (#1138)
• ef62c5ca feat: Jetstream support for replica of 1 Fixes #944 (#1177)
• 80294989 doc: minor clean up of JOIN doc (#1175)
• d4b5f1b2 fix: incorrect side inputs watch logic (#1164)
• 00e6b6ae fix: not considered as back pressured when onFull is discardLatest (#1153)
• 8b0d8cef fix(SERVER): handle states when status is unknown (#1154)
• 31b1aacd fix(SERVER): fix key warning (#1152)

Contributors

• Bradley Behnke
• Caroline Dikibo
• Chandan Kumar
• Darshan Simha
• Darshan Simha U
• Dennis Sosa
• Derek Wang
• Dillen Padhiar
• Joel Millage
• Jorvaulx
• Juanlu Yu
• Kayla Nussbaum
• Shubham Dixit
• Sidhant Kohli
• TASNEEM KOUSHAR
• Vigith Maurice
• aruwanip
• bpcarey01
• dependabot[bot]
• mshakira

v0.11.0 (2023-10-13)

• fbf51b2d Update manifests to v0.11.0
• f33d614f Namespace card status bar changes 0.11 (#1206)
• 391b75dc feat: added unit tests for PipelineCard component (#1205)
• 7bcc1f27 pods component error fix (#1203)
• df28f937 fix: updated image styles to a class (#1202)
• bbb7db77 feat: Changed the status bar component to an icon based component (#1198)
• f6f20d6c fix(SERVER): restructure pod details component (#1189)
• 96a6002e feat: updated the legend to a collapsible one on the top left (#1196)
• 575605b4 chore(deps): bump golang.org/x/net from 0.12.0 to 0.17.0 (#1190)
• bfec4fc5 added BCubed to the user list (#1184)
• 1e0b25fd feat: add timeout for pausing pipeline. Fixes #992 (#1138)
• ae232764 feat: Jetstream support for replica of 1 Fixes #944 (#1177)
• 1e405b37 doc: minor clean up of JOIN doc (#1175)
• 77c01811 fix: incorrect side inputs watch logic (#1164)

Contributors

• Darshan Simha U
• Derek Wang
• Dillen Padhiar
• Joel Millage
• Vigith Maurice
• dependabot[bot]
• mshakira

v0.11.0-rc2 (2023-10-03)

• 8ae28af3 Update manifests to v0.11.0-rc2
• f74bde39 fix: not considered as back pressured when onFull is discardLatest (#1153)
• 689aa295 fix(SERVER): handle states when status is unknown (#1154)
• f83ea00a fix(SERVER): fix key warning (#1152)

Contributors

• Derek Wang
• mshakira

v0.11.0-rc1 (2023-10-03)

• 1d89a12b Update manifests to v0.11.0-rc1
• 882bcef3 feat: Numaflow UI 1.0 (#1077)
• a7adee1e fix: treat ALL user-defined source vertices as scalable (#1132)
• 408ff389 feat: add doc link checker (#1130)
• dae11a7f extracting redis streams source (#1113)
• 9d65e229 refactor: shared kubeconfig util (#1095)
• bb6f29b2 fix: wrong api group in webhook rbac settings (#1086)
• 4150eb28 feat: set correct number of replices for some types of vertices after resuming pipeline (#1085)
• 2182b83b chore(deps): bump graphql from 16.6.0 to 16.8.1 in /ui (#1078)
• b2a377ca feat: add forest validation for pipelines. Fixes #1002 (#1063)
• 9035ba8b feat: implement pipeline validation for unsupported states (#1043)
• 4369d65f fix: message count read in forwarder (#1030)
• 8d52d78b fix(ci): Wrong test pod image tag used for running CI on release branch (#1041)
• 6ed2c381 fix: allow udsource pending api to return negative count to indicate PendingNotAvailable (#1040)
• c12e9094 refactor: re-arrange some of the rater implementations (#1036)
• abe332e6 fix(docs): file names and links for side inputs (#1037)
• 20fb7cc2 doc: Add side input docs (#1029)
• b373406c doc: kill-switch when buffer is full (#1034)
• 9f5127f0 fix: Idle handler refactor (#1021)
• b1f8b026 fix: calculate processing rate for sink vertices (#1025)
• c01a1406 feat: colored logs for UI with toggle for logs order (#1022)

Contributors

• Derek Wang
• Dillen Padhiar
• Juanlu Yu
• Julie Vogelman
• Keran Yang
• Sidhant Kohli
• Tianchu Zhao
• Vedant Gupta
• Vigith Maurice
• dependabot[bot]

v0.10.1 (2023-09-14)

• e5e2b619 Update manifests to v0.10.1
• 4702849c feat: implement pipeline validation for unsupported states (#1043)
• e36b3c6c fix: message count read in forwarder (#1030)
• 01d9abee fix: allow udsource pending api to return negative count to indicate PendingNotAvailable (#1040)
• 8cbb4d67 refactor: re-arrange some of the rater implementations (#1036)
• b511effc fix(docs): file names and links for side inputs (#1037)
• 5b083c2d doc: Add side input docs (#1029)
• 8d5c56f8 doc: kill-switch when buffer is full (#1034)
• 73db23a3 fix: Idle handler refactor (#1021)
• 7119ed96 fix: calculate processing rate for sink vertices (#1025)
• 6408137d feat: colored logs for UI with toggle for logs order (#1022)
• 2311260c fix(ci): Wrong test pod image tag used for running CI on release branch (#1041)

Contributors

• Derek Wang
• Dillen Padhiar
• Juanlu Yu
• Keran Yang
• Sidhant Kohli
• Vedant Gupta
• Vigith Maurice

v0.10.0 (2023-09-05)

• 10d1cfde Update manifests to v0.10.0
• 6f7c1f4a fix: seg fault inside controller (#1016)
• c2fdef16 fix: reconcile headless services before pods (#1014)
• 7d8b9087 fix: print version info when starting (#1013)
• 247b89ed feat: join vertex UI support (#1010)
• aabb8af0 feat: scaleUpCooldownSeconds and scaleDownCooldownSeconds to replace cooldownSeconds (#1008)
• ad647ab7 chore(deps): bump @adobe/css-tools from 4.2.0 to 4.3.1 in /ui (#1005)
• 92fbf7f1 fix: avoid unwanted watcher creation and reduce being stuck with udf is restarted (#999)
• bac06df0 fix: missing edges on UI (#998)
• f90d4fe7 feat: Add side input sdkclient and grpc (#953)
• d99480a8 feat: implement user-defined source (#980)
• 70685902 fix: send keys for udsink (#979)
• 8f32b9a3 fix bulleted list (#977)
• 1f33bf8b refactor: build wmstore and wmstorewatcher directly, and remove some unnecessary fields (#970)
• 4cea3444 feat: add vertex template to pipeline spec (#947)
• 4a4ed927 feat: Add side-input initializer and synchronizer (#912)
• d10f36e6 fix: npe when the ctx is canceled inside kv watcher (#942)
• 6b1b3337 fix: retry logic for fetching last updated kv time (#939)
• e3da4a3e fix: close the watermark fetcher and publishers after all the forwarders exit (#921)
• 2d6112bf Pipelines with Cycles: e2e testing, and pipeline validation (#920)
• 5e0bf77e docs quick fixes (#919)
• 0f8f7a17 docs updates (#917)
• b55566b8 feat: watermark delay in tooltip (#910)
• 667ada75 fix: removing WIP tag (#914)
• 872aa864 feat: emit k8s events for controller messages. Fixes #856 (#901)
• 0fbdb7ab fix: avoid potential deadlocks when operating UniqueStringList (#905)
• 2c85ec43 refactor: avoid exposing internal data structures of pod tracker to the rater (#902)
• 7e86306b feat: Join Vertex (#875)
• 85360f65 fix: stabilize nats connection (#889)
• d4f8f594 doc: Update multi partition doc (#898)
• 404672d6 fix: Reduce idle WM unit test fix (#897)
• a1bbdedf updated default version of Redis used for e2e (#891)
• 85ee4b0d fix TestBuiltinEventTimeExtractor (#885)
• f3e1044e chore(deps): bump word-wrap from 1.2.3 to 1.2.4 in /ui (#881)
• a02f29a7 fix: remove retry when the processor is not found. (#868)
• cfdeaa8a refactor: create a new data forwarder dedicated for source (#874)
• 6d14998a feat: controller changes for Side Inputs support (#866)
• 92db62a9 fix: highlight edge when buffer is full (#869)
• 9c4e83c0 fix: minor ui bugs (#861)
• b970b4cc fix: release script for validating webhook (#860)
• 7684aada fix: use windower to fetch next window yet to be closed (#850)
• 609d8b3c feat: implement optional validation webhook. Fixes #817. (#832)
• 3ae1cedb chore(deps): bump semver from 6.3.0 to 6.3.1 in /ui (#845)

Contributors

• Derek Wang
• Dillen Padhiar
• Jason Zesheng Chen
• Juanlu Yu
• Julie Vogelman
• Keran Yang
• RohanAshar
• Sidhant Kohli
• Vedant Gupta
• Vigith Maurice
• Yashash H L
• dependabot[bot]

v0.9.3 (2023-09-05)

• 6141719f Update manifests to v0.9.3
• 022f8bfa fix: seg fault inside controller (#1016)

Contributors

• Derek Wang
• Yashash H L

v0.9.2 (2023-08-23)

• 6f81361f Update manifests to v0.9.2
• 66c32197 fix: error when kv_watch with no keys (#981)

Contributors

• Derek Wang

v0.9.1 (2023-08-11)

• 4cbd729c Update manifests to v0.9.1
• aa5e8ae3 fix: npe when the ctx is canceled inside kv watcher (#942)
• e5a5cd6c feat: watermark delay in tooltip (#910)

Contributors

• Derek Wang
• Vedant Gupta
• Yashash H L

v0.9.0 (2023-08-02)

• 8e4b6ca1 Update manifests to v0.9.0
• 7424ae50 feat: emit k8s events for controller messages. Fixes #856 (#901)
• d0bfac6d fix: avoid potential deadlocks when operating UniqueStringList (#905)
• 75c7f975 fix: stabilize nats connection (#889)
• 0db1238d fix: Reduce idle WM unit test fix (#897)
• 5073f1c8 fix TestBuiltinEventTimeExtractor (#885)
• 33b7d1d0 fix: remove retry when the processor is not found. (#868)
• 89b2d1c4 fix: highlight edge when buffer is full (#869)
• 8d49c0f6 fix: minor ui bugs (#861)
• 9478e302 fix: release script for validating webhook (#860)

Contributors

• Derek Wang
• Dillen Padhiar
• Juanlu Yu
• Keran Yang
• Vedant Gupta
• Yashash H L

v0.9.0-rc2 (2023-07-13)

• d0df669a Update manifests to v0.9.0-rc2
• c8aaeff8 fix: use windower to fetch next window yet to be closed (#850)
• bcda8dcf feat: implement optional validation webhook. Fixes #817. (#832)
• e605504d chore(deps): bump semver from 6.3.0 to 6.3.1 in /ui (#845)

Contributors

• Derek Wang
• Dillen Padhiar
• Yashash H L
• dependabot[bot]

v0.9.0-rc1 (2023-07-11)

• 40f45410 Update manifests to v0.9.0-rc1
• f5276dbb fix: pod tracker logic for calculating processing rate (#838)
• db06e7e4 chore(deps): bump tough-cookie from 4.1.2 to 4.1.3 in /ui (#839)
• b660b6d9 fix: resource leak inside daemon server (#837)
• 1f19a742 feat: capability to increase max message size (#835)
• c61ce319 doc: update roadmap (#830)
• aca1c9bf feat: add stragglers (late data) into the window is window is open (#824)
• 0155b4a5 fix(docs): fixed some incorrect docs and renamed a timeExtractionFilter arg (#814)
• dd060cb8 feat: rater changes to track processing rate per partition (#805)
• 541ceb20 fix: metric to track watermark bug was wrongly tagged (#809)
• cf473151 feat: autoscaling changes to support multi partition (#806)
• 6a5ee1a5 fix: segmentation fault in daemon server (#804)
• 2ce0ac90 fix: Intermittent failure from Kafka to get consumer offsets (#803)
• 32be7fc5 feat: support UI for multipartition edges (#789)
• 97db1984 refactor: remove redundant delta calculations for rater (#795)
• 3406a130 fix: select pods not in evicted status (#786)
• a9204fbc feat: combine built-in UDTransformers for filter and eventTime assignment (#783)
• 85955e30 feat: support multi-partitioned edges (#751)
• 5ce6936d fix: duplicate ui served from gin Router (#781)
• e9ea7d85 fix: unexpected high processing rates (#780)
• f60b8ab9 chore(deps): bump github.com/gin-gonic/gin from 1.9.0 to 1.9.1 (#772)
• 466e3804 feat: gRPC error handling (#744)
• 9aff2bd8 feat: forwardAChunk to support multi partitioned edges (#757)
• f3273170 fix: pipeline view fix (#755)
• 037c9a61 fix: toVertexPartitions for reduce was incorrectly populated to 1 (#756)
• 099b914a feat: use metrics to calculate vertex processing rate (#743)
• 59880e97 feat: enable streaming message to next vertex when batch size is 1 (#709)
• 160b9414 fix: use int32 for message length (#750)
• e383ee2f feat: using one bucket for partitioned reduce watermark propagation (#742)
• ba1f493d fix(test): flakey test (#738)
• f0c83291 refactor: buffer, edge, bucket (#733)
• eb9a7c4c feat: change baseHref for Numaflow UI. Fixes #375. (#698)
• b1f639e7 fix: let kafka source crash and restart when there is any server side error (#735)
• c8aafc7f feat: Autoscale for Redis Streams Source (#726)
• d8074ec1 feat: Redis7 as an ISB svc (#717)
• 3dd2a31d chore(deps): bump github.com/gin-gonic/gin from 1.8.1 to 1.9.0 (#724)
• 15a229b5 docs(proposal): edges, buffers and buckets (#704)
• 714c8036 chore(doc): update README with demo (#718)
• 431778de doc: add overview (#713)
• f518d99c feat: allowedLateness to support late data ingestion (#703)
• 1efac426 fix: allow late message as long as window is not closed (#696)
• 3da84fa5 fix: add wal dir x permission (#689)
• 3a91cc52 chore(doc): refactor doc struct (#685)

Contributors

• Derek Wang
• Dillen Padhiar
• Juanlu Yu
• Julie Vogelman
• Keran Yang
• Vedant Gupta
• Vigith Maurice
• Yashash H L
• dependabot[bot]
• xdevxy

v0.8.1 (2023-05-30)

• 4b119387 Update manifests to v0.8.1
• 67277b79 fix: pipeline view fix (#755)
• 7cb399e9 fix: toVertexPartitions for reduce was incorrectly populated to 1 (#756)
• 16067af2 feat: use metrics to calculate vertex processing rate (#743)
• 11cd8e9f feat: enable streaming message to next vertex when batch size is 1 (#709)
• a5058840 fix: use int32 for message length (#750)
• c602a520 feat: using one bucket for partitioned reduce watermark propagation (#742)
• ccf79c6d fix(test): flakey test (#738)
• af8e3346 refactor: buffer, edge, bucket (#733)
• d57bfed4 feat: change baseHref for Numaflow UI. Fixes #375. (#698)
• 37dfae58 fix: let kafka source crash and restart when there is any server side error (#735)
• b7a0dda5 feat: Autoscale for Redis Streams Source (#726)
• 5654e0af feat: Redis7 as an ISB svc (#717)
• 9e8d8cc4 chore(deps): bump github.com/gin-gonic/gin from 1.8.1 to 1.9.0 (#724)

Contributors

• Derek Wang
• Dillen Padhiar
• Julie Vogelman
• Keran Yang
• Vedant Gupta
• Yashash H L
• dependabot[bot]
• xdevxy

v0.8.0 (2023-04-26)

• e57ca739 Update manifests to v0.8.0
• 652be8d6 feat: allowedLateness to support late data ingestion (#703)
• 8e7e3b61 fix: allow late message as long as window is not closed (#696)
• fcaed47d fix: add wal dir x permission (#689)
• aae08fa1 chore(doc): refactor doc struct (#685)

Contributors

• Derek Wang
• Vigith Maurice

v0.8.0-rc1 (2023-04-14)

• ca88313d Update manifests to v0.8.0-rc1
• b83525df feat: introducing tags for conditional forwarding (#668)
• a6e81746 feat: expose cpu/mem info to sidecar containers (#678)
• c7b853aa feat: Redis Streams source fixes (#669)
• 2f73b5b8 fix: skip empty Kafka partitions when calculating pending count (#666)
• eeb37d8b feat: support for multi keys (#658)
• 91b516b8 feat: Adds SASL (plain and gssapi) support for kafka sink (#656)
• 196f887d fix: vertex overlapping watermark (#660)
• 0db3248d feat: incremental search and namespace preview in search bar (#654)
• 46bb8750 feat: integrate serde WAL (#650)
• b0560876 fix: unit test (#653)
• 1b0ea088 feat: handle idle watermark for reduce vertex (#627)
• 33882628 feat: Redis streams source (#628)
• d85bf93f feat: Adds SASL (plain and gssapi) support for kafka source (#643)
• 60bb2bb9 feat: namespace scope api and disable namespace search on UI (#638)
• 38b5a9ec fix: GetHeadWatermark Logic (#636)
• 927b95cd feat: enable edge-level kill switch to drop messages when buffer is full, for the non-reduce forwarder (#634)
• 0c79113c fix: IdleWatermark unit test (#640)
• 924ad33e fix: desired replicas should not be greater than pending (#639)
• 150c5c23 fix: add timeout to the test (#618)
• 2f112fb2 feat: kustomize integration (#637)
• 5062aac6 fix: exclude ack pending messages (#631)
• e533ba35 feat: UI error component (#613)
• 20aaca9d fix: do not update status.replicas until pod operation succeeds (#620)
• cc62c81c feat: track and expose oldest work yet to be done to the reduce loop (#617)
• 4bbe80bb feat: handle watermark barrier for map vertex (#607)
• f9f05442 fix: corrected reduce vertex replica number. Fixes #593 (#616)
• c9815132 feat: add API for pipeline status check. Fixes #407. (#599)
• 5282766d chore(deps): bump webpack from 5.74.0 to 5.76.1 in /ui (#610)
• 927bfc04 feat: use randomized shuffle using vertex name as the seed (#601)
• d667d799 fix: ack the dropped messages as well (#603)
• 64e17d88 feat: enable controller HA (#602)
• 20ba722f feat: expose dnspolicy and dnsconfig to pod template (#598)
• abfdd78f Chore: tickgen changes to test reduce pipelines (#587)
• b2f8a12a feat: bidirectional streaming (#553)
• 148663e7 feat: use customized binary serde for nats message payload (#585)
• 8d339b68 fix: Idle watermark fix for read batch size > 0 and partial idle outgoing edges (#575)
• 87ab1e3d feat: implement watermark propagation for source data transformer (#557)
• d561867f feat: namespace search (#559)
• 1b4800af fix: refine log for buffer validation. Fixes #185 (#573)
• 4eb27eff feat: add readiness and liveness check for daemon server. Fixes #543 (#571)
• 8ae2116d feat: marshal/unmarshal binary for read message (#565)
• a0505e67 chore(deps): bump golang.org/x/net from 0.0.0-20220722155237-a158d28d115b to 0.7.0 (#568)
• fbf36894 chore(deps): bump golang.org/x/text from 0.3.7 to 0.3.8 (#567)
• 92c8009d feat: expose image pull policy to user defined containers (#563)
• 88a41c2b fix: typos in reduce examples (#556)
• 97567f3c feat: edge-watermark (#537)
• 1e06ba2f feat: enable envFrom for user defined containers (#554)
• 0dc85f69 feat: remove secret watch privilege dependency (#542)
• 8b7e397e fix: Use a copied object to update (#541)
• 943e7bd8 chore(deps): bump github.com/emicklei/go-restful from 2.9.5+incompatible to 2.16.0+incompatible (#539)
• 93753c15 feat: improve reduce performance (#501)
• b502fa93 feat: Offset time idle watermark put (#529)
• 4551505c fix: securityContext not applied to container templates (#528)
• 2727e62a feat: idle watermark v0 (#520)
• 1e34e315 feat: Reduce UI Support (#500)
• 39ecae42 feat: enable RuntimeClassName for vertex pod (#519)
• b965318d feat: add builtin filter and event time extractor for source transformer (#517)
• 077771ce chore(deps): bump ua-parser-js from 0.7.32 to 0.7.33 in /ui (#507)
• 58b12ec3 Add an e2e test for source data transformer (#505)
• 2af91933 feat: Implement source data transformer and apply to all existing sources (#487)
• a3024f4e fix: -ve metrics and return early if isLate (#495)
• 22986153 fix: JetStream context KV store/watch fix (#460)
• 2177d621 doc: reduce persistent store (#458)
• 3c621207 doc: reduce documentation (#448)
• 784fe15c chore(deps): bump json5 from 1.0.1 to 1.0.2 in /ui (#454)
• 659a98b5 refactor: simplify http request construction in test cases (#444)
• cc9c194b refactor: use exact matching instead of regex to perform e2e data validation. (#443)
• f7f712b3 doc: windowing fixed and sliding (#439)
• 9ad504b5 refactor: move redis sink resources creation to E2ESuite (#437)
• 0148258d refactor: a prototype for enhancing E2E test framework (#424)
• 8579dc67 feat: pipeline watermark (#416)

Contributors

• Alex Ang HH
• Derek Wang
• Dillen Padhiar
• Juanlu Yu
• Julie Vogelman
• Keran Yang
• Vedant Gupta
• Vigith Maurice
• Yashash H L
• ashwinidulams
• dependabot[bot]

v0.7.3 (2023-04-02)

• 68a14793 Update manifests to v0.7.3
• a17a41df feat: integrate serde WAL (#650)
• 096c6acf fix: unit test (#653)
• cce50ffd feat: handle idle watermark for reduce vertex (#627)
• 968cc5f5 feat: Redis streams source (#628)
• 38baae8a feat: Adds SASL (plain and gssapi) support for kafka source (#643)
• 563b85b1 feat: namespace scope api and disable namespace search on UI (#638)
• c8194690 fix: GetHeadWatermark Logic (#636)
• d37f4db5 feat: enable edge-level kill switch to drop messages when buffer is full, for the non-reduce forwarder (#634)
• a0dce69a fix: IdleWatermark unit test (#640)
• f840e1c3 fix: desired replicas should not be greater than pending (#639)
• 0ca5630e fix: add timeout to the test (#618)
• b049f0b4 feat: kustomize integration (#637)
• 01305ea3 fix: exclude ack pending messages (#631)
• 4ec4b3d7 feat: UI error component (#613)
• 90ca505b fix: do not update status.replicas until pod operation succeeds (#620)
• 339db43b feat: track and expose oldest work yet to be done to the reduce loop (#617)
• 1ce4c383 feat: handle watermark barrier for map vertex (#607)
• 90dbe1fa fix: corrected reduce vertex replica number. Fixes #593 (#616)
• a155f2af feat: add API for pipeline status check. Fixes #407. (#599)
• d9e3a56f chore(deps): bump webpack from 5.74.0 to 5.76.1 in /ui (#610)

Contributors

• Alex Ang HH
• Derek Wang
• Dillen Padhiar
• Juanlu Yu
• Julie Vogelman
• Keran Yang
• Vedant Gupta
• Vigith Maurice
• Yashash H L
• dependabot[bot]

v0.7.2 (2023-03-13)

• 1196a24b Update manifests to v0.7.2
• a6f64f8d feat: use randomized shuffle using vertex name as the seed (#601)
• 16b20a35 fix: ack the dropped messages as well (#603)
• 4cdef174 feat: enable controller HA (#602)
• d2e0513c feat: expose dnspolicy and dnsconfig to pod template (#598)
• 6002c829 Chore: tickgen changes to test reduce pipelines (#587)
• b1aee945 feat: bidirectional streaming (#553)
• 67fc688a feat: use customized binary serde for nats message payload (#585)
• cc104199 fix: Idle watermark fix for read batch size > 0 and partial idle outgoing edges (#575)
• df1574da feat: implement watermark propagation for source data transformer (#557)
• 45d5c396 feat: namespace search (#559)
• b14d470f fix: refine log for buffer validation. Fixes #185 (#573)
• a8e8bb15 feat: add readiness and liveness check for daemon server. Fixes #543 (#571)
• fd6acb6d feat: marshal/unmarshal binary for read message (#565)
• d1032b4c chore(deps): bump golang.org/x/net from 0.0.0-20220722155237-a158d28d115b to 0.7.0 (#568)
• fd00ebdf chore(deps): bump golang.org/x/text from 0.3.7 to 0.3.8 (#567)
• 05ec77f5 feat: expose image pull policy to user defined containers (#563)
• ecbe3a00 fix: typos in reduce examples (#556)
• 0dc3f5c6 feat: edge-watermark (#537)
• 77298c85 feat: enable envFrom for user defined containers (#554)

Contributors

• Derek Wang
• Dillen Padhiar
• Juanlu Yu
• Julie Vogelman
• Keran Yang
• Vedant Gupta
• Vigith Maurice
• Yashash H L
• ashwinidulams
• dependabot[bot]

v0.7.1 (2023-02-14)

• 92925c15 Update manifests to v0.7.1
• 2f8e147a feat: remove secret watch privilege dependency (#542)
• f8e7daae fix: Use a copied object to update (#541)
• 98de2459 chore(deps): bump github.com/emicklei/go-restful from 2.9.5+incompatible to 2.16.0+incompatible (#539)
• 0df812c1 feat: improve reduce performance (#501)
• ab49de68 feat: Offset time idle watermark put (#529)
• c0aa7c1e fix: securityContext not applied to container templates (#528)
• ac33fb02 feat: idle watermark v0 (#520)
• 2844cfb6 feat: Reduce UI Support (#500)
• e701180d feat: enable RuntimeClassName for vertex pod (#519)
• bb94f631 feat: add builtin filter and event time extractor for source transformer (#517)
• 4562196d chore(deps): bump ua-parser-js from 0.7.32 to 0.7.33 in /ui (#507)
• 764cefda Add an e2e test for source data transformer (#505)
• 7665d6ce feat: Implement source data transformer and apply to all existing sources (#487)
• d0226084 fix: -ve metrics and return early if isLate (#495)

Contributors

• Derek Wang
• Juanlu Yu
• Keran Yang
• Vedant Gupta
• Vigith Maurice
• ashwinidulams
• dependabot[bot]

v0.7.0 (2023-01-13)

• 734e5d3b Update manifests to v0.7.0
• 5d6c5336 fix: JetStream context KV store/watch fix (#460)
• d6152e77 doc: reduce persistent store (#458)
• ac77656d doc: reduce documentation (#448)
• 257356af chore(deps): bump json5 from 1.0.1 to 1.0.2 in /ui (#454)
• 7752db4b refactor: simplify http request construction in test cases (#444)
• 1a10af4c refactor: use exact matching instead of regex to perform e2e data validation. (#443)
• 2777e27a doc: windowing fixed and sliding (#439)
• 70fc008f refactor: move redis sink resources creation to E2ESuite (#437)
• 6c078b42 refactor: a prototype for enhancing E2E test framework (#424)
• e7021c9a feat: pipeline watermark (#416)

Contributors

• Derek Wang
• Juanlu Yu
• Keran Yang
• Vedant Gupta
• Vigith Maurice
• dependabot[bot]

v0.7.0-rc1 (2022-12-16)

• 71887db5 Update manifests to v0.7.0-rc1
• dda4835d feat: reduce metrics. Closes #313 (#414)
• 85dbe4d7 feat: udsink grpc stream (#421)
• fa07587f chore(doc): scope UDF under a dir (#426)
• 0a911da9 feat: sliding window. closes #339 (#354)
• a46fb964 refactor: nats/jetstream testing (#418)
• 13d95c48 feat: nats as source (#411)
• f1e7c737 fix: adding lock while discovering partitions, Closes #412 (#413)
• 3b64d674 fix(test): e2e-api-pod can not start on M1 mac (#410)
• 6504a562 fix: getWatermark to return-1 if any processor returns -1 (#402)
• d4d22041 fix: e2e testing for PBQ WAL with reduce pipeline (#393)
• 80e97850 feat: add Grafana instruction and a dashboard template. Closes #287 (#381)
• 2f94a915 fix: unit tests for replay. Closes #373 (#377)
• 8f367ab2 chore(docs): update docs (#380)
• efe4d41c fix: best effort processing during SIGTERM. Closes #371 (#372)
• 7e041d87 feat(wal): First pass to implement WAL and hook to PBQ store. (#344)
• 256e66b3 feat: watermark otwatcher enhancement (#364)
• f8170577 refactor(docs): group docs in categories (#362)
• 1a5d424f chore(deps): bump loader-utils from 2.0.3 to 2.0.4 in /ui (#356)
• 6c8f03f2 fix(controller): vertex nil check for edge listing. Fixes #352 (#353)
• f254c28a fix: data race in pbq manager. Closes #348 (#349)
• bc359457 Chore: Windower interface. closes #234 (#340)
• 3206bd12 feat: add minikube, kind and podman support (#206)
• d40ecdaa refactor: Close watermark fetcher and publisher correctly (#336)
• 0d8f659e passing window information inside the context (#341)
• 89516277 feat: timestamp in UI to display milliseconds. closes #280 (#337)
• 5c43f5aa Simple reduce pipeline. Fixes #289 (#317)
• 7f5d86c3 feat: add blackhole sink. Closes #329 (#330)
• 10f355c3 fix: move watermark based on the head of the read batch (#332)
• b2b975f3 feat: configurable jetstream storage (#328)
• 3fcf637c feat: support adding sidecars in vertex pods. Closes #323 (#325)
• 6eab1b5b feat: populate watermark settings to vertex spec. Closes #320 (#321)
• 2355978b doc: add few use cases (#318)
• bfc1eb60 Chore: run in sdks-e2e tests, python-udsink log check before go-udsink (#315)
• cda41eca fix: jetstream build watermark progressors bug (#316)
• bfab8f1d feat: update watermark offset bucket implementation (#307)
• 1d86aa5f feat: shuffling support (#306)
• b817920a feat: customize init-container resources. Closes #303 (#304)
• 0548d4d3 feat: watermark - remove non-share OT bucket option (#302)
• cc44875b feat: customization for batch jobs. Closes #259 (#300)
• d16015f3 refactor: abstract pod template (#296)
• 4550f459 feat: customization for daemon deployment. Closes #223 (#290)
• d61377a5 feat: add pvc support for reduce vertex PBQ (#292)
• b0e3f944 fix(doc): hyperlink for security doc (#288)
• 6c61728d feat: support adding init containers to vertices. Closes #284 (#285)
• 88cf272c fix: retry when getting EOF error at E2E test (#281)
• 1436071c feat: Watermark millisecond. Fixes #201 (#278)
• 7a7e7945 feat: add pipeline node counts and age to printcolumn. Closes #267 (#282)
• 5883e973 feat: introduce reduce UDF. Closes #246 (#262)
• a0dc17f8 feat: add pandoc to required tools development doc. Closes #276 (#277)
• 284be2d6 feat: add isbsvc type and age to printcolumn. Closes #268 (#275)
• 7bb689bc fix: watermark consumer fix (#273)
• 8ff9e28e refactor: generalize watermark fetching as an interface of ISB service. Fixes #252 (#263)
• 8e038d1e fix: set default property values for minimal CRD installation (#264)
• 57df392f fix: validate only one isbsvc implementation type is defined. Fixes #269 (#271)
• 21378a36 fix: main branch make build failure: math.MaxInt64 for int type (#265)
• 3d9997d6 fix: nil pointer deref when running example with minimal CRD. Fixes #260 (#261)
• 4b0cbc37 fix: retry only the failed offsets (#255)
• 27e6a875 fix: re-enable build constraint on processor manager test. Fixes #256 (#257)
• 98b3ec4d fix: container resource for jetstream isbsvc. Fixes #253 (#254)
• e615e16e fix: update vertex watermark fetching logic. Fixes: #134 (#245)
• 30c734bd fix: watermark watcher leak (#242)
• 3d29f79d fix(docs): fix a typo (#241)
• 0370fd6c feat: Support running UX server with namespace scope. Fixes #248 (#249)
• 29f15d57 fix(manifests): Include ServiceAccount in namespace scoped install (#240)
• 6870d2a4 fix: Watermark close fix and removed the nil check (#238)
• 998e3988 fix: skip publishing watermarks to unexpected vertices. Fixes #235 (#236)
• fff05f32 fix: update default watermark to -1. Fixes #133 (#218)
• a23e3592 feat: support disabling TLS and changing port for UX server (#228)
• 5a4387c7 feat: reducer for stream aggregation without fault tolerance (#208)
• fc2ba4e9 feat: in-memory watermark store for better testing (#216)
• c89aef31 Add USERS.md (#221)
• 2377c4c6 fix(watermark): generator should not publish wm for every message (#217)

Contributors

• David Seapy
• Derek Wang
• Ed Lee
• Juanlu Yu
• Keran Yang
• Shay Dratler
• SianLoong
• Vedant Gupta
• Vigith Maurice
• Yashash H L
• ashwinidulams
• dependabot[bot]
• xdevxy

v0.6.5 (2022-12-07)

• 845c9594 Update manifests to v0.6.5
• 676ea1c6 fix: adding lock while discovering partitions, Closes #412 (#413)
• c439a6a1 fix(test): e2e-api-pod can not start on M1 mac (#410)
• 115a69d6 fix: getWatermark to return-1 if any processor returns -1 (#402)
• e6e24eef fix: e2e testing for PBQ WAL with reduce pipeline (#393)
• 7ef3d47c feat: add Grafana instruction and a dashboard template. Closes #287 (#381)
• 13ce4d27 fix: unit tests for replay. Closes #373 (#377)

Contributors

• Derek Wang
• Keran Yang
• Yashash H L
• xdevxy

v0.6.4 (2022-11-28)

• ad9719a6 Update manifests to v0.6.4
• c5e82176 chore(docs): update docs (#380)
• 1b244c1a fix: best effort processing during SIGTERM. Closes #371 (#372)
• 9bb8ebd5 feat(wal): First pass to implement WAL and hook to PBQ store. (#344)

Contributors

• Derek Wang
• Vigith Maurice
• xdevxy

v0.6.3 (2022-11-18)

• 3cf391b1 Update manifests to v0.6.3
• bec020b7 feat: watermark otwatcher enhancement (#364)
• 2b5478fc refactor(docs): group docs in categories (#362)
• 6d9e129b chore(deps): bump loader-utils from 2.0.3 to 2.0.4 in /ui (#356)
• 77364a4d fix(controller): vertex nil check for edge listing. Fixes #352 (#353)
• 7db4fe56 fix: data race in pbq manager. Closes #348 (#349)
• 6f5e83a7 Chore: Windower interface. closes #234 (#340)
• 24ba5157 feat: add minikube, kind and podman support (#206)
• 12c6ca52 refactor: Close watermark fetcher and publisher correctly (#336)
• 2a8b97e1 passing window information inside the context (#341)
• d52a5a75 feat: timestamp in UI to display milliseconds. closes #280 (#337)
• de9059cf Simple reduce pipeline. Fixes #289 (#317)
• 3d936a50 feat: add blackhole sink. Closes #329 (#330)
• 45905475 fix: move watermark based on the head of the read batch (#332)
• 049e5c66 feat: configurable jetstream storage (#328)
• ee5cd642 feat: support adding sidecars in vertex pods. Closes #323 (#325)

Contributors

• David Seapy
• Derek Wang
• Juanlu Yu
• Shay Dratler
• Vedant Gupta
• Vigith Maurice
• Yashash H L
• ashwinidulams
• dependabot[bot]

v0.6.2 (2022-11-07)

• 99be6c08 Update manifests to v0.6.2
• dc733da1 feat: populate watermark settings to vertex spec. Closes #320 (#321)
• 2b247cad doc: add few use cases (#318)
• 07ffa168 Chore: run in sdks-e2e tests, python-udsink log check before go-udsink (#315)
• 7b3285b9 fix: jetstream build watermark progressors bug (#316)
• 1198a609 feat: update watermark offset bucket implementation (#307)
• 34a6d709 feat: shuffling support (#306)
• 448127ff feat: customize init-container resources. Closes #303 (#304)
• 61cf2272 feat: watermark - remove non-share OT bucket option (#302)
• 51c9ff42 feat: customization for batch jobs. Closes #259 (#300)
• afbe2557 refactor: abstract pod template (#296)
• 24089460 feat: customization for daemon deployment. Closes #223 (#290)
• bedf567c feat: add pvc support for reduce vertex PBQ (#292)
• 2341614b fix(doc): hyperlink for security doc (#288)
• 6c05190d feat: support adding init containers to vertices. Closes #284 (#285)
• dc96b872 fix: retry when getting EOF error at E2E test (#281)
• f5db937c feat: Watermark millisecond. Fixes #201 (#278)
• c1535365 feat: add pipeline node counts and age to printcolumn. Closes #267 (#282)

Contributors

• David Seapy
• Derek Wang
• Juanlu Yu
• Keran Yang
• Vigith Maurice

v0.6.1 (2022-10-26)

• 32b284f6 Update manifests to v0.6.1
• 9684e161 fix(manifests): Include ServiceAccount in namespace scoped install (#240)
• fe83918a fix(docs): fix a typo (#241)
• f2094b4b feat: introduce reduce UDF. Closes #246 (#262)
• e19a1e7d feat: add pandoc to required tools development doc. Closes #276 (#277)
• 9a937118 feat: add isbsvc type and age to printcolumn. Closes #268 (#275)
• f25e303e fix: watermark consumer fix (#273)
• d2a3d908 refactor: generalize watermark fetching as an interface of ISB service. Fixes #252 (#263)
• 5ffcadcc fix: set default property values for minimal CRD installation (#264)
• 17a99564 fix: validate only one isbsvc implementation type is defined. Fixes #269 (#271)
• 2272a1fc fix: main branch make build failure: math.MaxInt64 for int type (#265)
• 02c31d27 fix: nil pointer deref when running example with minimal CRD. Fixes #260 (#261)
• 391b53e1 fix: retry only the failed offsets (#255)
• 7b42dc80 fix: re-enable build constraint on processor manager test. Fixes #256 (#257)
• 34360490 fix: container resource for jetstream isbsvc. Fixes #253 (#254)
• 33ce7422 fix: update vertex watermark fetching logic. Fixes: #134 (#245)
• fd219a5c fix: watermark watcher leak (#242)
• 979a3a3f feat: Support running UX server with namespace scope. Fixes #248 (#249)
• 5e9d1c1c fix: Watermark close fix and removed the nil check (#238)
• 340bd820 fix: skip publishing watermarks to unexpected vertices. Fixes #235 (#236)
• 904b2cde fix: update default watermark to -1. Fixes #133 (#218)
• 321e285f feat: support disabling TLS and changing port for UX server (#228)
• d0d74e19 feat: reducer for stream aggregation without fault tolerance (#208)
• 06a9b58a feat: in-memory watermark store for better testing (#216)
• f25cc58e Add USERS.md (#221)
• a37cece9 fix(watermark): generator should not publish wm for every message (#217)

Contributors

• David Seapy
• Derek Wang
• Ed Lee
• Juanlu Yu
• Keran Yang
• SianLoong
• Yashash H L
• ashwinidulams

v0.6.0 (2022-10-12)

• 48aad5fc Update manifests to v0.6.0
• 09ce54f1 fix(autoscaling): Ack pending should be included in total pending calculation (#212)
• 9922787c fix(autoscaling): Skip autoscaling if vertex is not in running phase (#207)
• bc2380a7 feat: ISBSVC add support for redis cluster mode (#195)
• 72a96a58 refactor: move controllers package to pkg/reconciler (#192)
• b1b78faa fix: update udf fetchWatermark and publishWatermark initial values (#193)
• d4912600 fix(docs): readme for UI development (#181)
• 6b121c6e feat: grpc udsink (#174)
• 567da7b0 fix: numaflow-go udf example & docs (#177)
• 4652f808 fix: use scale.max if it is set (#179)
• 900314bc fix broken path (#176)
• 3b02f2a6 feat: Shuffle implementation (#169)
• 021bb9df feat: windowing operations (#157)
• 7d411294 feat: watermark for sources (#159)
• 5f5b2dfd fix: daemon service client memory leak (#161)
• bfe96695 pbq implementation (#155)
• 8dfedd83 feat: check if udf is running in liveness probe (#156)
• 81e76d82 feat: Add udf grpc support Fixes #145 (#146)
• 511faffc refactor: some refactor on watermark (#149)
• 7fe40c42 fix: Fixed JS bug (#144)
• 24a16a04 bug: watermark needs nil check
• f4ed831b fix: pipeline UI broken when vertex scaling down to 0 (#132)
• 0ae0377f feat: JetStream auto-reconnection (#127)
• 2fc04eb3 feat: Add watermark for sink vertex (#124)
• d958ee6d feat: autoscaling with back pressure factor (#123)
• b1f77682 feat: add watermark to UI (#122)
• 7feeaa87 feat: add processing rate to UI (#121)
• 43fae931 feat: Expose watermark over HTTP (#120)
• ec02304a fix: daemon service rest api not working (#119)
• f3da56d3 chore(deps): bump terser from 5.14.1 to 5.14.2 in /ui (#117)
• e2e63c84 feat: Numaflow autoscaling (#115)
• e5da3f54 feat: watermark for headoffset (#116)
• a45b2eed feat: support namespace scope installation (#112)
• ce39199e feat: Expose ReadTimeoutSeconds on Vertex (#110)
• 18ad1c5f fix: imagepullpocily for local testing (#113)
• 469849b5 feat: removed udfWorkers from limits and added some docs (#103)
• 3fada667 feat: Add icon and other minor changes (#94)
• a81838d7 feat: end to end tickgen watermark validation (#98)
• d7d93175 fix: Broken hyperlink (#96)
• a2e07926 add no-op KV Store (#91)
• 45c8cb69 feat: no operation watermark progressor (#90)
• 448c229a feat: kafka source pending messages (#80)
• 1aa39300 feat: Interface for Watermark (#82)
• be78c523 feat: expose pending messages and processing rate (#79)
• df30f2a8 feat: Added the right way to decipher from and to vertex (#78)
• 639c459a feat: define buffer limits on edges (#70)
• 41fdd38b feat: Merge UX server code (#67)
• ced99079 feat: auto-scaling (part 1) (#59)
• fd5b3741 Added name to service spec (#58)
• dc2badfd feat: introduce source buffer and sink buffer (#53)
• 4ed83a2a feat: async publishing for kafka sink (#51)
• 9f9f5ba7 fix spelling errors (#48)
• f423002e feat: controller to create buckets (#47)
• 8328739c turn on watermark only if ENV value is true (#46)
• 46f72e23 minimal end to end line-graph watermark integration (#43)
• 1f8203f4 Fixed spelling error (#44)
• f1e99eae Exponential buckets (#42)
• dfcfdeba fix: different behavior for time.After in go 1.18 (#39)

Contributors

• Chrome
• Derek Wang
• Juanlu Yu
• Krithika3
• Qianbo Huai
• Saravanan Balasubramanian
• Sidhant Kohli
• Vigith Maurice
• Yashash H L
• dependabot[bot]

v0.5.6 (2022-09-19)

• ac15d229 Update manifests to v0.5.6
• f2363757 feat: grpc udsink (#174)
• 2650c2de fix: numaflow-go udf example & docs (#177)
• c44f733f fix: use scale.max if it is set (#179)
• 39e92d06 fix broken path (#176)
• 46ce0f87 feat: Shuffle implementation (#169)
• 71ca00a1 feat: windowing operations (#157)
• ca00b78f feat: watermark for sources (#159)
• 91e21cee pbq implementation (#155)
• 65424004 feat: check if udf is running in liveness probe (#156)
• 79dce0b3 feat: Add udf grpc support Fixes #145 (#146)

Contributors

• Chrome
• Derek Wang
• Juanlu Yu
• Vigith Maurice
• Yashash H L

v0.5.5 (2022-09-07)

• 9aae638c Update manifests to v0.5.5
• 32414325 fix: daemon service client memory leak (#161)
• be47a26a refactor: some refactor on watermark (#149)
• 857cce75 fix: Fixed JS bug (#144)
• da16abc7 bug: watermark needs nil check
• c9998a1c fix: pipeline UI broken when vertex scaling down to 0 (#132)

Contributors

• Derek Wang
• Krithika3
• Vigith Maurice

v0.5.4 (2022-08-05)

• 57513b40 Update manifests to v0.5.4
• 94cdb82f feat: JetStream auto-reconnection (#127)
• 8d835408 feat: Add watermark for sink vertex (#124)
• 228ba321 feat: autoscaling with back pressure factor (#123)
• 9833efdf feat: add watermark to UI (#122)
• 0dab55d8 feat: add processing rate to UI (#121)
• ffd38a15 feat: Expose watermark over HTTP (#120)
• c09502a2 fix: daemon service rest api not working (#119)
• ebc10f41 chore(deps): bump terser from 5.14.1 to 5.14.2 in /ui (#117)
• 84490ca8 feat: Numaflow autoscaling (#115)
• 32b98486 feat: watermark for headoffset (#116)
• 283dae90 feat: support namespace scope installation (#112)
• 8e612b1f feat: Expose ReadTimeoutSeconds on Vertex (#110)
• d95d41bd fix: imagepullpocily for local testing (#113)

Contributors

• Derek Wang
• Krithika3
• Saravanan Balasubramanian
• Sidhant Kohli
• Vigith Maurice
• dependabot[bot]

v0.5.3 (2022-07-08)

• efee5442 Update manifests to v0.5.3
• 5895facd feat: removed udfWorkers from limits and added some docs (#103)
• 0b75495f feat: Add icon and other minor changes (#94)
• 7eb08f58 feat: end to end tickgen watermark validation (#98)
• 3338e658 fix: Broken hyperlink (#96)
• e3112229 add no-op KV Store (#91)
• 5d2f90ed feat: no operation watermark progressor (#90)
• f58d0f49 feat: kafka source pending messages (#80)
• cbb16ca2 feat: Interface for Watermark (#82)
• 5592bb1b feat: expose pending messages and processing rate (#79)
• 06a3df2d feat: Added the right way to decipher from and to vertex (#78)
• a0908ad4 feat: define buffer limits on edges (#70)
• a1d36395 feat: Merge UX server code (#67)
• 571c48eb feat: auto-scaling (part 1) (#59)
• 1e0384ba Added name to service spec (#58)

Contributors

• Derek Wang
• Krithika3
• Sidhant Kohli
• Vigith Maurice

v0.5.2 (2022-06-13)

• 2f2d10ce Update manifests to v0.5.2
• cedd0d1f feat: introduce source buffer and sink buffer (#53)
• d3301aa9 feat: async publishing for kafka sink (#51)
• 2474eb8e fix spelling errors (#48)
• c4a12f87 feat: controller to create buckets (#47)
• eb97dc3b turn on watermark only if ENV value is true (#46)
• f189ba30 minimal end to end line-graph watermark integration (#43)

Contributors

• Derek Wang
• Qianbo Huai
• Vigith Maurice

v0.5.1 (2022-06-02)

• bb9be807 Update manifests to v0.5.1
• 912747eb Fixed spelling error (#44)
• 3aeb33a8 Exponential buckets (#42)
• 1d656829 fix: different behavior for time.After in go 1.18 (#39)

Contributors

• Derek Wang
• Krithika3