Skip to content

Streamis 0.2.0 Release

Compare
Choose a tag to compare
@Davidhua1996 Davidhua1996 released this 06 Jul 02:45
0.2.0
0f3a750

Streamis-0.2.0 includes all of Project Streamis-0.2.0.
This release mainly add the architecture optimization and tuning, provide some important features, including Streaming job state(checkpoint and savepoint) management which enable user to do a snapshot for Flink application and restore state from it, New Streaming job configuration module which supports to add or delete configuration item flexibly, Streamis AppConn, Streaming job management in batch(such as start and stop in batch) and so on. Streamis 0.2.0 support multiple Linkis versions (>= 1.1.1)，but some features require version greater than 1.1.2.
The following key features have been added:

  • Provided AppConn for Streamis to synchronize project basic information and privilege relations from DataSphereStudio.
  • Provided strategy to obtain or create checkpoint/savepoint file information of Flink application through Linkis server. (require Linkis 1.1.2)
  • Provided strategy to restart the streaming job automatically and resolve the idempotent problem.
  • Provided the tenancy scheduler module to manage the lifecycle of streaming jobs in batch.
  • Provided the method to set the custom Flink configuration variables. (require Linkis 1.1.2)
  • Provided the interface to view the log of yarn container through Linkis server. (require Linkis 1.1.2)

Abbreviations:

  • SJL: Stream Job Launcher Module
  • SJDA: Stream Job Deploy API Module
  • SJHV: Stream Job History And Version Module
  • SBO: Stream Bulk Operation Module
  • SA: Stream AppConn Module

Features/Enhancement of version has been rolled up one pull request . [Streamis-28]

New Features

  • SJL: Add the JobStateManager module to provide JobStateFetcher to obtain job state info (checkpoint/savepoint).[Streamis-23]
  • SJL: Automatically restart failed tasks(related job) and automatically restore its state.[Streamis-22]
  • SBO: Job batch operation collection. [Streamis-19]
  • SJDA: Enable to fetch yarn application logs (should dependency on linkis version higher than 1.1.2). [Streamis-27]
  • SA: Streamis AppConn to sync the project and privilege information (should dependency on dss version higher than 1.1.0). [Streamis-24]

Enhancement

  • SJHV: Refactor the job configuration module and provide a new interface for job configuration acquisition and addition. [Streamis-21]
  • SJDA: Provide two methods to stop job: "stop directly" and "snapshot and stop". [Streamis-20]

Bugs Fix

  • SJDA: Fix the problem that exception in streaming job, but cannot view the error log. [Streamis-17]

Credits

   The release of Streamis 0.2.0 is inseparable from the contributors of the Streamis community. Thanks to all the community contributors!

Compiled Version

  1. Tencent Cloud:
    wedatasphere-streamis-0.2.0-dist.tar.gz.
    streamis-0.2.0-dist.zip
Assets 2
Loading