Skip to content

HiBench 5.0
Compare
Choose a tag to compare
@lvsoft lvsoft released this 01 Nov 18:25
· 850 commits to master since this release
HiBench-5.0
2f7ad75

We are happy to announce HiBench-5.0, a major release with streaming feature support!

We are now introducing streaming features

Streaming bench is all the news about HiBench 5.0. It provides benchmark against multiple frameworks, new streaming workloads abstractions.

Multiple Streaming Frameworks

Spark Streaming

SparkStreaming is a batch based extension of the core Spark API for streaming data processing feature. HiBench 5.0 supports SparkStreaming from spark1.3 to 1.5, Kafka and direct mode.

Storm & Trident

Apache Storm is a event based distributed realtime computation system contributed from Twitter. Trident is a high-level abstraction for doing realtime computing on top of Storm. Hibench 5.0 supports them both.

Samza

Apache Samza is a distributed stream processing framework contributed from LinkedIn, which is also supported in Hibench 5.0.

7 Streaming Workloads Abstraction

We introduce identity, sample, projection, grep for single step workloads, and wordcount, distinctcount statistics for multiple steps workloads. And text data generated from Hive's uservisits test cases, numeric data generated from Kmeans's vectors.

Flexible Data Source

You can feed data to Kafka from data stored in HDFS, which is a great helps to send distributed data concurrently.
You can push the data once for all or feed data continuously and periodically.
You can adjust data offset in Kafka to avoid reuse sent data.

Contributors

The following developers contributed to this release (ordered by Github ID):

Daoyuan Wang(@adrian-wang)
Earnest(@Earne)
Minho Kim (@eoriented)
Gayathri Mutrali(@GayathriMurali)
Jie Huang(@GraceH)
Joseph Lorenzini(@jaloren)
Jay Vyas(@jayunit100)
Jintao Guan(@jintaoguan)
Kai Wei(@kai-wei)
Zhihui Li(@li-zhihui)
Qi Lv(@lvsoft)
Nishkam Ravi (@nishkamravi2)
(@pipamc)
Kevin CHEN(@princhenee)
Neo Qian(@qiansl127)
Mingfei Shi(@shimingfei)
ShelleyRuirui(@ShelleyRuirui)
Imran Rashid(@squito)
(@Silent-Hill)
Viplav(@viplav)
(@XiaominZhang)
Dong Li(@zkld123)
Markus Z(@zyxar)

Thanks to everyone who contributed! We are looking forward to more contributions from every one for next release.

Assets 2
Loading