Apache Eagle

The intelligent monitoring and alerting solution instantly analyzes big data platforms for security and performance

Apache® Eagle™ is an open source analytics solution for identifying security and performance issues instantly on big data platforms e.g. Apache Hadoop, Apache Spark, NoSQL etc. It analyzes data activities, yarn applications, jmx metrics, and daemon logs etc., provides state-of-the-art alert engine to identify security breach, performance issues and shows insights.

For more details, please visit https://eagle.apache.org

Documentation

You can find the latest Eagle documentation on https://eagle.apache.org. This README file only contains basic setup instructions.

Downloads

• Development Version: eagle-0.5-SNAPSHOT (Under development)
• Latest Release
  • eagle-0.4.0-incubating
• Archived Releases
  • eagle-0.3.0-incubating
  • More releases

Getting Started

Prerequisites

• JDK 8: Java Environment Version 1.8
• Apache Maven: Project management and comprehension tool Version 3.x
• NPM: Node package management tool Version 3.x

Building Eagle

Since version 0.5, Eagle is only built on JDK 8.

Eagle is built using Apache Maven. NPM should be installed (On MAC OS try "brew install node"). To build Eagle, run:

mvn clean package -DskipTests 

After successfully building, you will find eagle binary tarball at:

eagle-assembly/target/eagle-${VERSION}-bin.tar.gz

Testing Eagle

mvn clean test

Developing Eagle

• (Optional) Install/Start HDP Sandbox which provide an all-in-one virtual machine with most dependency services like Zookeeper, Kafka, HBase, etc and monitored hadoop components.

• Import Eagle as maven project with popular IDE like IntelliJ IDEA

• Start Eagle Server in debug mode by running (default http port: 9090, default smtp port: 5025)

    org.apache.eagle.server.ServerDebug
  

  Which will start some helpful services for convenient development:

  • Local Eagle Service on http://localhost:9090
  • Local SMTP Service on localhost:5025 with REST API at http://localhost:9090/rest/mail

• Start Eagle Apps with Eagle Web UI in LOCAL MODE.

Getting Help

• Mail: The fastest way to get response from eagle community is to send email to the mail list dev@eagle.apache.org, and remember to subscribe our mail list via dev-subscribe@eagle.apache.org
• Slack: Join Eagle community on Slack via https://apacheeagle.slack.com
• JIRA: Report requirements, problems or bugs through apache jira system via https://issues.apache.org/jira/browse/EAGLE

FAQ

https://cwiki.apache.org/confluence/display/EAG/FAQ

Contributing

Please review the Contribution to Eagle Guide for information on how to get started contributing to the project.

License

Licensed under the Apache License, Version 2.0. More details, please refer to LICENSE file.

组件版本要求

Java Version Java8

Apache Storm 1.1.3

Apache HBase 1.4.9 (1.x later)

Apache Kafak 1.0.0(1.x later)

初始化系统

1 导入metadata-ddl.sql和init.sql,初始化mysql元数据表 ${eagle_home}/eagle-assembly/src/main/doc/metadata-ddl.sql ${eagle_home}/eagle-core/eagle-alert-parent/eagle-alert/alert-metadata-parent/alert-metadata/src/test/source/init.sql

注意: 追加监控类型需在 ${eagle_home}/eagle-core/eagle-alert-parent/eagle-alert/alert-metadata-parent/alert-metadata/src/test/source/init.sql 文件 中的publishment_type表添加插入值 如: INSERT INTO publishment_type(id, content) VALUES ('Open-Falcon', '{"name":"Open-Falcon","type":"org.apache.eagle.alert.engine.publisher.external.openfalcon.AlertOpenFalconPublisher","description":null,"fields":[]}'), ('RMS', '{"name":"RMS","type":"org.apache.eagle.alert.engine.publisher.external.rms.AlertRMSPublisher","description":null,"fields":[]}')

OS系统参数监控功能需安装以下工具

yum -y install sysstat yum -y install smartmontools

cd $eagle_home/lib/scripts/hadoop_jmx_collector 定时执行 python hadoop_jmx_kafka.py hadoop_jmx_config.json python system_metric_collector.py system_metric_config.json

把hadoop,os 监控参数push 到kafka 对应的topic

关于WEB UI 时区问题

timeSrv.js#Server.getPromise().then(function () 函数中,只是识别UTC和GMT,因此,如果要自动显示中国、亚洲时间, 需要在conf/eagle.conf 文件里面设置service.timezone = "GMT+8"