Merge pull request #134 from cryptape/release/v0.4.0

Release/v0.4.0

.gitignore

@@ -0,0 +1,7 @@
+tmp/
+cita-test-chain/
+server/grafana_data/
+server/prometheus_data/
+.vscode/
+.build-version
+test/dummy/*

CHANGELOG.md

@@ -4,7 +4,55 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/)
 and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.html).
 
-## [v0.3.0](https://github.com/cryptape/cita-monitor/releases/tag/v0.3.0) ([compare](https://github.com/cryptape/cita-monitor/compare/v0.2.0...v0.3.0))
+## [v0.4.0](https://github.com/cryptape/cita-monitor/releases/tag/v0.4.0) ([compare](https://github.com/cryptape/cita-monitor/compare/v0.3.0...v0.4.0))
+
+### Added
+- Agent script gets block consensus node votes ([66dbeb7](https://github.com/cryptape/cita-monitor/commit/66dbeb7683a46c8e204a87c884af19e74f5f2e75) by JiangXL).
+- Control the execution frequency of the du command, once every 1 hour ([ffb1929](https://github.com/cryptape/cita-monitor/commit/ffb192906ed3065396f0d371cb099fc9b744d509) by JiangXL).
+- Add cita_with_agent to Integrate CITA with agent services ([8eeddbd](https://github.com/cryptape/cita-monitor/commit/8eeddbd356ffd14e8dab04efc5004ab0a200ae8c) by mfuuzy).
+- Add block consensus node vote number displayed in chain info ([757a20d](https://github.com/cryptape/cita-monitor/commit/757a20d2e679622fc744153df83d3c0199775ed9) by JiangXL).
+- Add a node directory log file using the size of the display ([e42ad5f](https://github.com/cryptape/cita-monitor/commit/e42ad5f074047133c5a155885e2f06129fcc3018) by JiangXL).
+- Add agent proxy to get data using one port ([c91946b](https://github.com/cryptape/cita-monitor/commit/c91946ba61c5b787068fffe54c236d6f37cb4ebd) by JiangXL).
+- Add test/makefile to bring in build/test recipes ([576d8c9](https://github.com/cryptape/cita-monitor/commit/576d8c9de51c72cefb528197ef35a4916a441a15) by RainChen).
+- Add test dir and init system testing env ([7c3b237](https://github.com/cryptape/cita-monitor/commit/7c3b23746b8f388e0eca05833381c7db66b1f0f2) by RainChen).
+- Add `make generate-build-version`: generate build metadata using format "x.y.z+yyyymmddhhmmss.commitshortid", e.g.: "0.3.0+20190606200449.40564fe"; add `make show-full-version`: show product version and build version, e.g.: "v0.3.0+20190606201137.40564fe" ([bcc76f9](https://github.com/cryptape/cita-monitor/commit/bcc76f90e6bb64d84ba9d3bde4063e0067b022dc) by RainChen).
+- Add genesis block hash display for chain meta data ([2d38c53](https://github.com/cryptape/cita-monitor/commit/2d38c536b7e726ebec48fbb2539e1886948777e8) by JiangXL).
+- Dashboard add version display ([99447fd](https://github.com/cryptape/cita-monitor/commit/99447fd1e54f8220a68cc38d0d61cdbb29058a2a) by blankwu).
+
+
+### Changed
+- Change the service name ([a031404](https://github.com/cryptape/cita-monitor/commit/a031404d7b22ca1f83363eacea19ba0614e57463) by mfuuzy).
+- Update readme agent unified port ([9ccb065](https://github.com/cryptape/cita-monitor/commit/9ccb0658dd691321d91f741fbdd12f5a4076e179) by JiangXL).
+- Update test/makefile to add `make rpc-block-number-all` to call cita rpc method blocknumber for all nodes ([375eab4](https://github.com/cryptape/cita-monitor/commit/375eab44a1e5795127f9e857f58e545af22a2aff) by RainChen).
+- Update prometheus.yml for deployment only when you need to sed once ([c13611e](https://github.com/cryptape/cita-monitor/commit/c13611e073060e074d4e800728430f3bc62cef00) by JiangXL).
+- Update document instructions to modify the agent to expose only one port ([e0abb7b](https://github.com/cryptape/cita-monitor/commit/e0abb7be03d3f0c850ff75994e641bb645f15b07) by JiangXL).
+- Update grafana build process ([50311b5](https://github.com/cryptape/cita-monitor/commit/50311b5c0ddcfca3bb0de326cae0338d00b342bc) by blankwu).
+- Update grafana build process, del compose mount dashboard files ([d228c24](https://github.com/cryptape/cita-monitor/commit/d228c2413a69669fe8adddcb544ac577eb2169c8) by blankwu).
+- Modify the agent .env.example annotation ([359c6dc](https://github.com/cryptape/cita-monitor/commit/359c6dcad118324cbd3ec84fe433199fead4ed8c) by JiangXL).
+
+### Documented
+- Docs: update readme about introduction ([e5d9f00](https://github.com/cryptape/cita-monitor/commit/e5d9f0072850907cfe8e5d1a40c00064beced1ad) by RainChen).
+- Docs: add coding style guide for prometheus ([44ea685](https://github.com/cryptape/cita-monitor/commit/44ea68508037e3eb38bdc97514efb109c37c348c) by RainChen).
+- Update agent readme ([57fa9a5](https://github.com/cryptape/cita-monitor/commit/57fa9a5da3d8b16b6731fa63fa2b31d66215fac7) by JiangXL).
+- Update grafana build command readme ([e98bd4b](https://github.com/cryptape/cita-monitor/commit/e98bd4b16934c121df5421df980b4a474f9e7048) by blankwu).
+
+### Fixed
+- Fix show full pie chart, delete dup ip and port information ([9bc30b6](https://github.com/cryptape/cita-monitor/commit/9bc30b659f6c7cf4708cb39e188672797c87c3ab) by blankwu).
+- Fix the new version image and get the cita version number. ([0d8a763](https://github.com/cryptape/cita-monitor/commit/0d8a7633c2d2f3d9a94bf1d243bd0b45eccaa7a5) by JiangXL).
+- Fix dockerfile comment ([f0c8717](https://github.com/cryptape/cita-monitor/commit/f0c87172b2fdd545650bbd5248285e6fde548db2) by blankwu).
+- Fix container startup failed after modifying docker-compose.xml service name ([154729d](https://github.com/cryptape/cita-monitor/commit/154729ddb2b5bc04215ce7c596806f75081f094a) by JiangXL).
+
+### Removed
+- Remove redundant readme ([3fc0234](https://github.com/cryptape/cita-monitor/commit/3fc02346d2dd2a17cafe800e37c7fa9848c7cefa) by JiangXL).
+- Remove timezone from docker-compose.xml ([da017ab](https://github.com/cryptape/cita-monitor/commit/da017ab5cd73432218e46a6bcf8e302b50c4569a) by JiangXL).
+
+### Misc
+- Specify flask and psuti version number ([abbba92](https://github.com/cryptape/cita-monitor/commit/abbba929b17803d984717982153d1355017977d3) by JiangXL).
+- Reduce the size of the cita-exporter-image from 970mb to 478mb ([1b27e5d](https://github.com/cryptape/cita-monitor/commit/1b27e5d8498662d7010083340f6adcbbf72e6a3e) by JiangXL).
+- Allow to start test node using cita v0.24.0 ([7f07819](https://github.com/cryptape/cita-monitor/commit/7f07819f4a17458c4afbedc7840234f5cd2f4b26) by RainChen).
+
+
+## [v0.3.0](https://github.com/cryptape/cita-monitor/releases/tag/v0.3.0) ([compare](https://github.com/cryptape/cita-monitor/compare/v0.2.0...v0.3.0)) - 2019-06-05
 
 ### Added
 - Add devtools/release/keepachangelog-template ([92dffa1](https://github.com/cryptape/cita-monitor/commit/92dffa1baf0b2a5bfb6f2086f8f2bc310e0cfbf6) by RainChen).
@@ -28,6 +76,8 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.
 - Fix: install requirements for agent/cita_exporter in `make deps` ([2427dd9](https://github.com/cryptape/cita-monitor/commit/2427dd96735cd1689dfcd19ce18c773da6d74b03) by RainChen).
 
 ### Misc
+- Bump version to v0.3.0 ([6390089](https://github.com/cryptape/cita-monitor/commit/63900895a67cfeecbf2d2e2b45937c5e9dff3d21) by RainChen).
+- Update the readme description of the agent and server ([9c024fb](https://github.com/cryptape/cita-monitor/commit/9c024fbf1423263ca1a3118b4adb0b64873253b4) by JiangXL).
 - Update script format ([05652bc](https://github.com/cryptape/cita-monitor/commit/05652bc35a82390c8e337b6b986c97ab4250927a) by blankwu).
 - Update the cita node info dashboard layout ([a1b9fbb](https://github.com/cryptape/cita-monitor/commit/a1b9fbbdc766bead93b3097530d15df4660355c4) by JiangXL).
 - Update dashboard, fix pie graph ([6c4cfd7](https://github.com/cryptape/cita-monitor/commit/6c4cfd7123e9a984d3ff035987fe51e78b771445) by blankwu).

Makefile

@@ -8,7 +8,7 @@ SHELL = /bin/sh
 # receipts for Testing
 .PHONY: test test-unit test-intergration
 # receipts for Release
-.PHONY: changelog changelog-check
+.PHONY: changelog changelog-check generate-build-version show-full-version
 
 .SILENT: help
 
@@ -39,6 +39,12 @@ clean: ## Clean up.
 
 build: clean deps ## Compile binary targets.
 	$(info Building the project)
+	@echo "[build]build all images"
+	@make generate-build-version
+	@FULL_VERSION=$(shell make show-full-version) ;\
+		cd ./server ;\
+		[ ! -f .env ] && cp .env.example .env ;\
+		docker-compose build --build-arg VERSION="$${FULL_VERSION}"
 
 ##@ Testing
 
@@ -135,6 +141,18 @@ bump-version-file: ## Update version number in VERSION file.
 		echo "New version: $${new_version}" ;\
 	fi
 
+generate-build-version: ## Generate build version using format "yyyymmddHHMMSS.commitshortid", e.g.: "20190606201137.40564fe".
+	@# build version are only ASCII alphanumerics and hyphen [0-9A-Za-z-] alllowed
+	@commitid=$(shell git rev-parse --short HEAD); \
+	builddate=$(shell date +"%Y%m%d%H%M%S");  \
+	printf "%s.%s" $$builddate $$commitid > .build-version
+	@cat .build-version
+
+show-full-version: ## Show product version and build version, e.g.: "v0.3.0+20190606201137.40564fe".
+	@product_version=$(shell cat VERSION); \
+	build_version=$(shell cat .build-version);  \
+	printf "v%s+%s" $$product_version $$build_version
+
 send-pull-request: ## Send a Pull Request with current git branch
 	@repo_url=`git ls-remote --get-url` ;\
 		user=`echo $${repo_url}|cut -d : -f 2|cut -d / -f 1` ;\

README-CN.md

@@ -2,9 +2,13 @@
 
 [EN](README.md) | [CN](README-CN.md)
 
-这是一套基于 Prometheus 框架的 [CITA](https://github.com/cryptape/cita) 区块链服务运行状态监控系统 。
+CITA Monitor 是为监控 [CITA](https://github.com/cryptape/cita) 区块链服务运行状态的开源监控系统。
 
-监控指标包括区块链数据、服务进程状态、运行环境的 CPU /存储器/磁盘使用率等主机信息等。
+监控的指标包括节点的出块高度、出块间隔、交易量历史等区块链数据信息、CITA的各微服务进程状态、节点连接数等软件信息、以及服务运行环境的 CPU /存储器/磁盘使用率等主机信息等。制定了数据可视化的面板，节点管理员可轻松了解节点的运行健康状态。还内置了一些关键的告警规则，例如服务进程状态告警，如各微服务、依赖服务进程存活；区块链数据状态告警，如出块高度、出块间隔时间、交易数据的TPS；运行环境状态告警，如磁盘空间不足，经过简单配置收发邮箱即可第一时间收到相关告警邮件。
+
+软件架构：基于 Prometheus 框架开发；使用 Grafana 做可视化面板；使用 Alertmanager 做告警消息分发；架构上分为用于数据存储、信息展示、告警分发的 Server 端；区块链数据、软件进程、运行环境监控指标收集的 Agent 端；各端均使用微服务架构分离功能职责，并使用 Docker Compose 部署方式解决服务的依赖管理，轻松做到一行命令更新、启动服务。
+
+开源协议：本项目使用 Apache 2.0 开源协议，可自由二次开发、闭源商用。
 
 [![Travis Build Status](https://img.shields.io/travis/com/cryptape/cita-monitor/master.svg)](https://travis-ci.com/cryptape/cita-monitor)
 [![License: Apache-2.0](https://img.shields.io/github/license/cryptape/cita-monitor.svg)](https://github.com/cryptape/cita-monitor/blob/master/LICENSE)
@@ -89,11 +93,8 @@ Rabbitmq Dashboard Demo
     * Prometheus Alertmanager: 1917
     * Prometheus Console: 1918
     * Grafana: 1919
-* CITA-Monitor agent
-    * host_exporter：1920
-    * process_exporter：1921
-    * rabbitmq_exporter：1922
-    * cita_exporter：1923 
+* CITA-Monitor agent 
+    * agent_proxy_exporter：1920
 
 ## 参与贡献
 

README.md

@@ -90,10 +90,7 @@ For installing agent, read [agent/README.md](agent/README.md)
     * Prometheus Console: 1918
     * Grafana: 1919
 * CITA-Monitor agent
-    * host_exporter：1920
-    * process_exporter：1921
-    * rabbitmq_exporter：1922
-    * cita_exporter：1923 
+    * agent_proxy_exporter：1920
 
 ## Contributing
 
@@ -140,6 +137,10 @@ git clone git@github.com:cryptape/cita-monitor.git
 * conventions: https://www.gnu.org/prep/standards/html_node/Makefile-Conventions.html
 * best-practices: https://suva.sh/posts/well-documented-makefiles/
 
+#### Coding style for Prometheus
+
+* coding style guide: https://prometheus.io/docs/practices/naming/
+
 ### Running the tests
 
 PENDING: Explain how to run the automated tests for this system

VERSION

@@ -1 +1 @@
-0.3.0
+0.4.0

agent/.env.example

@@ -1,26 +1,15 @@
-# container hostname
+# citamon_agent_cita_exporter HostName
 HOSTNAME=nodehostname
 
-# prometheus_node_exporter HostPort and ContainerPort
-HOST_HOSTPORT=1920
-HOST_CONTAINERPORT=9100
-
-# process_node_exporter HostPort and ContainerPort
-PROCESS_HOSTPORT=1921
-PROCESS_CONTAINERPORT=9256
-
-# rabbitmq_node_exporter HostPort and ContainerPort
-RABBITMQ_HOSTPORT=1922
-RABBITMQ_CONTAINERPORT=1922
-
-# prometheus_citaMonitorAgent_exporter HostPort and ContainerPort
-AGENT_HOSTPORT=1923
-AGENT_CONTAINERPORT=1920
-
 # CITA node ip and port
 NODE_IP=1.1.1.1
 NODE_PORT=1337
 
 # CITA node directory analysis
 NODE_DIR=/data/cita_secp256k1_sha3/test-chain/0
 SOFT_PATH=/data/cita_secp256k1_sha3/
+
+# CITA NODE INFO
+CITA_NODENAME=node_3
+CITA_CHAIN_ID=opc-devnet/3
+CITA_NETWORKPORT=4003

agent/README.md

@@ -17,7 +17,7 @@ Monitor Agent 是使用 `Python` 作为脚本语言，使用 `cita-cli` 工具
 ```
 cd agent
 cp .env.example .env
-修改 .env 配置文件中 HOSTNAME、NODE_IP、NODE_PORT、NODE_DIR、SOFT_PATH 为实际部署的信息
+修改 .env 配置文件中 HOSTNAME、NODE_IP、NODE_PORT、NODE_DIR、SOFT_PATH、CITA_NODENAME、CITA_CHAIN_ID、CITA_NETWORKPORT 为实际部署的信息
 ---
 
 #采集端显示的主机名
@@ -37,27 +37,33 @@ SOFT_PATH=/data/cita_secp256k1_sha3/
 ```
 docker-compose up -d
 ```
+**可选项**
+```
+注意：CITA目录下应当存在bin目录
+启动 CITA 与 Agent 服务容器
+docker-compose -f docker-compose.yml -f cita_with_agent.yml up -d
+```
 3、查看数据采集信息
 ```
-#prometheus_host_exporter
-curl http://localhost:1920/metrics
+#citamon_agent_host_exporter
+curl http://localhost:1920/metrics/host
 
-#prometheus_process_exporter
-curl http://localhost:1921/metrics
+#citamon_agent_process_exporter
+curl http://localhost:1920/metrics/process
 
-#prometheus_rabbitmq_exporter
-curl http://localhost:1922/metrics
+#citamon_agent_rabbitmq_exporter
+curl http://localhost:1920/metrics/rabbitmq
 
-#prometheus_cita_exporter
-curl http://localhost:1923/metrics
+#citamon_agent_cita_exporter
+curl http://localhost:1920/metrics/cita
 ```
 4、关闭agent容器
 ```
 docker-compose down
 ```
 
 ### 单个容器部署
-如果你希望只采集 CITA 服务的运行状态信息，你可以使用 `docker` 命令来运行一个数据采集进程；
+如果你希望只采集 CITA 服务的运行状态信息，你可以使用 `docker` 命令来运行一个数据采集容器；
 
 #### 步骤
 1、编译镜像
@@ -75,20 +81,18 @@ Tips：
 docker run -d --name="citamon_agent_cita_exporter_1337" \
 --pid="host" \
 -p 1923:1920 \
--v /etc/timezone:/etc/timezone \
 -v /etc/localtime:/etc/localtime \
 -v "/data/cita_secp256k1_sha3/":"/data/cita_secp256k1_sha3/" \
 -v "/data/cita_secp256k1_sha3/test-chain/0":"/data/cita_secp256k1_sha3/test-chain/0" \
 -v "`pwd`/cita_monitor_agent.py":"/config/cita_monitor_agent.py" \
 -e NODE_IP_PORT="x.x.x.x:1337" \
--e PORT=1920 \
 -e NODE_DIR="/data/cita_secp256k1_sha3/test-chain/0" \
 citamon/agent-cita-exporter
 ```
 3、查看数据采集信息
 ```
-#prometheus_cita_exporter
-curl http://localhost:1923/metrics
+#citamon_agent_cita_exporter
+curl http://localhost:1920/metrics/cita
 ```
 
 ### 错误信息

agent/cita_exporter/Dockerfile

@@ -1,12 +1,16 @@
-## 一个基础的 python 运行环境
-FROM python:3
-## 设置工作目录
+FROM ubuntu:18.04
 WORKDIR /config
-## 将当前系统文件夹内容复制到容器的 app 目录
 ADD . /config
-## 安装必要的依赖包
+RUN apt-get update \
+  && apt-get install -y  \
+	  python3 \
+	  python3-pip \
+	  libcurl4-openssl-dev \
+    && apt-get clean \
+    && apt-get autoclean \
+    && rm -rf /var/lib/apt/lists/*
+
 RUN pip3 install -r requirements.txt
 RUN mv cita-cli /bin/cita-cli && chmod +x /bin/cita-cli
-ENV PORT=1920
-## 运行命令
+ENV PORT=1923
 CMD "python3" "cita_monitor_agent.py" "$NODE_IP_PORT" "$PORT" "$NODE_DIR"

agent/cita_exporter/requirements.txt

@@ -1,3 +1,3 @@
 prometheus-client==0.5.0
-flask
-psutil
+flask==1.0.3
+psutil==5.6.3