Skip to content

Commit

Permalink
rewrite metrics system to provide more data (#492)
Browse files Browse the repository at this point in the history
  • Loading branch information
@aler9
aler9 committed Aug 12, 2021
1 parent 6702cb4 commit f1a812b
Show file tree
Hide file tree
Showing 15 changed files with 391 additions and 174 deletions.
87 changes: 48 additions & 39 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -56,8 +56,9 @@ Plus:
* [Fallback stream](#fallback-stream)
* [Start on boot with systemd](#start-on-boot-with-systemd)
* [Corrupted frames](#corrupted-frames)
* [Monitoring](#monitoring)
* [HTTP API](#http-api)
* [Metrics](#metrics)
* [Pprof](#pprof)
* [Command-line usage](#command-line-usage)
* [Compile and run from source](#compile-and-run-from-source)
* [Links](#links)
Expand Down Expand Up @@ -497,62 +498,70 @@ In some scenarios, the server can send incomplete or corrupted frames. This can
readBufferSize: 8192
```

### Monitoring
### HTTP API

There are multiple ways to monitor the server usage over time:
The server can be queried and controlled with an HTTP API, that must be enabled by setting the `api` parameter in the configuration:

* Use the [HTTP API](#http-api), described below.
```yml
api: yes
```

* A metrics exporter, compatible with Prometheus, can be enabled with the parameter `metrics: yes`; then the server can be queried for metrics with Prometheus or with a simple HTTP request:
The API listens on `apiAddress`, that by default is `127.0.0.1:9997`; for instance, to obtain a list of active paths, run:

```
wget -qO- localhost:9998/metrics
```
```
curl http//127.0.0.1:9997/list
```

Obtaining:
Full documentation of the API is available on the [dedicated site](https://aler9.github.io/rtsp-simple-server/).

```
rtsp_clients{state="publishing"} 15 1596122687740
rtsp_clients{state="reading"} 8 1596122687740
rtsp_sources{type="rtsp",state="idle"} 3 1596122687740
rtsp_sources{type="rtsp",state="running"} 2 1596122687740
rtsp_sources{type="rtmp",state="idle"} 1 1596122687740
rtsp_sources{type="rtmp",state="running"} 0 1596122687740
```
### Metrics

where:
A metrics exporter, compatible with Prometheus, can be enabled with the parameter `metrics: yes`; then the server can be queried for metrics with Prometheus or with a simple HTTP request:

* `rtsp_clients{state="publishing"}` is the count of clients that are publishing
* `rtsp_clients{state="reading"}` is the count of clients that are reading
* `rtsp_sources{type="rtsp",state="idle"}` is the count of rtsp sources that are not running
* `rtsp_sources{type="rtsp",state="running"}` is the count of rtsp sources that are running
* `rtsp_sources{type="rtmp",state="idle"}` is the count of rtmp sources that are not running
* `rtsp_sources{type="rtmp",state="running"}` is the count of rtmp sources that are running
```
wget -qO- localhost:9998/metrics
```

* A performance monitor, compatible with pprof, can be enabled with the parameter `pprof: yes`; then the server can be queried for metrics with pprof-compatible tools, like:
Obtaining:

```
go tool pprof -text http://localhost:9999/debug/pprof/goroutine
go tool pprof -text http://localhost:9999/debug/pprof/heap
go tool pprof -text http://localhost:9999/debug/pprof/profile?seconds=30
```
```
paths{state="ready"} 2 1628760831152
paths{state="notReady"} 0 1628760831152
rtsp_sessions{state="idle"} 0 1628760831152
rtsp_sessions{state="read"} 0 1628760831152
rtsp_sessions{state="publish"} 1 1628760831152
rtsps_sessions{state="idle"} 0 1628760831152
rtsps_sessions{state="read"} 0 1628760831152
rtsps_sessions{state="publish"} 0 1628760831152
rtmp_conns{state="idle"} 0 1628760831152
rtmp_conns{state="read"} 0 1628760831152
rtmp_conns{state="publish"} 1 1628760831152
```

### HTTP API
where:

The server can be queried and controlled with an HTTP API, that must be enabled by setting the `api` parameter in the configuration:
* `paths{state="ready"}` is the count of paths that are ready
* `paths{state="notReady"}` is the count of paths that are not ready
* `rtsp_sessions{state="idle"}` is the count of RTSP sessions that are idle
* `rtsp_sessions{state="read"}` is the count of RTSP sessions that are reading
* `rtsp_sessions{state="publish"}` is the counf ot RTSP sessions that are publishing
* `rtsps_sessions{state="idle"}` is the count of RTSPS sessions that are idle
* `rtsps_sessions{state="read"}` is the count of RTSPS sessions that are reading
* `rtsps_sessions{state="publish"}` is the counf ot RTSPS sessions that are publishing
* `rtmp_conns{state="idle"}` is the count of RTMP connections that are idle
* `rtmp_conns{state="read"}` is the count of RTMP connections that are reading
* `rtmp_conns{state="publish"}` is the count of RTMP connections that are publishing

```yml
api: yes
```
### PProf

The API listens on `apiAddress`, that by default is `127.0.0.1:9997`; for instance, to obtain a list of active paths, run:
A performance monitor, compatible with pprof, can be enabled with the parameter `pprof: yes`; then the server can be queried for metrics with pprof-compatible tools, like:

```
curl http//127.0.0.1:9997/list
go tool pprof -text http://localhost:9999/debug/pprof/goroutine
go tool pprof -text http://localhost:9999/debug/pprof/heap
go tool pprof -text http://localhost:9999/debug/pprof/profile?seconds=30
```

Full documentation of the API is available on the [dedicated site](https://aler9.github.io/rtsp-simple-server/).

### Command-line usage

```
Expand Down
70 changes: 25 additions & 45 deletions internal/core/api.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,10 @@ import (
"github.com/aler9/rtsp-simple-server/internal/logger"
)

func interfaceIsEmpty(i interface{}) bool {
return reflect.ValueOf(i).Kind() != reflect.Ptr || reflect.ValueOf(i).IsNil()
}

func fillStruct(dest interface{}, source interface{}) {
rvsource := reflect.ValueOf(source)
rvdest := reflect.ValueOf(dest)
Expand Down Expand Up @@ -149,6 +153,7 @@ type apiPathsListData struct {
}

type apiPathsListRes1 struct {
Data *apiPathsListData
Paths map[string]*path
Err error
}
Expand All @@ -157,13 +162,9 @@ type apiPathsListReq1 struct {
Res chan apiPathsListRes1
}

type apiPathsListRes2 struct {
Err error
}

type apiPathsListReq2 struct {
Data *apiPathsListData
Res chan apiPathsListRes2
Res chan struct{}
}

type apiRTSPSessionsListItem struct {
Expand All @@ -176,13 +177,12 @@ type apiRTSPSessionsListData struct {
}

type apiRTSPSessionsListRes struct {
Err error
}

type apiRTSPSessionsListReq struct {
Data *apiRTSPSessionsListData
Err error
}

type apiRTSPSessionsListReq struct{}

type apiRTSPSessionsKickRes struct {
Err error
}
Expand All @@ -201,12 +201,12 @@ type apiRTMPConnsListData struct {
}

type apiRTMPConnsListRes struct {
Err error
Data *apiRTMPConnsListData
Err error
}

type apiRTMPConnsListReq struct {
Data *apiRTMPConnsListData
Res chan apiRTMPConnsListRes
Res chan apiRTMPConnsListRes
}

type apiRTMPConnsKickRes struct {
Expand Down Expand Up @@ -495,44 +495,32 @@ func (a *api) onConfigPathsDelete(ctx *gin.Context) {
}

func (a *api) onPathsList(ctx *gin.Context) {
data := apiPathsListData{
Items: make(map[string]apiPathsItem),
}

res := a.pathManager.OnAPIPathsList(apiPathsListReq1{})
if res.Err != nil {
ctx.AbortWithStatus(http.StatusInternalServerError)
return
}

for _, pa := range res.Paths {
pa.OnAPIPathsList(apiPathsListReq2{Data: &data})
}

ctx.JSON(http.StatusOK, data)
ctx.JSON(http.StatusOK, res.Data)
}

func (a *api) onRTSPSessionsList(ctx *gin.Context) {
if reflect.ValueOf(a.rtspServer).IsNil() {
if interfaceIsEmpty(a.rtspServer) {
ctx.AbortWithStatus(http.StatusNotFound)
return
}

data := apiRTSPSessionsListData{
Items: make(map[string]apiRTSPSessionsListItem),
}

res := a.rtspServer.OnAPIRTSPSessionsList(apiRTSPSessionsListReq{Data: &data})
res := a.rtspServer.OnAPIRTSPSessionsList(apiRTSPSessionsListReq{})
if res.Err != nil {
ctx.AbortWithStatus(http.StatusInternalServerError)
return
}

ctx.JSON(http.StatusOK, data)
ctx.JSON(http.StatusOK, res.Data)
}

func (a *api) onRTSPSessionsKick(ctx *gin.Context) {
if reflect.ValueOf(a.rtspServer).IsNil() {
if interfaceIsEmpty(a.rtspServer) {
ctx.AbortWithStatus(http.StatusNotFound)
return
}
Expand All @@ -549,26 +537,22 @@ func (a *api) onRTSPSessionsKick(ctx *gin.Context) {
}

func (a *api) onRTSPSSessionsList(ctx *gin.Context) {
if reflect.ValueOf(a.rtspsServer).IsNil() {
if interfaceIsEmpty(a.rtspsServer) {
ctx.AbortWithStatus(http.StatusNotFound)
return
}

data := apiRTSPSessionsListData{
Items: make(map[string]apiRTSPSessionsListItem),
}

res := a.rtspsServer.OnAPIRTSPSessionsList(apiRTSPSessionsListReq{Data: &data})
res := a.rtspsServer.OnAPIRTSPSessionsList(apiRTSPSessionsListReq{})
if res.Err != nil {
ctx.AbortWithStatus(http.StatusInternalServerError)
return
}

ctx.JSON(http.StatusOK, data)
ctx.JSON(http.StatusOK, res.Data)
}

func (a *api) onRTSPSSessionsKick(ctx *gin.Context) {
if reflect.ValueOf(a.rtspsServer).IsNil() {
if interfaceIsEmpty(a.rtspsServer) {
ctx.AbortWithStatus(http.StatusNotFound)
return
}
Expand All @@ -585,22 +569,18 @@ func (a *api) onRTSPSSessionsKick(ctx *gin.Context) {
}

func (a *api) onRTMPConnsList(ctx *gin.Context) {
if reflect.ValueOf(a.rtmpServer).IsNil() {
if interfaceIsEmpty(a.rtmpServer) {
ctx.AbortWithStatus(http.StatusNotFound)
return
}

data := apiRTMPConnsListData{
Items: make(map[string]apiRTMPConnsListItem),
}

res := a.rtmpServer.OnAPIRTMPConnsList(apiRTMPConnsListReq{Data: &data})
res := a.rtmpServer.OnAPIRTMPConnsList(apiRTMPConnsListReq{})
if res.Err != nil {
ctx.AbortWithStatus(http.StatusInternalServerError)
return
}

ctx.JSON(http.StatusOK, data)
ctx.JSON(http.StatusOK, res.Data)
}

// OnConfReload is called by core.
Expand All @@ -611,7 +591,7 @@ func (a *api) OnConfReload(conf *conf.Conf) {
}

func (a *api) onRTMPConnsKick(ctx *gin.Context) {
if reflect.ValueOf(a.rtmpServer).IsNil() {
if interfaceIsEmpty(a.rtmpServer) {
ctx.AbortWithStatus(http.StatusNotFound)
return
}
Expand Down
Loading

0 comments on commit f1a812b

Please sign in to comment.