Skip to content

Commit

Permalink
refine metrics type and large refactor
Browse files Browse the repository at this point in the history
- drop support for Scala 2
- add support for Scala Native
- add macro-generated refinements for groups of metrics, adding
  compile-time safeties for setting declared labels
  • Loading branch information
jodersky committed Nov 4, 2022
1 parent 6711e0a commit 8c8ac8b
Show file tree
Hide file tree
Showing 27 changed files with 637 additions and 576 deletions.
58 changes: 22 additions & 36 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,18 +4,12 @@
[![ustats Scala version support](https://index.scala-lang.org/jodersky/ustats/ustats/latest.svg)](https://index.scala-lang.org/jodersky/ustats/ustats)
[![stability: soft](https://img.shields.io/badge/stability-soft-white)](https://www.crashbox.io/stability.html)

A simple and intuitive metrics collection library for Prometheus.

## Highlights

- ease of use: *does what you want it to do without any setup ceremony*

- efficiency: *light memory footprint and extremely fast*
A simple and intuitive metrics collection library.

## Getting Started

ustats is available from maven central (for Scala 2.13 and Dotty). Add its
coordinates to your build config:
μstats is available from maven central for Scala 3.1 and above, for the JVM and
Scala Native. Add its coordinates to your build config:

- mill: `ivy"io.crashbox::ustats:<latest_version>"`
- sbt: `"io.crashbox" %% "ustats" % "<latest_version>"`
Expand All @@ -28,10 +22,10 @@ where `<latest_version>` is
Basic example, create a counter using the default collector:

```scala
val myCounter = ustats.counter("my_counter", "This is just a simple counter.")
val myCounter = ustats.global.counter("my_counter", "This is just a simple counter.")
myCounter += 1

println(ustats.metrics())
println(ustats.global.metrics())

// # HELP my_counter This is just a simple counter.
// # TYPE my_counter counter
Expand All @@ -41,11 +35,11 @@ println(ustats.metrics())
You can also add label-value pairs to individual metrics:

```scala
val myGauge = ustats.gauge("my_gauge", labels = Seq("label1" -> "foo", "label2" -> 42))
val myGauge = ustats.global.gauge("my_gauge", labels = Seq("label1" -> "foo", "label2" -> 42))
myGauge += 1
myGauge += 2

println(ustats.metrics())
println(ustats.global.metrics())
// # TYPE my_counter gauge
// my_gauge{label1="foo", label2="42"} 3.0
```
Expand All @@ -54,27 +48,27 @@ However, you'd usually want to declare one metric sharing a common basename, and
add labels on demand:

```scala
val queueSizes = ustats.gauges("queue_size", labels = Seq("queue_name"))
val queueSizes = ustats.global.gauges("queue_size").labelled("queue_name")

queueSizes.labelled("queue1") += 10
queueSizes.labelled("queue1") -= 1
queueSizes.labelled("queue2") += 2
queueSizes(queue_name = "queue1") += 10
queueSizes(queue_name = "queue1") -= 1
queueSizes(queue_name = "queue2") += 2

println(ustats.metrics())
println(ustats.global.metrics())
// # TYPE queue_size gauge
// queue_size{queue_name="queue1"} 9.0
// queue_size{queue_name="queue2"} 2.0
```

Use your own collector:
User-defined grouping of metrics:

```scala
val collector = new ustats.Stats()
val mymetrics = ustats.Metrics()

val currentUsers = collector.gauge("my_app_current_users")
val currentUsers = mymetrics.gauge("my_app_current_users")
currentUsers += 10
currentUsers -= 1
println(collector.metrics())
println(mymetrics.metrics())
```

## Probing
Expand All @@ -85,23 +79,15 @@ modifying it. ustats has a builtin "probe" mechanism to run batch jobs
repeatedly at customizable intervals.

```scala
val counter1 = ustats.counter("counter1")
val gauge1 = ustats.gauge("gauge1")
val counter1 = ustats.global.counter("counter1")
val gauge1 = ustats.global.gauge("gauge1")

// run this action every 10 seconds
ustats.probe("query_database", 10){
ustats.global.probe("query_database", 10){
// query database
counter1 += 1
gauge1.set(42)
}

// also works with async code
ustats.probe.async("query_database", 10) { implicit ec =>
val f: Future[_] = // something that returns a Future[_]
f.map{ _ =>
counter1 += 1
}
}
```

Note that failures of probes themselves are recorded and exposed as a metric.
Expand All @@ -117,11 +103,11 @@ over HTTP, under the standard `/metrics` endpoint. The server module is based on

```scala
// global server for global stats
ustats.server.start("localhost", 10000)
ustats.server.global.start("localhost", 10000)

// custom server for custom stats
val stats = new ustats.Stats()
val server = new ustats.MetricsServer(stats)
val metrics = ustats.Metrics()
val server = ustats.server.MetricsServer(metrics)
server.start("localhost", 10000)
```

Expand Down
2 changes: 1 addition & 1 deletion benchmark/src/Counter.scala
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ import org.openjdk.jmh.annotations.{Benchmark, BenchmarkMode, Mode, OutputTimeUn

@State(Scope.Benchmark)
class state {
val collector = new Stats()
val collector = new Metrics()

val counter = collector.counter("counter")
}
Expand Down
76 changes: 33 additions & 43 deletions build.sc
Original file line number Diff line number Diff line change
@@ -1,10 +1,9 @@
import $file.jmh
import jmh.Jmh
import mill._, scalalib._, scalafmt._, publish._
import mill._, scalalib._, scalafmt._, publish._, scalanativelib._

val scala213 = "2.13.7"
val scala3 = "3.0.2"
val dottyCustomVersion = Option(sys.props("dottyVersion"))
val scala3 = "3.1.2"
val scalaNative = "0.4.5"

trait Publish extends PublishModule {
def publishVersion = "0.5.0"
Expand All @@ -21,76 +20,67 @@ trait Publish extends PublishModule {
}

trait Utest extends ScalaModule with TestModule {
def ivyDeps = Agg(ivy"com.lihaoyi::utest:0.7.10")
def ivyDeps = Agg(ivy"com.lihaoyi::utest::0.7.11")
def testFramework = "utest.runner.Framework"
}

class UstatsModule(val crossScalaVersion: String)
extends CrossScalaModule
trait UstatsModule
extends ScalaModule
with ScalafmtModule
with Publish {
def scalaVersion = scala3
def artifactName = "ustats"
object test extends Tests with Utest
// FIXME: scaladoc 3 is not supported by mill yet. Remove the override
// once it is.
override def docJar =
if (crossScalaVersion.startsWith("2")) super.docJar
else T {
val outDir = T.ctx().dest
val javadocDir = outDir / 'javadoc
os.makeDir.all(javadocDir)
mill.api.Result.Success(mill.modules.Jvm.createJar(Agg(javadocDir))(outDir))
}
def ivyDeps = Agg(
ivy"com.lihaoyi::geny::1.0.0"
)
}

object ustats extends Cross[UstatsModule]((Seq(scala213, scala3) ++ dottyCustomVersion): _*) {
object ustats extends Module {

object jvm extends UstatsModule {
override def millSourcePath = super.millSourcePath / os.up
def sources = T.sources(super.sources() ++ Seq(PathRef(millSourcePath / "src-jvm")))
object test extends Tests with Utest
}
object native extends UstatsModule with ScalaNativeModule {
def scalaNativeVersion = scalaNative
override def millSourcePath = super.millSourcePath / os.up
def sources = T.sources(super.sources() ++ Seq(PathRef(millSourcePath / "src-native")))
object test extends Tests with Utest
}

class UstatsServerModule(val crossScalaVersion: String)
extends CrossScalaModule
with ScalafmtModule
with Publish {
def artifactName = "ustats-server"
def moduleDeps = Seq(ustats(crossScalaVersion))
object server extends ScalaModule with Publish {
def scalaVersion = scala3
def moduleDeps = Seq(ustats.jvm)
def ivyDeps = Agg(
ivy"io.undertow:undertow-core:2.2.3.Final"
ivy"io.undertow:undertow-core:2.3.0.Final"
)
object test extends Tests with Utest
// FIXME: scaladoc 3 is not supported by mill yet. Remove the override
// once it is.
override def docJar =
if (crossScalaVersion.startsWith("2")) super.docJar
else T {
val outDir = T.ctx().dest
val javadocDir = outDir / 'javadoc
os.makeDir.all(javadocDir)
mill.api.Result.Success(mill.modules.Jvm.createJar(Agg(javadocDir))(outDir))
}
}
object server extends Cross[UstatsServerModule]((Seq(scala213, scala3) ++ dottyCustomVersion): _*)

}

object benchmark extends ScalaModule with Jmh {
def scalaVersion = scala213
def moduleDeps = Seq(ustats(scala213))
def scalaVersion = scala3
def moduleDeps = Seq(ustats.jvm)
}

object examples extends Module {

trait Example extends ScalaModule {
def scalaVersion = scala213
def moduleDeps: Seq[ScalaModule] = Seq(ustats(scala213))
def scalaVersion = scala3
def moduleDeps: Seq[ScalaModule] = Seq(ustats.jvm)
}

object cask extends Example {
def ivyDeps = Agg(
ivy"com.lihaoyi::cask:0.7.9"
ivy"com.lihaoyi::cask:0.8.1"
)
}
object cask2 extends Example {
def ivyDeps = cask.ivyDeps
}
object probe extends Example {
def moduleDeps = Seq(ustats(scala213), ustats.server(scala213))
def moduleDeps = Seq(ustats.jvm, ustats.server)
}
}
6 changes: 3 additions & 3 deletions examples/cask/src/Main.scala
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
object Main extends cask.MainRoutes {

val httpRequestsSeconds = ustats.histogram("http_requests_seconds", labels = Seq("path" -> "/index"))
val randomFailures = ustats.counter("random_failures")
val httpRequestsSeconds = ustats.global.histogram("http_requests_seconds", labels = Seq("path" -> "/index"))
val randomFailures = ustats.global.counter("random_failures")

@cask.get("/")
def index() = httpRequestsSeconds.time {
Expand All @@ -12,7 +12,7 @@ object Main extends cask.MainRoutes {
}

@cask.get("/metrics")
def metrics() = ustats.metrics()
def metrics() = ustats.global.metrics()

initialize()

Expand Down
26 changes: 14 additions & 12 deletions examples/cask2/src/Main.scala
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ import io.undertow.server.HttpServerExchange
trait Timed extends cask.Main {

lazy val histograms = {
val hs = ustats.histograms("http_requests_seconds", labels = Seq("method", "path"))
val hs = ustats.global.histograms("http_requests_seconds").labelled("method", "path")

// prepopulate histogram with all known routes
for {
Expand All @@ -16,7 +16,7 @@ trait Timed extends cask.Main {
x: cask.router.EndpointMetadata[_]
)
m <- route.endpoint.methods
} hs.labelled(m, route.endpoint.path)
} hs(m, route.endpoint.path)

hs
}
Expand All @@ -27,15 +27,17 @@ trait Timed extends cask.Main {
val timedHandler = new HttpHandler {
def handleRequest(exchange: HttpServerExchange): Unit = {
val effectiveMethod = exchange.getRequestMethod.toString.toLowerCase()
val endpoint = routeTries(effectiveMethod).lookup(cask.internal.Util.splitPath(exchange.getRequestPath).toList, Map())

endpoint match {
case None => parent.handleRequest(exchange)
case Some(((_,metadata), _, _)) =>
histograms.labelled(effectiveMethod, metadata.endpoint.path).time(
parent.handleRequest(exchange)
)
}
val endpoint = dispatchTrie.lookup(cask.internal.Util.splitPath(exchange.getRequestPath).toList, Map())
endpoint match
case None =>
parent.handleRequest(exchange)
case Some((methodMap, routeBindings, remaining)) =>
methodMap.get(effectiveMethod) match
case None => parent.handleRequest(exchange)
case Some((routes, metadata)) =>
histograms(method = effectiveMethod, path = metadata.endpoint.path).time(
parent.handleRequest(exchange)
)
}
}

Expand All @@ -62,7 +64,7 @@ object Main extends cask.MainRoutes with Timed {
}

@cask.get("/metrics")
def metrics() = ustats.metrics()
def metrics() = ustats.global.metrics()

initialize()

Expand Down
13 changes: 6 additions & 7 deletions examples/probe/src/Main.scala
Original file line number Diff line number Diff line change
@@ -1,16 +1,16 @@
object Main extends App {

@main
def main() =
var database = collection.mutable.ListBuffer.empty[String]

val itemsTotal = ustats.gauge("items_total")
val itemsTotal = ustats.global.gauge("items_total")

ustats.probe("query_database", 10){
val l = database.length
ustats.global.probe("query_database", 10){
val l = synchronized {database.length}
if (l > 5) sys.error("random failure")
itemsTotal.set(l)
}

ustats.server.start("localhost", 8081)
ustats.server.global.start("localhost", 8081)

println("Go to http://localhost:8081/metrics to see current metrics.")
println("Ctrl+D to exit")
Expand All @@ -25,4 +25,3 @@ object Main extends App {
}

println("bye")
}
2 changes: 1 addition & 1 deletion mill
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
# This is a wrapper script, that automatically download mill from GitHub release pages
# You can give the required mill version with MILL_VERSION env variable
# If no version is given, it falls back to the value of DEFAULT_MILL_VERSION
DEFAULT_MILL_VERSION=0.9.7
DEFAULT_MILL_VERSION=0.10.8

set -e

Expand Down
Loading

0 comments on commit 8c8ac8b

Please sign in to comment.