a profiler for time series data - breaking changes may occur, since currently under development.
+------------+ +------------+ +---------------+
| TimeSeries | +---> | TSProfiler | +--> | Statistical |
| Data | | | | Profile |
+------------+ +------------+ +---------------+
TSProfiler provides a go implementation to convert time series stream data like monitoring data online into statistical representative profiles. TSProfiler is integrated into the KVM monitoring tool kvmtop directly, or for distributed setups into the DisResc Monitoring Library.
The core concept bases on Markov Chains to represent the probability of a discretized utilisation states, and a Decision Tree to handle periodic recurrent Markov transition matrices (the period tree). Automatic phase detection further identifies pattern changes.
The TSProfiler comes with a command line tool to read a CSV file and generate a TSProfile. Get the most recent stable build from Releases.
Usage:
csv2tsprofile [OPTIONS]
Reads time series values from a CSV file and generates a tsprofile
Application Options:
--states=
--buffersize=
--history=
--filterstddevs=
--fixedbound
--fixedmin= if fixedbound is set, set the min value (default: 0)
--fixedmax= if fixedbound is set, set the max value (default: 100)
--periodsize= comma separated list of ints, specifies descrete states per period
--phasechangelikeliness=
--phasechangehistory=
--output= path to write profile to, stdout if '-' (default: -)
--out.history= path to write last historic values to, stdout if '-', empty to disable
--out.phases=
--out.periods=
--out.states=
Help Options:
-h, --help Show this help message
Example: csv2tsprofile --states 4 --history 1 --filterstddevs 4 --buffersize 6 --periodsize 2,24,48 path/to/tsinput.csv
The TSPredictor reads a TSProfile and the current position to provide simulation or likeliness calculations for future next states. The mode can be either 0 (root tx), 1 (detected phases), or 2 (periods). Simulation or likeliness has to be specified as the requested task.
Usage:
tspredictor [OPTIONS]
Reads a TSProfile from file and runs tasks on in (Simulate or Likeliness)
Application Options:
--steps=
--mode=
--periodDepth=
-p, --profile=
-h, --history=
Help Options:
-h, --help Show this help message
Example (with csv2tsprofile):
csv2tsprofile \
--fixedbound \
--fixedmin 0 \
--fixedmax 100 \
--states 10 \
--buffersize 1 \
--history 1 \
--periodsize 16,4 \
--phasechangelikeliness 0.50 \
--phasechangehistory 10 \
--out.history /tmp/history.json \
--output /tmp/profile.json \
--out.phases /tmp/out.phases.log \
--out.periods /tmp/out.periods.log \
--out.states /tmp/out.states.log \
tsinput.csv
tspredictor \
--profile /tmp/profile.json \
--history /tmp/history.json \
--steps 4 \
--mode 0 \
simulate
Create a new TSProfiler:
tsprofiler := profiler.NewProfiler(models.Settings{
Name: "profiler-hostX",
BufferSize: 10,
States: 4,
FilterStdDevs: 4,
History: 1,
FixBound: false,
PeriodSize: []int{60,720,1440},
// ... many more settings
OutputFreq: time.Duration(20) * time.Second,
OutputCallback: profileOutput,
})
func profileOutput(data models.TSProfile) {
// handle profiler output via OutputFreq
}
// get profile independently of OutputFreq
profile := profiler.Get()Provide metric value to profiler:
metrics := make([]models.TSInputMetric, 0)
metrics = append(metrics, models.TSInputMetric{
Name: "CPU-Util",
Value: float64(utilValue),
FixedMin: options.FixedMin, // optional, required for FixBound = true
FixedMax: options.FixedMax, // optional, required for FixBound = true
})
tsinput := models.TSInput{
Metrics: metrics,
}
profiler.Put(tsinput)