Skip to content

Commit

Permalink
examples - add symlinks to advanced exapled, update README
Browse files Browse the repository at this point in the history
  • Loading branch information
@SiskaPavel
SiskaPavel committed Oct 7, 2024
1 parent e20088e commit 5df9ed9
Show file tree
Hide file tree
Showing 5 changed files with 96 additions and 48 deletions.
45 changes: 27 additions & 18 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -104,16 +104,17 @@ $ cat /tmp/telemetry/*
## Advanced Example

The advanced example demonstrates a telemetry data structure that organizes and stores metrics
from multiple servers located in different data centers.
from multiple servers located in different data centers, utilizing symbolic links for easier
access by location and ID.

### Advanced Example Directory Structure
After running the advanced example, the resulting directory structure is as follows:

```bash
$ tree /tmp/telemetry
/tmp/telemetry
$ tree /tmp/telemetry/
tmp/telemetry/
└── data_centers
├── new_york
├── 0-prague
│ ├── server_count
│ ├── servers
│ │ ├── server_0
Expand All @@ -124,7 +125,7 @@ $ tree /tmp/telemetry
│ │ └── stats
│ └── summary
│ └── summary_stats
├── prague
├── 1-new_york
│ ├── server_count
│ ├── servers
│ │ ├── server_0
Expand All @@ -135,25 +136,33 @@ $ tree /tmp/telemetry
│ │ └── stats
│ └── summary
│ └── summary_stats
└── tokyo
├── server_count
├── servers
│ ├── server_0
│ │ └── stats
│ ├── server_1
│ │ └── stats
│ └── server_2
│ └── stats
└── summary
└── summary_stats
├── 2-tokyo
│ ├── server_count
│ ├── servers
│ │ ├── server_0
│ │ │ └── stats
│ │ ├── server_1
│ │ │ └── stats
│ │ └── server_2
│ │ └── stats
│ └── summary
│ └── summary_stats
├── by-id
│ ├── 0 -> ../0-prague
│ ├── 1 -> ../1-new_york
│ └── 2 -> ../2-tokyo
└── by-location
├── new_york -> ../1-new_york
├── prague -> ../0-prague
└── tokyo -> ../2-tokyo
```

### Example Telemetry Output
Telemetry metrics are stored in files such as stats, which contain real-time data.
Below is an example of the contents of a stats file for a specific server:

```bash
$ cat /tmp/telemetry/data_centers/prague/servers/server_0/stats
$ cat /tmp/telemetry/data_centers/0-prague/servers/server_0/stats
cpu_usage: 74.28 (%)
disk_usage: 13.48 (%)
latency: 170.20 (ms)
Expand All @@ -164,7 +173,7 @@ timestamp: 2024-10-03 15:18:41
You can also view summary statistics for a data center by reading the summary_stats file:

```bash
$ cat /tmp/telemetry/data_centers/prague/summary/summary_stats
$ cat /tmp/telemetry/data_centers/by-location/prague/summary/summary_stats
cpu_usage [avg]: 44.20 (%)
disk_usage [avg]: 35.78 (%)
latency [avg]: 121.82 (ms)
Expand Down
44 changes: 26 additions & 18 deletions examples/README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ This example demonstrates two different telemetry applications: a simple example
The simple example showcases the basic functionality of the telemetry system, providing essential information
such as process ID, start time, uptime, and version. The advanced example, on the other hand, offers a more
comprehensive telemetry data structure that organizes and stores metrics from multiple servers located in
different data centers.
different data centers, utilizing symbolic links for easier access by location and ID.

## Simple Example

Expand Down Expand Up @@ -42,10 +42,21 @@ telemetry metrics such as CPU usage, memory usage, latency, and disk usage.
The resulting directory structure after running the advanced example is as follows:

```bash
$ tree /tmp/telemetry
/tmp/telemetry
$ tree /tmp/telemetry/
tmp/telemetry/
└── data_centers
├── new_york
├── 0-prague
│ ├── server_count
│ ├── servers
│ │ ├── server_0
│ │ │ └── stats
│ │ ├── server_1
│ │ │ └── stats
│ │ └── server_2
│ │ └── stats
│ └── summary
│ └── summary_stats
├── 1-new_york
│ ├── server_count
│ ├── servers
│ │ ├── server_0
Expand All @@ -56,7 +67,7 @@ $ tree /tmp/telemetry
│ │ └── stats
│ └── summary
│ └── summary_stats
├── prague
├── 2-tokyo
│ ├── server_count
│ ├── servers
│ │ ├── server_0
Expand All @@ -67,25 +78,22 @@ $ tree /tmp/telemetry
│ │ └── stats
│ └── summary
│ └── summary_stats
└── tokyo
├── server_count
├── servers
│ ├── server_0
│ │ └── stats
│ ├── server_1
│ │ └── stats
│ └── server_2
│ └── stats
└── summary
└── summary_stats
├── by-id
│ ├── 0 -> ../0-prague
│ ├── 1 -> ../1-new_york
│ └── 2 -> ../2-tokyo
└── by-location
├── new_york -> ../1-new_york
├── prague -> ../0-prague
└── tokyo -> ../2-tokyo
```

### Example of Telemetry Output

Here is an example of the contents of the `stats` file for a specific server:

```bash
$ cat /tmp/telemetry/data_centers/prague/servers/server_0/stats
$ cat /tmp/telemetry/data_centers/0-prague/servers/server_0/stats
cpu_usage: 74.28 (%)
disk_usage: 13.48 (%)
latency: 170.20 (ms)
Expand All @@ -96,7 +104,7 @@ timestamp: 2024-10-03 15:18:41
Example of the contents of the `summary_stats` file for a data center:

```bash
$ cat /tmp/telemetry/data_centers/prague/summary/summary_stats
$ cat /tmp/telemetry/data_centers/by-location/prague/summary/summary_stats
cpu_usage [avg]: 44.20 (%)
disk_usage [avg]: 35.78 (%)
latency [avg]: 121.82 (ms)
Expand Down
7 changes: 6 additions & 1 deletion examples/advanced/dataCenter.cpp
View file
Original file line number Diff line number Diff line change
Expand Up @@ -43,10 +43,15 @@ static std::shared_ptr<telemetry::AggregatedFile> createSummaryFile(
return dir->addAggFile(filename, filePattern, aggOps, patternRootDir);
}

DataCenter::DataCenter(std::string location, std::shared_ptr<telemetry::Directory>& dataCenterDir)
DataCenter::DataCenter(
std::string location,
uint64_t dataCenterId,
std::shared_ptr<telemetry::Directory>& dataCenterDir)
: m_rootDir(dataCenterDir)
, m_location(std::move(location))
, m_dataCenterId(dataCenterId)
{
(void) m_dataCenterId;
setupTelemetry(dataCenterDir);
}

Expand Down
6 changes: 5 additions & 1 deletion examples/advanced/dataCenter.hpp
View file
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,10 @@ class DataCenter {
* @param location The physical location of the data center.
* @param dataCenterDir Shared pointer to the telemetry directory for this data center.
*/
DataCenter(std::string location, std::shared_ptr<telemetry::Directory>& dataCenterDir);
DataCenter(
std::string location,
uint64_t dataCenterId,
std::shared_ptr<telemetry::Directory>& dataCenterDir);

/**
* @brief Adds a server to the data center.
Expand All @@ -58,6 +61,7 @@ class DataCenter {
std::shared_ptr<telemetry::Directory>
m_rootDir; ///< Pointer to the root data center telemetry directory.
std::string m_location; ///< The location of the data center.
uint64_t m_dataCenterId; ///< The unique identifier of the data center.
telemetry::Holder m_holder; ///< Holder for managing telemetry files.
std::vector<Server> m_servers; ///< Vector to store added servers.
};
Expand Down
42 changes: 32 additions & 10 deletions examples/advanced/main.cpp
View file
Original file line number Diff line number Diff line change
Expand Up @@ -46,18 +46,30 @@ void signalHandler(int signum)
/**
* @brief Creates a DataCenter object for a given location.
*
* This function initializes a DataCenter for the specified location and associates it with a
* This function initializes a DataCenter for the specified location, id and associates it with a
* telemetry directory.
*
* @param location The location of the data center.
* @param dataCenterId The unique identifier of the data center.
* @param dataCentersDir Shared pointer to the directory where the data center will be created.
* @param holder The telemetry holder for managing telemetry files.
* @return A DataCenter object initialized for the specified location.
*/
DataCenter
createDataCenter(const std::string& location, std::shared_ptr<telemetry::Directory>& dataCentersDir)
DataCenter createDataCenter(
const std::string& location,
uint64_t& dataCenterId,
std::shared_ptr<telemetry::Directory>& dataCentersDir,
telemetry::Holder& holder)
{
auto dataCenterDir = dataCentersDir->addDir(location);
return {location, dataCenterDir};
auto dataCenterDir = dataCentersDir->addDir(std::to_string(dataCenterId) + "-" + location);
const auto symlinkByLocation
= dataCentersDir->addDir("by-location")->addSymlink(location, dataCenterDir);
const auto symlinkById
= dataCentersDir->addDir("by-id")->addSymlink(std::to_string(dataCenterId), dataCenterDir);
holder.add(symlinkByLocation);
holder.add(symlinkById);

return {location, dataCenterId++, dataCenterDir};
}

/**
Expand All @@ -66,14 +78,17 @@ createDataCenter(const std::string& location, std::shared_ptr<telemetry::Directo
* This function initializes a set of data centers, each containing a predefined number of servers.
*
* @param dataCentersDir Shared pointer to the directory where the data centers will be created.
* @param holder The telemetry holder for managing telemetry files.
* @return A vector of initialized DataCenter objects.
*/
std::vector<DataCenter> createDataCenters(std::shared_ptr<telemetry::Directory>& dataCentersDir)
std::vector<DataCenter>
createDataCenters(std::shared_ptr<telemetry::Directory>& dataCentersDir, telemetry::Holder& holder)
{
uint64_t dataCenterId = 0;
std::vector<DataCenter> dataCenters;
dataCenters.emplace_back(createDataCenter("prague", dataCentersDir));
dataCenters.emplace_back(createDataCenter("new_york", dataCentersDir));
dataCenters.emplace_back(createDataCenter("tokyo", dataCentersDir));
dataCenters.emplace_back(createDataCenter("prague", dataCenterId, dataCentersDir, holder));
dataCenters.emplace_back(createDataCenter("new_york", dataCenterId, dataCentersDir, holder));
dataCenters.emplace_back(createDataCenter("tokyo", dataCenterId, dataCentersDir, holder));

const std::size_t serversPerDatacenter = 3;

Expand Down Expand Up @@ -105,8 +120,15 @@ int main(int argc, char** argv)
// create telemetry root directory
telemetryRootDirectory = telemetry::Directory::create();

/**
* The telemetry holder for managing telemetry files. It ensures that the files are
* not prematurely destroyed. Because the telemetry directory holds weak pointers to the
* files, the holder ensures that the files are not deleted until the holder is destroyed.
*/
telemetry::Holder holder;

auto dataCentersDir = telemetryRootDirectory->addDir("data_centers");
auto dataCenters = createDataCenters(dataCentersDir);
auto dataCenters = createDataCenters(dataCentersDir, holder);

const bool tryToUnmountOnStart = true;
const bool createMountPoint = true;
Expand Down

0 comments on commit 5df9ed9

Please sign in to comment.