[QoS Oracle V2] Processing pipeline for the Oracle #4
Labels
meta:triaged
This issue has been triaged (has a good description, as well as labels for priority, size and type)
p1
High priority
size:large
Large
type:feature
New or enhanced functionality
Depending on the implementation of the Kafka consumer, we might need to either have some more complex aggregation/processing pipeline or not, but we need to make sure our time buckets are properly processed to be published to IPFS.
This means we have to make sure the data points received from Kafka are aggregated properly per time bucket (5m), thus ensuring all relevant data points are included into the aggregation for that bucket, and afterwards, for that IPFS file.
As an example, if we have multiple data points that denote different indexers QoS data that all share the same bucket, we must ensure that not only the aggregations for each of those indexers data points are done correctly, but that all of those indexer-wide aggregations are included in the IPFS file for that particular bucket.
The text was updated successfully, but these errors were encountered: