Enabling Metric/Log retention based on size

[ayush-rathore-quartic]

As given a choice in Prometheus with the --storage.tsdb.retention.size flag, I think it would be of great value to allow users to enable log/metric/trace retention based on the size of the total data, along with the current time based retention feature.

Most environments do not have a static enough metric/log stream to be able to appropriately size their Clickhouse disk , purely based on the time period. Meanwhile, it is also not a straight forward process to recover if your clickhouse disk is full

Let me know your thoughts, Team!

[jkomulai]

This would be a nice feature!

I have an idea for further development. I've used Graylog before and I liked that you can set different retention policy for different types of logs. For example, syslog logs are placed in indexes where retention time is 60 days and other logs are placed in indexes where retention is based on number of log rows. In this way, important logs can be kept for a certain period of time and other logs can't flood them out. A feature like this would be nice in SigNoz.

[nitishfy]

cc: @srikanthccv