chore: use map to reduce writes to clickhouse tag tables

[makeavish]

fix: https://github.com/SigNoz/engineering-pod/issues/1028

[srikanthccv]

Please share how you measured the change and the difference observed with some non-trivial workloads.

[makeavish]

With this change CPU time of writeTagBatch decreases by 30-70% depending on the duplicates attributes in batch.
Tested this with Flask application running at 100rps. Didn't see any significant changes in memory profile.

[srikanthccv]

I think 100 rps is not enough for this kind of benchmark. Can you use some synthetic load generator like https://github.com/Omnition/synthetic-load-generator to create a much higher load around 5k-10k span/s? I would put a load on the collector as much as it starts to give the context deadline error and see when it reaches this peak state before and after the change. And it helps if you share the before and after profiles as well.

[makeavish]

I think 100 rps is not enough for this kind of benchmark. Can you use some synthetic load generator like https://github.com/Omnition/synthetic-load-generator to create a much higher load around 5k-10k span/s? I would put a load on the collector as much as it starts to give the context deadline error and see when it reaches this peak state before and after the change. And it helps if you share the before and after profiles as well.

I think synthetic load generator doesn't reproduce real-world use case well so tried 1000rps(generating around 5-6k spans/s) with flask app with DB calls. The result was the same.

[ankitnayan]

this PR should be part of next release