SI-2749-first-and-last-agg

[Allyson-English]

Return the first and last values, chronologically, that appear in the bucket
ex:

• argMin: return value associated with minimum timestamp
• argMax: return value associated with maximum timestamp

Note:

• I changed how we were inserting dummy string values in insertTestData to make sure that the first string value and the last string value were distinct so that the test was more clear. This also required me to update the expected response for StringAggregationUnique

[linear]

SI-2749 First and last signal aggregations in Telemetry

[elffjs]

Validate the assumption. I was afraid so I suggested something else.

[elffjs]

Early experiments seem pretty good.

[elffjs]

[elffjs]

@KevinJoiner If this does work then we can probably replace the argmaxes for latest signals.

[KevinJoiner]

Unfortunately, I don't think we want to use these since they are not deterministic, since first_value is an alias for any

danger
The query can be executed in any order and even in a different order each time, so the result of this function is indeterminate. To get a determinate result, you can use the min or max function instead of any.

-- https://clickhouse.com/docs/en/sql-reference/aggregate-functions/reference/any

anyLast

Selects the last value encountered. The result is just as indeterminate as for the any function.

--https://clickhouse.com/docs/en/sql-reference/aggregate-functions/reference/anylast

[elffjs]

From the first_value doc

As with any, without Window Functions the result will be random if the source stream is not ordered and the return type matches the input type

Maybe the bucket here is an ordered stream (inheriting some ordering from the index). It seems to behave that way.

I’m nervous.

[Allyson-English]

fwiw, this is what I was referencing when initially thinking that the source was ordered https://github.com/DIMO-Network/telemetry-api/blob/main/internal/service/ch/queries.go#L316

[elffjs]

That’s ordering the buckets.

[KevinJoiner]

I think it is ordered based on the signal table's create statement, but I don't love relying on that. max and min seem safest unless we hit some performance problems

[KevinJoiner]

I think you need to rebase with main to get CI working.