chore(grouping): migration for hnsw index

[trillville]

based on analysis here

[JoshFerge]

we should turn ingestion off before merging this, just to be safe and observe what the database does

[trillville]

we should turn ingestion off before merging this, just to be safe and observe what the database does

i've updated my migration so it applies concurrently, we should be good to apply without pausing ingestion

[trillville]

I've updated this migration to do the following:

1. switch ef_construction to 200
2. apply hash partitioning on project_id with 100 different partitions

I've also made this apply in a single txn rather than concurrently, with the intention being that we run this with ingestion paused.

[mrduncan]

Added a couple of comments but neither are necessarily blocking, especially since this table won't be used until a followup migration swaps it out

[mrduncan]

Sorry, just noticed this - should this be just id? If feels surprising that id wouldn't be unique

[trillville]

this is necessary for hash partitioning AFAIK (cant hash partition on something that isnt PK)

[mrduncan]

Ahh, that makes sense.

If we have any places where we're querying with only an id we should see what the query planner does - theoretically the id, project_id index should be able to be applied to just an id query but I'm not super familiar with how partitioning affects that.