You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
We currently don't have a built in feature to search for specific property values in a datasets. Currently we solve this by piping the whole dataset to jq.
This can be very memory demanding for large datasets and quite unreliable. I propose that until we have a proper search to handle this in the backend, we add a command that will do more or less the same in cli. That would allow us to go through the dataset batch by batch, avoiding filling up the memory with temp entities.
We currently don't have a built in feature to search for specific property values in a datasets. Currently we solve this by piping the whole dataset to jq.
This can be very memory demanding for large datasets and quite unreliable. I propose that until we have a proper search to handle this in the backend, we add a command that will do more or less the same in cli. That would allow us to go through the dataset batch by batch, avoiding filling up the memory with temp entities.
It could look something like this:
I suggest that we also add a pterm progressbar output while searching. https://github.com/pterm/pterm/tree/master/_examples/progressbar
The text was updated successfully, but these errors were encountered: