- Status: accepted
- Date: 2022-05-12
We want to store processed data in efficient maner. Which format should we use?
- Matlab
- Parquet/Avro/Arrow
- CSV
Chosen option: Parquet, because
- Parquet files is the most efficient option for quering via Athena.
- Parquet files more cost effective option compared with .csv.
- Parquet can be asily parsed using Python by data science team.