- Data Munging and Cleaning - First clean the data set as much as we can, i.e filter only what we want, rename variables for clean format, take all null values from all the datasets. I have mostly done this and but I am still working on the 311 dataset. Sana has worked on the zipcodeconversion notebook. So in total we will have 3 jupyter notebooks- sharedmobility, 311 and sharemobility_merged_zip.
- Merge Zipcode to Shared mobility. This step is done by Sana today but would be helpful if can tally and crosscheck the data accuracy of the merge dataframe. Again analze this merged data set and clean up.
- Convert these two notebook datasets (shared_mobility_merged_zip and 311_complaints) to a clean_df.csv so that we create read it in a new notebook and start visualization using these csv files instead api requests everytime. Lets take up visualization tasks on Tuesday.
- Recognize all the hypothesis questions and tackle them one by one. Plotting comes as a part of this.
- Decide on visualization and glyphs types? Graphs should be “More Data, Less Ink”, Easy to read but not boring.
- Converting dataframe to sql. Need to watch Eds video on this.
- Create a a notebook ‘Presentation.ipynb’. This will be our final notebook for the presentation.
After cleaning and data munging:
- shared_mobility_AODP
- 311_AODP
- shared_mobility_merged_zip (This can be done in shared_mobility_AODP too)
Convert these df into csv in their respective notebooks and then load them into a new notebook:
- scooter_data_analysis (for analysis and plotting - this is our main master)
After saving plots and answering our various hypothesis Qs:
- presentation.ipynb