Skip to content

Latest commit

 

History

History
33 lines (25 loc) · 2.75 KB

README.md

File metadata and controls

33 lines (25 loc) · 2.75 KB
Raw

voice-grabber

This repo is a collection of scripts to download the dataset necessary to train the jibjib-model

Repo layout

The complete list of JibJib repos is:

  • jibjib: Our Android app. Records sounds and looks fantastic.
  • deploy: Instructions to deploy the JibJib stack.
  • jibjib-model: Code for training the machine learning model for bird classification
  • jibjib-api: Main API to receive database requests & audio files.
  • jibjib-data: A MongoDB instance holding information about detectable birds.
  • jibjib-query: A thin Python Flask API that handles communication with the TensorFlow Serving instance.
  • gopeana: A API client for Europeana, written in Go.
  • voice-grabber: A collection of scripts to construct the dataset required for model training

Scripts

In the top level of this repo, there are several helper scripts to create/change JSON and CSV files, as well as converter.py to convert audio files from mp3 to wav.

data_grabber/

This Go script uses gopeana to populate both a JSON and CSV file with information about the on Europeana published bird voices from the Tierstimmenarchiv (open dataset of the Museum für Naturkunde Berlin)

file_grabber/

This Go script uses the output of data_grabber/ to follow the links provided on Europeana and download the audio files.

wiki_grabber/

This Python script takes input from a CSV file and uses the Wikipedia API to extract summaries about birds, then saves it in a seperate CSV.

xeno_grabber/

This is a collection of scripts to:

  • clean the files directory (in our case, in order to bring down the total number of classes, birds with a German Wikipedia entry were used.)
  • nicely crawl Xeno Canto for audio files of birds
  • download the audio files from Xeno Canto