Fully-convolutional speakers identification solution
General architecture:
- A specific encoder.
- A selection of points of interest in a time windows.
- A concatenation of results for a place.
Directories:
- Data is reponsible to get data-set and transform the data.
- Encoder is the deep learning part of the project that takes samples and produce vectors.
- Identification uses vectors to identify the speaker.
- Localization is in charge to determine the place of the speaker.
Data: The data-set used is voxCeleb.
- getitem() return (anchor, positive, negative) for the encoder part
- getitem() return (label, sample) for the identification part
Encoder: We compare two approaches, one that use STFT, the other is fully temporal.
- "TransFourier.py" is main STFT pyton file.
- "Time.py" is main pyton file for time approach.
- Freq directory is for notebook related to STFT approach. We try 2 ways for vectors distances (euclidian distance vs cosine). We also try 3 differents sample size (1, 2 or 3 seconds).
- Time directory is for notebook related to the fully temporal approach. We try 3 differents sample size (1, 2 or 3 seconds).
Identification:
- "Conference.py" is a class that correspond to a conference.
- "Place.py" is a class that correspond to a specific place in the conference room.
- "Speaker.py" is a class that correspond to a speaker involved in the conference. We continue to compare time vs STFT approach. 20 and 40 is for the nb of speaker in the conference.
- "xxx_enc_1_xxx.ipynb" is for using encoder train with 1 second sample.
- "xxx_sample_3_xxx.ipynb" is for using 3 seconds sample for the identification.
- "xxx_speaker_xxx.ipynb" is for identify a speaker with one sample.
- "Place_after_3_samples_xxx.ipynb" is for identify a speaker on a place with a concatenation of 3 samples.
- "xxx_1_ref_in_pool.ipynb" is for identify speaker with one reference in the pool.
Localization:
- This part is under process.
- The goal is to determine the delay between the mics.
- We try different output vecor size (16, 32, 64 or 128).
- "LocateEncoder.py" contain the specific encoder class in charge of localization.
- That is fully time encoder responsibles for the concordance of points of interest between mics sample.