This repository takes care of the generation of word lists using the open source version of KNIME. Text files are read in and processed for this purpose. The resulted wordlist is the json file in the root of the repository.
corpus
contains all textual pre-processed input filesmanual
contains manually defined words that are easy to remember, e.g. colorsstopwords
contains words that are removed from the corpusCovappWordList.knwf
is the data processing workflow of KNIME
The corpus contains the german most frequent words according to researchers from University of Leipzig. Additionally the corpus extended by the biggest geographic terms such as the biggest cities in Germany and neighboring countries. These information was crawled from Wikipedia and News Articles, that were crawled with Newsplease.
- Thomas Hepp
MIT 2.0