Dataset hosting and verification

[mikeheddes]

Currently the datasets are hosted on the UCI machine learning repository and Google Drive. However, the Google Drive links are not very robust as they modify their download flow sometimes. We are thus looking for a host that provides great spreed, reliability and is easy to download from programmatically. Something like Google Cloud Storage or AWS S3, however these are not free.

At the same time for security and usability reasons we should verify if the download succeeded by checking the file's checksum.

[mikeheddes]

I recently got this exception:

Too many users have viewed or downloaded this file recently. Please try accessing the file again later. If the file you are trying to access is particularly large or is shared with many people, it may take up to 24 hours to be able to view or download the file. If you still can't access a file after 24 hours, contact your domain administrator.

Although I was downloading the datasets pretty often so the exception is understandable, if this library is to grow among the community I expect this error to happen more often which is not an acceptable user experience.