Moodle-Books can be downloaded

Added

• Now the book mod can be downloaded, we generate a simple table of contents for every book

Changes

• From now on, I try to follow semantic versioning.

Fix wiki URLs

Just fix the URL to https://github.com/C0D3D3V/Moodle-DL/wiki/Example-Config.json-and-Options in the Readme

Add max_file_size option

New Feature:

• You can now use the option max_file_size to exclude internal moodle files that are bigger than max_file_size in bytes. This does not affect external files (URLs).

See also: https://github.com/C0D3D3V/Moodle-DL/wiki/Example-Config.json-and-Options

2.3.1.5

New Features

• URL shortcuts on MacOS are now webloc files (#184)
• It is now possible to configure if URL Shortcuts should be created using config options:

    "write_url_link": false, # Windows Shortcut File
    "write_webloc_link": false, # MacOS Shortcut File
    "write_desktop_link": false, # Unix Shortcut File
    "write_link": true, # Write a shortcut file that matches your OS 

• You can now set the download location and location of all misc files in your config.json (#157):

    "download_path": "C:\PATH\TO\DOWNLOAD\LOCATION",
    "misc_files_path": "C:\PATH\TO\FILES_LIKE_moodle_state.db_and_Cookies.txt",

• If you have activated download of assignment submissions, now as a teacher, also the assignments of all students are downloaded (#178)

Do not overwrite external URLs by its redirected URL

Changes

• Do not overwrite external URLs by its redirected URL (This fixes a bug with the new SharePoint Extractor)
• If you are using external downloaders, check that your config matches also the not redirected domains.

Major Change in diff check

Warning: If you do not want to re-download already downloaded HTML content, please read carefully.

In the new version, a change has been made to the difference detection engine to fix a bug. This has the side effect of re-downloading almost all HTML content (like quiz attempts, lesson attempts, pages). To prevent this (or to discard the duplicate files), you have two options:

1. You can ignore the fact that the files are being re-downloaded (the download should be very fast, since only plain text files are affected).
   So you update moodle-dl, then run moodle-dl normally as you normally would, and to remove the newly created duplicates you can run moodle-dl --delete-old-files.

2. You can prevent downloading all new (probably duplicated) files: Run moodle-dl normally before updating, then update with pip install -U moodle-dl, then run moodle-dl --without-downloading-files.
   This will update the moodle-dl database to the latest state of your moodle, but will not download any files. But be warned a side effect is that missing files that are only downloaded in the new version due to the bug fix will not be downloaded.

Bug fixed

• If you make multiple quiz attempts they get now all downloaded correctly (reported by @juffis)

API calls are now asynchronous

Changes

• Overall the code was refactored and cleaned up, so please report bugs!
• The logging to the terminal changed a lot and is now more pythonic
• new dependencies:
  aiodns, aiofiles, aiohttp, colorlog, readchar (again),

New Features

• API calls are now asynchronous, so moodle-dl is now a lot faster
• Also, downloads are no longer performed in threads but asynchronously.

Bug fixes

• embedded files like images now get a unique name based on a hash #181
• SSL-Context is now created once for all calls, and is no longer dependent that a ssl chain is found on the system (but if no chain is found the ssl calls will fail, then you can use --skip-cert-verify) #173
• Cookie file is now longer invalid if a cookie is discarded because it was a session cookie #168
• Telegram messages are now always shorted no matter what #167
• Filenames are now all truncated to 200 characters + the extension, even files downloaded using yt-dlp should be truncated now #59

Non-indexed modules are downloaded

New features:

• There are courses that don't index all modules on their main course page (i.e. don't have them listed in sections) but only link them on pages that are themselves linked in descriptions or other pages, and generally create their page structure exclusively via moodle pages. These modules are now also downloaded (into special folders for modules that are not linked on the main page).

Do you want to use a whitelist or blacklist?

New Features

• If a connection error with the moodle server occurs while requesting the API, an API call is now repeated up to 5 times before giving up.
• You can now select in the configurator whether a blacklist or a whitelist is to be created.

Bug Fixes

• Windows users can now control the selection lists with the arrow keys again
• The configurator is faster again because now the sections are downloaded only when they are needed.

Individual sections can now be excluded

New Features

• Individual sections can now be excluded (by @jack96489);
  For this purpose, it is also possible to use the configurator and exclude certain sections in each individual course.
• Old versions / copies of files can now be permanently deleted with the command moodle-dl --delete-old-files (by @jack96489)
• Descriptions of folders are now also downloaded

Changes

• Files are now considered "different" even if only the file size has changed (by @jack96489)
• In Telegram notifications, course names are now bold (by @jack96489)
• Pages (/mod/page) are now downloaded as HTML again (by @jack96489) To prevent re-downloading, use the options.
• In windows relative paths are used again by default, if there is a max path limit in your windows, you can still work around it with the --max-path-length-workaround option (by @jack96489)
• The content of Descriptions and Pages will only be attached to the notification if the file was created or modified.
• Forum posts are now downloaded even if downloading descriptions is disabled.
• The proposal for a cron job was adjusted at the end of the execution of moodle-dl --init

Bug Fixes

• If the changelog file is not present on a Moodle, the SSO token process can now still be performed.
• External downloaders now always get the unmodified URL, means also without token from Moodle (even if the token would be necessary).