Dragonload Distributed Download Manager

Dragonload is a state of the art Distributed Download Manager written in Python to distribute large scale downloads on different machines to better utilize high-speed intranet connected to low-speed internet.

Developed by Team Madcaps

Download

Head over to out Source Repo and clone the project

git clone github.com/sayoojsamuel/dragonload
cd dragonload

Installation

python setup.py install

Developer Mode

python setup.py develop

Sample Config File

Download Engine Sequence Diagram

Web App User Engagements

Table of Contents

1. dragonload Distributed Downlaod Manager
   1. Tasks
   2. Tesing
   3. Protocol
      1. Splitfire Protocol
      2. ChainRain Protocol
      3. Dragonvault Protocol
   4. Ideas
      1. Simultaneous running of Splitfile and ChainRain in pipeline
      2. Add MultiThreaded Download for individual parts
      3. Authenticated Private Rooms
      4. curses based application
      5. Use click and pyinquirer, prompt-toolkit
      6. Use grpc for client, server, and all comms
   5. Necessary Reads
      1. Pathlib: Article to make os independent paths
      2. py_cui: alternative to ncurses
   6. ISSUES:
   7. ENHANCEMENTS:

Project Board

Tasks

• [-] Design a robust backend
• [-] Manage the protocol
• Create a config directory in POSIX and MAC machines. This about NS machines later. Config directory is necessary to store the db and the config file. DB is necessary for the downloading to happen. Also fix the directory where the downloading happens. Parital downloads in the temp directory or the main download directory.
• Test the platform
• Try to convert the commands to python only; platform independance

Tesing

• [-] Check for partial download
• Test the merge functionality

Protocol

Splitfire Protocol

This protocol is responsible for managing all the interested parties into one pool.

1. Objective

   • Create Pool for interested Parties; N party proto
   • Check the file size;
   • Split the fileSize by a multiple of N
   • Collect user IP’s
   • Generate the download script. Notify the CNC after each successful download; Realtime progress for the pool
   • Schedule Download??? Take care of the sleep
   • Manage Parties who leave in between the protocol
   • Split part of the remaining itemsList to the newcomming member; Dynamic Pool Model

ChainRain Protocol

This protocol is for efficiently sharing the files after partly download among the decorated users.

1. Objectives

   • Verify if parties are still active during the process
   • Check connection with the CNC
   • Share files parallely, efficiently and equally among the users. (check for corruptions)
   • Test if local hosting is proper
   • Share in a binary Tree manner - Will it work out?
   • Merge the parts as and when they arrive. Notify other about the status of the proces

Dragonvault Protocol

This is the main server application, which will handle the whole download manager. This can be implemented in two ways:

• A centralized main server
• A requirement based

Ideas

[ ] Simultaneous running of Splitfile and ChainRain in pipeline

[ ] Add MultiThreaded Download for individual parts

[ ] Authenticated Private Rooms

[X] curses based application

[ ] Use click and pyinquirer, prompt-toolkit

[ ] Use grpc for client, server, and all comms

1. Create robust framework, and build curses app, web app on top

Necessary Reads

Pathlib: Article to make os independent paths

py_cui: alternative to ncurses

ISSUES:

• Fix the port issue. Update the server with the users port address in addition to the IP address. Code everything in main file.

ENHANCEMENTS:

• Use python dataclasses
• Make a custom URI scheme (dragon://)
• Gossip framework for status sharing??