Contents
"dl" is a simple file sharing service for quick/one-off file transfers. Upload a file to get a link you can share. Or create a sharing link to receive files from others. The uploaded files are automatically removed when left unused, requiring zero additional maintenance.
"dl" is built for your users: easy to use with any browser, integrates smoothly with Thunderbird for large attachments, works on Android, Windows and OSX or straight from the command line for maximum convenience.
- PHP 5.5 or higher.
- PHP SQLite module (or another PDO database module).
- PHP mbstring module.
- PHP OpenSSL module.
- PHP Zip module.
- Web server access for installation and setup.
The installation is divided into three steps:
- Installation of the files
- Database creation/setup or database upgrade
- Initial user creation
Due to the number of possible configurations, the installation must be carried on manually.
Copy the
htdocsdirectory contained in the archive to a directory of choice under your web server.Copy
include/config.php.distto eitherinclude/config.phpor/etc/dl.phpand customize as needed. This usually means changing the URL and E-Mail of the service.Ensure the
includeandstyle/*/includedirectories are not directly accessible externally. Typehttp://dl.example.com/include/config.phpand verify that you correctly get an "Access Denied" error.If you use Apache, the provided
.htaccessfiles should be already sufficient; consult your web server documentation otherwise.Create a spool directory that will be used by the service to store the files, user and ticket information. This directory should be outside of the document root of your web server. Fix the permissions so that PHP has read/write access to it.
In the provided
include/config.phpthis is configured as/var/spool/dl. If you web server runs as "nobody:nogroup", issue:mkdir -p -m770 /var/spool/dl chgrp nogroup /var/spool/dl
to create correctly this directory.
Create a database (as described in Database setup).
Create at least one user (as described in Internal authentication).
DL needs a database to store the ticket and user information. By default, DL will use an embedded "sqlite" database stored within the spool directory, but some manual setup is still required.
To create the database, you need the sqlite3 command.
On Ubuntu/Debian, sqlite3 can by installed by executing:
sudo apt-get install sqlite3
You should then execute the provided SQL batch for SQLite with the same user
as your web server (by using su or sudo if necessary):
cd /var/spool/dl/ sqlite3 data.sdb < /your-installation-directory/include/scripts/db/sqlite.sql chmod 660 data.sdb
If you want to change the database path, or use a real database server, you
need to properly configure the $dsn parameters in include/config.php
according to your setup. The DSN string changes according to the PDO module
that you want to use. Please see one of:
for the most popular configuration choices. When a username/password is
required, using the appropriate variables $dbUser/$dbPassword is
preferred instead of embedding the values in the DSN string.
The directory include/scripts/db/ provides SQL initialization scripts for
SQLite, MySQL and PostgreSQL.
The following parameters are required to be set in your php.ini:
date.timezone: must be set to your system preference.session.auto_start: must be "Off".
The maximal upload limit is determined by several PHP configuration parameters.
Check your php.ini for:
upload_max_filesize: change as needed.post_max_size: must be at least 1M larger than upload_max_filesize.session.gc_maxlifetime: must be long enough to allow large uploads to finish.
All the above settings can be configured per-directory and affect only DL. In addition, the following parameters need to be set for the entire PHP server:
file_uploads: must be "On".upload_tmp_dir: ensure enough space is available for temporary files.max_file_uploads: change as needed.
The upload limit as shown in the submission form is determined automatically
from the upload_max_filesize parameter.
Any upload beyond post_max_size will be completely ignored: users will get
a blank page instead of an error message. You should raise post_max_size
above upload_max_filesize to get an acceptable "error window" in old
browsers or browsers with JavaScript disabled.
session.gc_maxlifetime (which is in seconds) needs to be long enough for
your users to complete a large upload. Uploading 500MB on a slow ADSL
connection can take as much as 12 hours, so set it to at least 43200.
If PHP was built as an Apache module you can set per-directory parameters
through .htaccess (see
http://www.php.net/manual/en/configuration.changes.php) or directly inside your
Apache's configuration (see Apache/mod_php for an example). The provided
htdocs/.htaccess contains a reasonable pre-set, but requires
AllowOverride All to be set for the directory in your main configuration.
DL can use both an internal and an external user database, by trusting the authentication credentials provided by your web server.
dl supports both "normal" users and "administrators". A normal user can only see and manage tickets created by himself. Administrators can see/manage all the tickets.
Once dl has been installed and PHP is set-up correctly, you have to create at
least one user to be able to log in. User management is handled through the
command line by using the bundled useradmin.php utility.
On the server, execute the following commands with the same user as your web
server (by using su or sudo if necessary):
cd /your-installation-directory/include/scripts php useradmin.php add "admin" "true" "change me"
where:
adminis the user nametrue(orfalse) sets the administrator statuschange meis the password
Repeat as many times as necessary. You should now be able to use the web service. Other users/administrators can be added through the web interface.
External authentication should be the preferred form of authentication for corporate use since it supports whatever authentication scheme your web server already supports (for example, LDAP, ActiveDirectory, etc).
To enable external authentication you have to protect the two files:
admin.phprest.php
using a "Basic" authentication scheme. You should then set $authRealm to
the same authentication realm used in your web server. The other files must
not be protected.
DL will implicitly trust the credentials provided by the web server. All users
are logged in as "normal" by default. The only setup required is adding the
administrators with useradmin.php without using any password.
Logout with HTTP authentication is not guaranteed to work: users should simply close their browser to clear their session (closing a tab or window is not enough in many browsers). Currently, logout works as expected on:
- Firefox
- Safari
- Google Chrome/Chromium
Logout does not work on:
- Internet Explorer 7/8.
- Opera 9/10.
Again, only the Basic authentication is supported, which transmits the password in clear-text unless you use SSL.
When using external authentication, the HTTP header USER_EMAIL can
additionally provide the user's default email address. Such header is provided
automatically, for example, when using "LemonLDAP::NG".
Large file support (for uploads larger than 2GB) requires a combination of PHP version, web server and browser support.
Apache 2.2 and above support large request bodies but needs to be built for
64bit (see LimitRequestBody). Same for Lighttpd 1.4 (>2gb but only for
64bit builds, see server.max-request-size).
Due to several bugs in PHP prior to 5.6, upload_max_filesize and
post_max_size are limited to a 31/32bit integer, which limits the upload
size to 2/4GB even on 64bit systems. The maximal uploadable sizes are shown
below:
| PHP Version | Upload limit |
|---|---|
| <5.4 | 2gb: post_max_size = 2147483647 |
| 5.4-5.5 | 4gb: post_max_size = 4294967295 |
| >=5.6 | no limit |
Note that PHP versions before 5.5 are no longer supported, and older versions are shown here for reference purposes only.
Finally, not all browsers support large file uploads:
| Browser | Upload limit |
|---|---|
| IE <= 8 | 2gb |
| IE >= 9 | no limit |
| Firefox <= 16 | 2gb |
| Firefox >= 17 | no limit |
| Chrome | no limit |
| Opera >= 10 | no limit |
Sources:
- http://www.motobit.com/help/scptutl/pa98.htm
- https://bugzilla.mozilla.org/show_bug.cgi?id=215450
- http://blogs.msdn.com/b/ieinternals/archive/2011/03/10/wininet-internet-explorer-file-download-and-upload-maximum-size-limits.aspx
With internal authentication:
<Directory /your-installation-directory> AcceptPathInfo On AllowOverride Limit Options -Indexes DirectoryIndex index.php index.html </Directory>
With external authentication:
<Directory /your-installation-directory>
# Normal DL configuration
AcceptPathInfo On
AllowOverride Limit
Options -Indexes
DirectoryIndex index.php index.html
# Require a Basic authentication scheme for admin/rest.php
<FilesMatch "^(admin|rest)\.php$">
# The scheme must be Basic
AuthType Basic
AuthName "Restricted Area"
Require valid-user
# You'll need to provide a valid source for passwords using either the
# following or some other authentication source (such as LDAP)
AuthBasicProvider file
AuthUserFile /path/to/passwd/file
</FilesMatch>
</Directory>
With LDAP or ActiveDirectory authentication:
<Directory /your-installation-directory>
# Normal DL configuration
AcceptPathInfo On
AllowOverride Limit
Options -Indexes
DirectoryIndex index.php index.html
# Require a Basic authentication scheme for admin/rest.php
<FilesMatch "^(admin|rest)\.php$">
# The scheme must be Basic
AuthType Basic
AuthName "Restricted Area"
Require valid-user
# Use the LDAP provider (just an example query)
AuthBasicProvider ldap
AuthzLDAPAuthoritative off
AuthLDAPURL ldap://XXXXXX:XXXX/ou=XXXX,dc=XXXX,dc=XXX?sAMAccountName?sub?(objectClass=*)
AuthLDAPBindDN "cn=XXXX,ou=XXXXX,dc=XXX,dc=XXX"
AuthLDAPBindPassword "XXXXX"
</FilesMatch>
</Directory>
FastCGI support in Apache up to 2.2.x is severely lacking with all the
available modules: mod_fcgi, mod_fcgid (now merged officially into
Apache's mod_fcgi) and mod_fastcgi.
mod_fcgiandmod_fcgidbuffer the entire request in memory before handing-off the request to PHP, meaning that the maximal upload limit is bound to your available memory at the time of the request, independently of how PHP is setup. This is a known, old bug that's still present in bothmod_fcgi2.2.14 andmod_fcgid2.3.4. There is no known work-around: either usemod_phpor use a different server.mod_fastcgihas been proved to be slow (and sometimes unstable) in most configurations. It is not advisable to use PHP withmod_fastcgi.
For the REST service to work, independently of the authentication method,
mod_rewrite needs to be enabled and configured as follows:
<Directory /your-installation-directory>
# Normal DL configuration
AcceptPathInfo On
AllowOverride Limit
Options -Indexes
DirectoryIndex index.php index.html
<FilesMatch "^(admin|rest)\.php$">
# Forward the credentials for the PHP process
RewriteEngine on
RewriteCond %{HTTP:Authorization} ^(.*)
RewriteRule ^(.*) - [E=HTTP_AUTHORIZATION:%1]
</FilesMatch>
</Directory>
This is required to correctly pass the Authorization header to the PHP
process.
If you want to enable HTTP/External authentication, just add the usual authorization configuration as well:
<Directory /your-installation-directory>
# Normal DL configuration
AcceptPathInfo On
AllowOverride Limit
Options -Indexes
DirectoryIndex index.php index.html
<FilesMatch "^(admin|rest)\.php$">
# Forward the credentials for the PHP process
RewriteEngine on
RewriteCond %{HTTP:Authorization} ^(.*)
RewriteRule ^(.*) - [E=HTTP_AUTHORIZATION:%1]
# Require a Basic authentication scheme for admin/rest.php
AuthType Basic
AuthName "Restricted Area"
...
Require valid-user
</FilesMatch>
</Directory>
PHP/FastCGI works fine with Lighttpd 1.4.x without any special setup. The following configuration is required to protect the include directories:
$HTTP["url"] =~ "^/dl(?:/|/.*/)include/" {
url.access-deny = ( "" )
}
You can also enable external authentication with the following:
$HTTP["url"] =~ "^/dl/(?:admin|rest)\.php$" {
auth.require += ( "" => (
"method" => "basic",
"realm" => "Restricted Area",
"require" => "valid-user"
) )
}
Nginx in combination with PHP/FastCGI works fine but needs special configuration to
setup PATH_INFO correctly. Here is an example configuration with DL
installed as a subdirectory in the document root:
location ^~ /dl {
# Protect the include directories
location ~ ^/dl(?:/|/.*/)include {
deny all;
}
index index.php index.html;
try_files $uri $uri/ =404;
# Enable PHP
location ~ \.php(?:$|/) {
include fastcgi_params;
# Set maximum body size (should be the same as PHP's post_max_size)
client_max_body_size 512M;
# Setup PATH_INFO (http://trac.nginx.org/nginx/ticket/321)
fastcgi_split_path_info ^(.+\.php)(/.+)$;
set $path_info $fastcgi_path_info;
fastcgi_param PATH_INFO $path_info;
try_files $fastcgi_script_name =404;
fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
fastcgi_index index.php;
fastcgi_pass unix:/var/run/php5-fpm.sock;
}
}
Ticket expiration can be either performed internally to DL (the default), or by
using the external include/scripts/expire.php utility with a cron job. This
preference can be set by controlling the $gcInternal parameter.
The internal method requires no setup, but the external method has the added advantage of not interrupting the web interface during the expiration process, and also ensures that the spool is emptied when DL itself is not used actively.
Expiration is usually performed aggressively at every page request. You can
control this behavior (thus reducing the DB pressure) by tuning the
$gcProbability and $gcLimit parameters.
If you notice too much load on your DB, start by lowering $gcProbability to
0.5 and set $gcLimit to roughly the number of active tickets currently
present in your DB.
Continue to lower $gcProbability even further until the load becomes
acceptable. When the load is acceptable, but queries take too long, reduce
$gcLimit.
Simply call include/scripts/expire.php within a cron job, which should be
executed with the same user as the web server. Executing the script once a
day is sufficient for low traffic websites, but can be executed as often as
needed. $gcLimit still controls how many tickets are expired for each run
to limit the execution time.
Backup your current
config.phpfile and spool directory.Overwrite the DL installation directory with the new copy.
Either copy over the old
config.phpfile or customize the new version.Run
dbupgrade.phpas your web server user:cd /your-installation-directory/include/scripts php dbupgrade.php
Test your new setup.
DL has been translated in several languages and will attempt to detect the correct locale of the browser and use it automatically when available. If no matching translation can be found, a default will be used, which is configured to be English in the main distribution. The user can however switch the language anytime.
Adding a new translation is easy for anyone familiar with the gettext tools: you don't need programming knowledge. If you want to contribute a new language, or suggest a better translation, we recommend to subscribe to the mailing list and ask for guidance. We really appreciate your help.
Contributing a new translation is easy enough:
Edit
htdocs/include/lang.phpand add your new language name and alias to$langData, as done for the other languages.Execute:
cd htdocs/include/scripts/ ./langgen.php
to freshen the strings to be translated.
Translate the generated
messages.poin the directoryhtdocs/include/locale/<locale_NAME>/LC_MESSAGES/using a text editor, or by using PoEdit, or any other "po" editing tool.Optionally translate the user guide, which is located in
htdocs/include/static/guide/.Copy the english directory tree into a new tree with the new locale name and translate
index.rst.index.htmlis regenerated automatically.To test/update the translations run
langupd.php:cd htdocs/include/scripts/ ./langupd.php
DL should be usable by users without any training. The web interface must be self-explanatory. If you find the usage to be difficult or that the interface could be improved, it's a bug. Please let us know.
A command-line client to the REST interface is included in the distribution in
client/dl-cli.py. This client requires a simple text configuration file, by
default stored in ~/.dl.rc, containing the following values:
- url: REST URL of the service
- user: your user name
- pass (optional): your password (if not specified, you will be prompted for it by the client)
- passcmd (optional): invoke the supplied command to obtain the password
- verify (optional): "true" or "false": enable/disable SSL verification (might be required for testing, but defaults to true)
- email (optional): default email address for grant notifications
- fingerprint: Validate the server against the specified certificate or fingerprint (See Public key pinning).
An example:
url=https://dl.example.com/rest.php user=test pass=test
Simply run the command with no arguments to see usage information. At least Python 2.7 is required (with Python 3+ being recommended), with the "PycURL" and "ConfigObj" modules installed. Under Debian/Ubuntu systems you can install the required dependencies by doing the following:
sudo apt-get install python3-pycurl python3-configobj
A graphical client is also included in the distribution, which allows to create
tickets easily from the system's taskbar. The client can be run by executing
client/dl-wx/dl-wx.py or by downloading an executable client.
Upon first execution the user will be prompted for the basic configuration. After that all DL functions can be operated through the taskbar icon:
- Left-clicking on the taskbar will create a new ticket using the default settings.
- Right-clicking allows to select different actions.
- On OSX, you can drop files directly on the dock.
At least Python 2.7 is required, with the "ConfigObj", "PycURL" and "wxPython" modules installed. Under Debian/Ubuntu systems you can install the required dependencies by doing the following:
sudo apt-get install python-pycurl python-configobj python-wxgtk2.8
dl-cli and dl-wx share the same configuration file, so both can be used
interchangeably.
A dl-wx pre-built binary is also available online on the dl-wx page, which
includes installation instructions and a simple tutorial.
The fingerprint option in the ~/.dl.rc configuration file implements
public key pinning, and supports any value directly accepted by cURL.
It can be a path to a DER/PEM certificate file, or a SHA256 hash of the public
key of the host. The hash can be either a hex-encoded string (with optional
colon separators), or a string starting with sha256// and followed by a
base64-encoded value of the hash.
The quickest way to obtain the fingerprint is to use curl directly:
curl -vkI --pinnedpubkey 'sha256//' https://example.com/
Look for the "public key hash" in the generated output.
The bundled extension "Thunderbird-Filelink-DL" integrates with the new Thunderbird's Filelink functionality, by using the REST service provided by DL 0.10 and onward. The extension allows to convert large attachments to links automatically, directly within the Composer window.
The extension also allows the user to generate/insert a new upload grant in the current message from the composer window. Both a menu command (under "Tools" .. "Insert upload grant") and a toolbar item (that you manually need to drag in the composer toolbar) are provided.
To install the extension, go to Thunderbird's "Tools" .. "Addons" menu, and
click on the "Settings" icon just next to the search bar. Select "Install
Add-on from file..." and choose the file client/thunderbird-filelink-dl.xpi
as provided in the distribution.
See full installation and usage instructions on the extension web page.
- <dl-ticket-service@thregr.org>:
- Go-to list for general discussions, troubleshooting and suggestions. You can subscribe to dl-ticket-service by either sending an empty email to <dl-ticket-service+subscribe@thregr.org> or by using GMane (group "gmane.comp.web.dl-ticket-service.general"). The archives are accessible via web through https://www.mail-archive.com/dl-ticket-service@thregr.org/ or via news directly.
- <dl-announces@thregr.org>:
- DL release (and release candidate) announcements (read-only list). Very low traffic. To subscribe, send an email to <dl-announces+subscribe@thregr.org>.
- <dl-translators@thregr.org>:
- Mailing list reserved for translators coordination.
You can contact the main author directly at <wavexx@thregr.org>, though using the general list is encouraged.
You are encouraged to change DL as you see fit under the terms of the GNU GPL 2 license, or (at your option) any later version. DL's GIT repository is publicly accessible at:
https://github.com/DownloadTicketService/dl
or at git://src.thregr.org/dl
Development releases directly downloaded from git do not include pre-processed files. To build the localization data gettext and docutils need to be installed. You'll then need to execute:
cd htdocs/include/scripts/ ./langupd.php
To build the Thunderbird add-on, the Thunderbird SDK needs to be installed as
well. You might need to change the paths inside
client/thunderbird-filelink-dl/config_build.sh (which is tuned for Debian's
thunderbird-dev package) and execute:
cd client/thunderbird-filelink-dl/ ./build.sh
Database schema changes are not gracefully handled while following a development release. Do not run development releases on a production environment.
"dl" can be found at https://www.thregr.org/~wavexx/software/dl/
dl's GIT repository is publicly accessible at:
https://github.com/DownloadTicketService/dl
or at git://src.thregr.org/dl