Welcome to the project page of my GSoC 2017 project!
The goal of my project was to implement Jingle File Transfer for Smack. Smack is a client library for the XMPP protocol for Java and Android.
XMPP stands for eXtensible Message and Presence Protocol. The focus lays on extensible. Many features are not part of the core specification, but instead are defined in so called XEPs (XMPP Extension Protocols). XEPs are numbered, so every XEP has a unique number and a name to identify it. The goal of my project was to implement XEP-0234 - Jingle File Transfer. In order to achieve this goal, I had to implement a whole number of other XEPs, since there are some transitive dependencies (see below).
I documented my progress in weekly blog posts. Those can be found in my blog.
This XEP describes how two entities can exchange files using the Jingle Protocol. Jingle itself is another XEP which I also had to implement (see below), since Jingle File Transfer is defined on top of it. The specification for XEP-0234 can be found here. The code I wrote for that XEP is available in this package.
The Jingle XEP defines how two entities can negotiate bytestreams between each other. This involves, which transport methods are used (examples for that down below), which parameters are used for the application of the bytestream (eg. which audio codec in a voice chat) and so on. Also security plays a role in the negotiation. The Jingle protocol consists of three main components:
- Description: Describes the application of the session (eg. file transfer - see XEP-0234)
- Transport: The transport method used. This might be InBandBytestreams, SOCKS5 proxies...
- Security: The encryption used. Describes how the bytestream is secured.
A session consists of one or more contents, which in turn consist of a description, a transport method and a security component. Those components can be composed and replaced in arbitrary ways.
Implementing the Jingle XEP was the biggest part of my project. I worked using the waterfall model to learn from previous errors I made and it took me 3 iterations, until I was happy with my code. Now I have a very modular codeset which can be easily extended and composed in any way. This blog post gives an overview of the structure. The specification is available here, while my implementation can be found here.
The easiest, most basic transport method is described in XEP-261. When data is sent using InBandBytestreams, the raw data is split into blocks of typically 4096 base64 encoded byte, which are sent one after another via the XMPP protocol itself. This method is occasionally very slow, but since it depends only on the XMPP protocol itself, it does not require additional server infrastructure. The XEP can be found here, my implementation resides in this package.
A more advanced transport method is described in Jingle SOCKS5Bytestream Transports (short Jingle S5B). Here SOCKS5 proxy servers are utilized to enabled clients behind firewalls or NATs to establish connections with each other. It is possible to use either external servers (eg. proxy components of the XMPP servers), or local servers for this purpose.
Working on the implementation for that XEP was a pain. For half a week I was stuck at finding one little bug. My frustrated blog post has more information about that. The XEP can be found here, my implementation here.
XEP-0261 and XEP-0260 are based on two older XEPs (XEP-0047 and XEP-0065), so I could reuse some already existing code for those two transport methods.
A Jingle File Transfer involves checksums to ensure the integrity of the sent file. Since computer science is a rapidly changing field, hash functions might quickly get obsolete (see SHA1 - well done you, Google :D), so it is desirable to be able to quickly replace hash functions. For that purpose XEP-0300 defines namespaces for hash functions, so that negotiators can use different algorithms as they wish. The specification can be found here, while my implementation lives here.
The most exciting part of my project was to create my own specification! With JET I specified a way to utilize OMEMO and OX (OpenPGP) end-to-end encryption to secure Jingle transports. This way entities can transfer bytestreams end-to-end encrypted. The specification allows for easy extensibility with other encryption mechanisms additionally to OMEMO or OX. My plan is to propose the specification to the XSF for approval, so that it might one day become an official XEP.
This blog post goes into more details on how the protocol works.
My draft specification can be found here, while my prototype implementation can be found here (only OMEMO encryption so far).
The following of my code has been merged into Smacks master branch. Some parts are still missing, which is partially due to experimental features, which need approval by the XSF. Unmerged code can be found in my repository.
Merged
- XEP-0166 - Jingle (first iteration)
- XEP-0260 - Jingle S5B (first iteration)
- XEP-0261 - Jingle IBB (first iteration)
- XEP-0300 - Hashes
Immediate Merge Pending
- XEP-0234 - Jingle File Transfer
Prototypical Code
- XEP-xxxx - Jingle Encrypted Transports (will be merged after XEP-approval by the XSF)
The whole point of XMPP is to avoid walled gardens, so there are many clients implementing many of the specified features. Ideally all clients are interoperable to one another (in reality it doesn't always go so well unfortunately). This is why I'm very happy to announce that my file transfer code has been successfully tested against Gajim, a well established desktop XMPP client, as well as against Conversations, a popular XMPP client for Android.
Since my works is mostly invisible library code, I wrote a small application to demonstrate some features. XMPP_SYNC is a command line application, which can be used to synchronize files between computers via (encrypted) Jingle File Transfer.
A blog post about XMPP_SYNC can be found here and the source code for it is available in this repository.
During my time implementing the XEPs, I often gave feedback to the XSFs standards mailing list.
- Encrypted Jingle File Transfers
- Ciphers XEP?
- XEP-0166 - Missing unsuported-security action
- XEP-0166 - Multiple Contents in transport-info message
- XEP-0234 - NonNegativeInteger
- Jingle XEPs - Use of session-info vs. description-info
- XEP-0234 - Wrong date format
- XEP-0260 - Redundant DstAddr
- XEP-0234 - Add option to request hash of offered file
- XEP-0234 - Denote used hash function in session-initiate
- XEP-0234 - Missing error condition for checksum mismatch
Whenever I could, I created Pull Request to address errors I found in the XEPs.
- XEP-0065 - Remove duplicate example
- Proto-XEP JET
- XEP-0234 - Fix xml in examples
- XEP-0234 - Add missing length attribute to XML schemas
- XEP-0234 - Fix wrong date format to conform with XEP-0082
- XEP-0234, XEP-0300 - Add hash-used element
Big thanks to all, who welcomed me :)