Skip to content

Project 3 Report

Jump to bottom
aksrajvanshi edited this page May 1, 2020 · 10 revisions

Problem Statement

Our problem statement focused on the previous scenario of MFT, which only had support for 3 protocols namely SCP, Local, and S3. Moreover, the system was not a standalone system, as it was bound to an existing service that required MFT rather than a user directly talking to it. Furthermore, it was not deployed in a distributed environment to enable better scalability and fault tolerance.

Differences from Initial Problem Statement

Initially, when we started, we aimed at contributing Google Cloud Storage transport provider support to the project. Furthermore, we identified several bugs and reported them to the Airavata community in order to make the project more robust and reliable.

<... API gateway work for testing the working of MFT and which was used by fellow classmates to get a better insight into the project. Also mention the standalone factor here....>

< Add the initial README issues? because of no initial documentation, we had to put in a lot of time to getting started with the existing system and the overall investigation too.>

We did look into the design of the project and how it could be extended to be deployed in the cloud environment. However, since multiple teams were focusing on the same problem, we decided to focus on adding a new protocol and finding more issues in the quickly evolving environment of MFT. Hence, we provided support for Dropbox transport and identified a qualitative issue with file transfer in the mft-core module. We also worked with the Airavata developer community to create a design document to give developers a good understanding of how to add a new transport provider support to MFT.

Problem Statement Development

We regularly communicated with the Airavata developer community regarding doubts about our contributions and also reporting the issues we found with the existing codebase. The following are the links to the communication between the pika-pika team and the developer community.

  1. https://lists.apache.org/thread.html/rab4f83ef4edbe105bcaf6db334146529ddc7850445b93a4d3093a445%40%3Cdev.airavata.apache.org%3E
  2. https://lists.apache.org/thread.html/r74e8ecb2a7f9e7b57404858da790dd81df8e2037c6e959e0d6d03cdf%40%3Cdev.airavata.apache.org%3E

  3. https://lists.apache.org/thread.html/rc0fadd042b9d604e1130feb29ee6c8afd591b20e2978e612717f3e0f%40%3Cdev.airavata.apache.org%3E

  4. https://lists.apache.org/thread.html/r38ff4fb7fed5fff9b4373d1ec7b80fe94df52befb819a8505041f25a%40%3Cdev.airavata.apache.org%3E
  5. https://lists.apache.org/thread.html/r907c04e56ade0db2fdf0bb532676555bac2394071db49e1604c0a720%40%3Cdev.airavata.apache.org%3E
  6. https://lists.apache.org/thread.html/rd7c21f388be5404b77fb56398dacca47afca3de1b37db165b5d8a6da%40%3Cdev.airavata.apache.org%3E
  7. https://lists.apache.org/thread.html/r6564088c48d20b556be95b930dd1cab946dc8ba073cd445a8f2c9a70%40%3Cdev.airavata.apache.org%3E
  8. https://lists.apache.org/thread.html/ra5c9c440a14cc9b6134f3261447e52390d0d7c9dfe93338141e708f7%40%3Cdev.airavata.apache.org%3E
Clone this wiki locally