Documentation
[TODO NICO] Ich glaube, die Grafik müssen wir in zweierlei Hinsicht anpassen: (Könntest du mir eine editierbare Version der Grafik schicken?)
-
"Aware" ersetzen durch etwas generischeres wie "Context Provider"
-
Der neue zentrale K/V-Store muss rein. Im zuge dessen, würde es sinn machen das mit dem Block "Configuration File" zusammen zu legen?
The general architecture of vStore and its interfaces can be seem in the above figure. The main components of the framework are:
This component is responsible for storing new usage context and to provide interfaces for deleting and retrieving the current usage context. The external application should use this component when providing new context information to the framework, mainly when context conditions have changed.
The actual context data has to be provided from the application. This ensures a real platform-independence. An example implementation of a context aggregator for Android devices can be found in the code of the demo application vstore-android-filebox. It uses the AWARE framework and plugins, as well as the Google Places API.
Currently, the following context types are implemented:
- Location: The physical location represented by latitude and longitude, and a timestamp.
- Places: The large amount of different place types possible are represented by 3 groups: points of interest (POI), event (such as stadiums, city halls and night clubs) and Social (such as restaurants, cafes and bars).
- Activity: The user's physical activity. Denotes if the user is sitting still, walking, driving or if the activity is unknown.
- Network: The network context can currently consist of a WiFi context or a cellular network context.
In our framework, rules are used to make the storage decision and are evaluated by the matching engine, as described later. In vStore, rules can either be defined globally or created individually by the users. A rule specifies certain conditions that have to be fulfilled in order for that rule to be triggered. We specify our rules to consist of three parts:
-
Metadata properties: These denote for which MIME type and file size the rule should be taken into account during the matching process.
-
Context triggers: These properties determine under which contextual conditions a rule is triggered. Any of the aforementioned contextual information can be specified here. All of the configured context properties have to match the context that is given at the time of evaluating the rule.
-
Decision layers: The decision layers determine which storage nodes are chosen for file placement. A rule can consist of several of these layers where each layer constitutes a possible decision outcome. A layer can either be configured to match storage nodes that are of a certain type or that match certain constraints such as bandwidth or distance. A decision layer can also point to one specific storage node. In this case, the file will be stored on this particular node. The decision layers are evaluated only if the first two tiers of the rule (i.e. metadata properties and context triggers) are fulfilled.
In addition, a rule can have a replication factor k, which defines the required number of file replications.
The matching engine is the main part of the framework. It makes the decision where to store data, given data, contextual information, a set of available storage nodes and rules as input. The matching process is twofold: First, all rules are evaluated with respect to the configured trigger conditions and the type of data. We only consider rules where these two parts match the input. For instance, a rule that only applied to image files would not be evaluated further if the data the users wants to store is a document. Similarly, for the contextual information, let’s assume the rule specifies that it only applies to files of a certain size and is restricted to a specific location or within a range from a point of interest. If these properties do not match with the file that is to be stored, this particular rule is discarded.
For each of the remaining rules that satisfy the metadata and context triggers of the input data, a score s, 0 <= s <= 100 is computed to determine the most detailed rule, meaning the rule that incorporates the most contextual information.
To make it all work, you need the following things besides an instance of the framework:
-
Master Node: The master node serves configuration to the framework clients. This includes
- A list of global decision rules
- A list of storage nodes currently known
- A global key/value lookup service: Requests of file identifiers are served with a json array of identifiers of storage nodes on which the file is stored.
-
Storage nodes: These are the devices that are available to store the data. In a real-world deployment, a storage node could be hosted on a variety of devices, either close-by or distant to the user. To take into account this heterogeneity, we define different types of storage nodes. Besides cloud nodes, we consider cloudlets, gateway nodes and nodes in the core net. The latter could be represented by network layer middleboxes, who could have additional capabilities to store data as it passes through those devices. Gateway nodes on the other hand are devices users have a direct wireless connection to, such as WiFi access points or cellular base stations. In addition, we also consider private clouds as a type of storage nodes, for instance systems such as ownCloud, that are owned by end users themselves. Including this kind of nodes allows to define storage rules for the storing of private data, i.e., data that is not shared among different users of the framework.
-
Context providers: To provide the context-aware file placement feature, it is mandatory to provide the framework with context information. The acquisition of context data is highly platform-dependent, thus it cannot be included in the framework itself. We implemented an exemplary context aggregator in Java for Android devices, which is based on AWARE and Google APIs (see section ContextManager above for details).
For applications that want to use the framework, the following interfaces are provided:
-
store: To allow an application to store a file, this interface is provided. This method expects the URI to a file that the framework should store, and a flag which determines if the file should be stored in private or public mode. -
get: To allow an application to retrieve a file, this interface is provided. It expects the application to pass a file identifier, and the framework then retrieves this file.
For context providers, the following interfaces are provided by the framework:
-
provideContext: Should be used to provide new context to the framework. -
persistContext: Should be used, if the given context should be stored persistently on disk (if it should reloaded the next time the framework is started). -
clearCurrentContext: Enables the application or context provider to clear the context which is currently stored in the framework.
Communication with storage nodes happens via a defined REST-ful API. The following endpoints are available:
GET /uuid
When this route is requested, the node returns its identifier and type.
POST /file/data
Route for uploading a new file. The request should be a multipart request and the request body shouldcontain the following fields:
First part of the multipart request:
| filedata | The binary file content. |
Second part of the multipart request:
| descriptiveName | The file name that describes the file. |
| mimetype | The mime type of the file (e.g. image/jpeg). |
| extension | The file extension (e.g. jpeg). |
| filesize | The file size in bytes. |
| creationdate | The unix timestamp of the creation date of the file |
| isPrivate | True if the file should be stored for private access only. |
| phoneID | The identifier of the user device that uploads the file. |
| context | The usage context given at the time the file was saved in JSON format. |
GET /thumbnail/:uuid/:phoneID
This is the route for requesting the thumbnail of a file by providing the file identifier. The thumbnail will only be provided if the requested file is not marked as private. If it is marked private, it will only be provided if the identifier of the requesting phone id matches the file creator’s device id in the database.
GET /file/metadata/full/:uuid/:phoneID
Route for requesting the full metadata information of a file. Here, the same restrictions as above apply: The metadata is only provided if the file is either marked as public, or if the requesting phone id is the one that initially uploaded the file.
GET /file/metadata/light/:uuid/:phoneID
Route for requesting a shorter version of the metadata information of a file. This means, the response will not contain the usage context of the file. Here, the same restrictions as above apply regarding the phone id.
GET /file/data/:uuid/:phoneID
Route for downloading the full file by its identifier. Here, the same restrictions as above regarding the phone id also apply.
GET /file/mimetype/:uuid/:phoneID
Route for requesting the mime type of the file that matches the provided identifier.
POST /file/search
Route for retrieving a file list based on the context passed as parameter. In the request body parameter "context", the desired context entities should be listed and the node will reply with the identifiers of the matching files in a JSON array.
DELETE /file/:uuid/:phoneID
Route for deleting a file. Only the creator of the file can issue a delete request by providing his phone id.
-
BASE_DIRECTORY_DOES_NOT_EXIST- The working directory given as parameter to the framework does not exist.
-
COPYING_INTO_FRAMEWORK_FAILED- Copying the given file into the framework directory failed.
-
URI_SCHEME_NOT_SUPPORTED- The given URI has a scheme which is not supported.
-
FILE_ALREADY_EXISTS- The file to be stored already exists.
-
PARAMETERS_MUST_NOT_BE_NULL- The parameters of the method which returned this error must not be null or empty.
-
USAGE_CONTEXT_MUST_NOT_BE_NULL- The given usage context must not be null.
-
REQUEST_NOT_STARTED- A network request could not be started.
-
DB_LOCAL_ERROR- An error occurred while using the local sqlite database.
-
CONFIG_CONNECTION_FAILED- While downloading the configuration, a connection error occurred.
-
CONFIG_DOWNLOAD_FAILED- The download of the configuration file has failed.
-
CONFIG_PARSE_ERROR- Parsing of the configuration json string has failed.
-
FILE_NOT_FOUND- The given file cannot be found. Maybe the path is wrong.
-
MASTERNODE_WRONG_RESPONSE- The master node sent a wrong response.
The framework uses GreenRobot EventBus, to post different status information to the application. All events are listed and explained under this section.
Sticky events have to be removed from the EventBus manually by the subscriber function using EventBus.getDefault().removeStickyEvent(event).
For more detailed documentation regarding each event, please see the JavaDoc documentation of the framework
- Upload-related events
- Download-related events
- Configuration-related events
- Context-related events
- Error-related events
- File-related events
- Rule-related events
AllUploadsDoneEvent
This event gets published when all uploads are done.
UploadBeginEvent
Published when a new upload starts.
UploadDoneEvent (sticky)
This event gets published once a file is done uploading to one storage node.
UploadDoneCompletelyEvent (sticky)
This event gets published, once the file has been uploaded to all required storage nodes.
UploadFailedEvent (sticky)
This event gets published if an upload failed.
UploadFailedCompletelyEvent (sticky)
This event gets published when the download failed completely, regardless of the storage node.
UploadFailedPermanentlyEvent (sticky)
This event gets published if an upload failed permanently (e.g. because storage node replied with an error)
UploadStateEvent Gets published regularly during an upload to notify about the current state of a file being uploaded.
DownloadedFileReadyEvent (sticky)
Published when a requested download is ready.
DownloadFailedEvent (sticky)
Published if the download for a file failed.
DownloadProgressEvent (sticky)
This event gets published regularly during a file download.
DownloadStartEvent
Published when a download for a file starts.
MetadataDownloadFailedEvent (sticky)
Published when the download of metadata has failed.
MetadataEvent (sticky)
This event gets published once a request for metadata is done.
NewThumbnailEvent (sticky)
This event is published when a thumbnail has finished downloding
ThumbnailDownloadFailedEvent (sticky)
This event gets published by the framework if the download of a thumbnail failed.
ConfigDownloadFailedEvent
This event gets published when the download of the configuration file failed.
ConfigDownloadSucceededEvent
This event gets published when the download of the configuration file was successful.
NewContextEvent
This event gets published when the Context Aggregator received new context.
ErrorEvent
This event gets published if an error in the framework occurred. The different error codes are listed in the enum vstore.framework.error.ErrorCode.
FileDeletedEvent
This event will be published once a file has been deleted from the vStore framework. This event only gets published if the node the file was saved on has replied that he deleted the file.
FilesReadyEvent (sticky)
This event gets published when your file request you made to the framework is ready. If you want your app to get notified, you need to subscribe to this event.
NewFilesMatchingContextEvent (sticky)
This event gets published by the framework, if a node has responded with files matching the current context. Thus, each event of this type represents the reply of a search from one single node.
RulesReadyEvent (sticky)
This event gets published once the framework has fetched the rules from the database.