Locker Axioms

Here for clarification of focus around the Locker Project

Connectors

Connectors retrieve data from external sources and keep a local reference or copy of that data
Connectors can be written in any language
Connectors run in their own child process, regardless of language
Connectors can provide data to any other part of the locker
Connectors only speak HTTP with other parts of the Locker when providing their data
Connectors can connect to any API given the proper credentials
Connectors can also collect non-API data
Connectors keep the data retrieved in a basic indexed JSON file on disk
Connectors know how to keep track of synchronization, paging, and other low-level aspects of data retrieval from a data source. How connectors do this is up to the implementor of the connector
Connectors can, but aren't required to download local copies of remote data, specifically in regards to binary data. Some local data may only contain references to remote resources or other large data objects
Connectors can have several implementations for a particular service type provided
Connectors define what service type or types they provide
Connectors can be queried for their data, but only given a unique ID range, or a timestamp range, including wildcards
Connectors emit events based on their service type, whenever retrieved data is created, updated, or deleted

Collections retrieve data from one or more connectors
Collections can be written in any language
Collections run in their own child process, regardless of language
Collections can provide data to any other part of the locker
Collections only speak HTTP with other parts of the locker when providing their data
Collections can make queries from connectors or other collections to populate their own datastore
Collections only retrieve data from internal to the locker, never externally
Collections are responsible for handling the de-duplication of duplicate data from multiple sources
Collections can contain additional behavior that creates, updates, and/or deletes additional data
Collections can have several implementations for a particular service type provided
Collections define what service type or types they provide
Collections can be queried for their data, but only given a unique ID range, or a timestamp range, including wildcards
Collections emit events based on their service type, whenever retrieved data is created, updated, or deleted

Applications can do anything they like, and have access to both connector and collection data, based on permission levels
Applications can be written in any language
Applications will run in their own processes, outside of the locker process
Applications will only be able to retrieve data from collections and connectors via HTTP
Applications can run on servers external to the locker itself
Applications will use OAuth to authenticate against a locker, and will have permission schemes available to it
Applications can request certain service types as required in order to be installed against a locker
Applications can cache retrieved data, and store it in any method desired as needed for its functionality
Applications can have their OAuth permissions revoked, at which time they will no longer be able to access the revoker's locker data
Applications who do have their OAuth permissions revoked should fail gracefully

Service Types are how locker components identify the data they provide
Service Types have a primary and a secondary type. A primary type example is "message" and a secondary type example is "facebook" or "imap"
Service Types used by a Facebook connector may be "message/facebook" or "status/facebook" or "contact/facebook"
Service Types used by a message collection may be just "message"
Service Types are used as event names that are emitted when a provider of that service type creates, updates, or deletes its data
Service Types provide data maps for all primary service types, that all data providers of that type must adhere to when emitting data

Data Maps provide the data schema common across service types, for a particular primary service type
Data Maps are provided as JSON-formatted strings, and can be retrieved via an HTTP request to the locker
Data Maps have a version number
Data Maps denote what fields are named, the maximum length of the values, as well as the required/optional status of each field
Data Maps must be adhered to by any data provider emitting events for a given primary service type
Data Maps don't exist for secondary service types

Events provide notification for any data change across the locker
Events can be emitted by any provider of that particular service type
Events are named after the service type they represent
Events have payload bodies that are JSON-formatted strings
Event bodies always have the following header fields: id, timestamp, action, source, data
Event body header "id" is the unique identifier for a particular data provider
Event body header "timestamp" is the timestamp in UTC for when the event was emitted
Event body header "action" is one of the following: create, update, delete
Event body header "source" is the URI of the source connector or collection that emitted this event
Event body header "data" is also JSON-formatted
Event body header "data" contains the data map representation of the event, if the event was a primary service type. Otherwise contains the raw data from the service provider