Collector

Responsible for writing Events to the Archive.

Overview

Collectors are write-only programs that generate Events for a particular set of Namespaces from some other device or service. Generally, Collectors should be grouped so that data that is requested/recorded together is within the same Collector.

There can be multiple instances of the same collector if you want different Namespaces to be collected at different intervals (ex: if downloading from Google Takeout, you may wish to synchronize all of your youtube videos once per month because it is a lot of data, but synchronize your chats every day because it isn't much data)

Collector Types

There are three main types of Collectors:

1. Historical. These run once for a set of data, and then they are done.
2. Samplers. These run at a fixed interval, and query the current values of some external device or service. This current value is recorded. Note that data for one of these Collectors is only recorded while the Collector is running (however, they are much simpler to write). These are great for things like location, heart rate, mood, current window title, etc.
3. Synchronizers. These run at a fixed interval, and query the external device or service for its full state, then record any differences from the last run. These are more complicated to write because they must maintain some state about what Events have already been recorded (typically a date of last synchronization, or a set of unique ids), but they can be very useful. These are great for things like email, facebook messages, flickr, other web services, etc.

Designing a new Collector

1. Decide what types of data you want to record (ie, which Namespaces you would like to include) See a list of already defined Namespaces or, if none of those are what you want, how to create your own Namespace.
2. Pick the appropriate type of collector from above depending on how you would like to collect the Namespaces that you decided on.
3. Use a sample collector (coming soon) for that type of collector and fill in the details!

General hints for new Collectors

• Don't worry if you generate duplicate Events--they will NOT be duplicated in the Archive! In fact, sometimes the easiest way to create a new Collector is to simply insert everything from the original source all the time.
• Group together Namespaces that can all be collected at the same time. Don't try to make a separate Collector for each Namespace