Add NATS support

[udf2457]

I thought it might be nice to add NATS support (storing objects on NATS JetStream Object Store)

PR in #59 (but pending resolution of issues #51 and #53)

[wojas]

We have been evaluating NATS for an internal project and so far we really like it. Having a NATS backend for the simpleblob interface would be a great addition. We do feel however that any additional backends should live in their own repository and have their own release cycles.

The fact that the S3 backend has been included in this repository instead of living in its own repository is because of historical reasons. Now this backend generates most of the code churn and pulls in all kinds of extra dependencies for every user of the simpleblob interface. We may yet consider to split the S3 backend out in the future. The main thing holding us back is not wanting to break any existing code, but perhaps it is better to bite the bullet sooner rather than later while the number of external users is still limited. I think we are at a point where we do not foresee any breaking changes to the interface that would make keeping it within the repo more convenient.

Backends are registered with an _ import in the application, like this (this should also go into the readme of this repo):

import (
	// Register storage backends
	_ "github.com/PowerDNS/simpleblob/backends/fs"
	_ "github.com/PowerDNS/simpleblob/backends/memory"
	_ "github.com/PowerDNS/simpleblob/backends/s3"
)

These backends can live in any importable Go module, so you could create a github.com/udf2457/simpleblob-nats with the backend.

We are more than happy to link to external simpleblob backends in the README of this repository.

I will review your code in PR #59, as I appreciate the work you put into this, but please understand that it will not be merged in this repo.

Regarding Lightning Stream

I suspect that the main reason you created this backend is for use with Lightning Stream. Having a storage backend live in an external repository does not disqualify it from being included in the official LS release, but adding a storage backend does add a support burden on us and an expectation from users that it can be used in a reliable way.

Before we would consider adding an additional backend, we would like to see:

• It has seen actual production use over a period of time.
• There is interest from multiple users in actually using it.
• The backend is stable with no breaking changes, unless really unavoidable.
• The backend code is of good quality and maintained.
• The backend performance characteristics and usage are documented.

In the meantime, you can always build a LS binary with a custom backend by add an import for registration, for example like this:

echo -e "package main
import _ \"example.com/simpleblob-foo\"" > cmd/lightningstream/extrabackend.go

go install ./cmd/lightningstream

I'm curious to hear about your experience with this backend!