Adding InfluxDB v2 support

[stevenleadbeater]

fixes: #12230
This PR introduces a new plugin for InfluxDB v2.0+, it's based on the influxdb-client-go library which provides a fluent API to influxdb. I've read through the comments on the issue. It looks as though development has stalled on the WIP mentioned and I'm not sure if a Vault team member has weighed on on whether to use a combined plugin or separates.
There are a lot of differences between Influxdb <= 1.8 and 2.0+, so much so making a combined plugin felt to me like treating 2 disparate systems as the same thing.
I took the original influxdb plugin as a starting point, moved it over to using the new client library and assessed the differences from there. I had to cut out the concept of roles entirely. They just don't exist in version 2.0+, there is the concept of organisations however, it's not the same and the mapping between them didn't feel like a good idea.
I've also redeveloped the same test cases as used by the original plugin, added all the documentation I could find and confirmed all the tests work.

[hashicorp-cla]

All committers have signed the CLA.

[stevenleadbeater]

Please hold off on reviewing this for now @taoism4504 @acahn I've been looking into this a bit more and vault needs to send API tokens to consumers, not Username / Password. It's not looking like it fits 100% with the plugin strcture at the moment. I'm doing some digging

[medykn-git]

Is this functionality still in progress?

[stevenleadbeater]

Is this functionality still in progress?

I think the functionality is fundamentally incompatible with vault. You need an API key for influxdb which it generates, vault expects to be able to generate a username and password. I may resurrect this in the near future as there may be a path through to getting an API key for a generated account using vault-agent and consul template. If I'm going down that route though I want to have a working example I can add to the documentation section

[rhyas]

FWIW, I think you are on the right path, and it's not as incompatible as you think. (: Yes, the functions expect 2 value response, which is typically user/pass, but it's essentially the same thing for a user/token. Anything using this endpoint is going to understand that Auth is done with the Token in Influxdb2, and the User is just a descriptive element. But you'll still create a user, and then a Token for that user, and return 2 values as user/pass (where pass == token).

The bigger challenge I think is passing args/params into the plugin for user/token relations to OrgID, and dealing with Permissions.

[stevenleadbeater]

I can't set the password in the plugin though, it's supplied by the caller and isn't returnable. All the permissions and organisation settings are pretty mappable. It's not trivial but it is possible.

// Database to manipulate users within an external system (typically a database).
type Database interface {
	// Initialize the database plugin. This is the equivalent of a constructor for the
	// database object itself.
	Initialize(ctx context.Context, req InitializeRequest) (InitializeResponse, error)

	// NewUser creates a new user within the database. This user is temporary in that it
	// will exist until the TTL expires.
	NewUser(ctx context.Context, req NewUserRequest) (NewUserResponse, error)

	// UpdateUser updates an existing user within the database.
	UpdateUser(ctx context.Context, req UpdateUserRequest) (UpdateUserResponse, error)

	// DeleteUser from the database. This should not error if the user didn't
	// exist prior to this call.
	DeleteUser(ctx context.Context, req DeleteUserRequest) (DeleteUserResponse, error)

	// Type returns the Name for the particular database backend implementation.
	// This type name is usually set as a constant within the database backend
	// implementation, e.g. "mysql" for the MySQL database backend. This is used
	// for things like metrics and logging. No behavior is switched on this.
	Type() (string, error)

	// Close attempts to close the underlying database connection that was
	// established by the backend.
	Close() error
}

NewUser must take a request object and return a response:

// NewUserRequest request a new user is created
type NewUserRequest struct {
	// UsernameConfig is metadata that can be used to generate a username
	// within the database plugin
	UsernameConfig UsernameMetadata

	// Statements is an ordered list of commands to run within the database when
	// creating a new user. This frequently includes permissions to give the
	// user or similar actions.
	Statements Statements

	// RollbackStatements is an ordered list of commands to run within the database
	// if the new user creation process fails.
	RollbackStatements Statements

	// Password credentials to use when creating the user
	Password string

	// Expiration of the user. Not all database plugins will support this.
	Expiration time.Time
}

// NewUserResponse returns any information Vault needs to know after creating a new user.
type NewUserResponse struct {
	// Username of the user created within the database.
	// REQUIRED so Vault knows the name of the user that was created
	Username string
}

I can't return a password I set on the response object.

[padraic-padraic]

Given that limitation of the database API, would it make more sense to implement InfluxDB 2.0 support as a full secrets-engine plugin, rather than just a Database plugin? Because yeah, while you can create users with basic auth via the API it doesn't seem like you can easily set permissions boundaries for those users without using tokens.

Of course, given that InfluxDB 2.0 does has a secrets store, which can use Vault as a backend, this could get a bit snake-eating-its-own-tail if you went down that route?

[TheOneValen]

Of course, given that InfluxDB 2.0 does has a secrets store, which can use Vault as a backend, this could get a bit snake-eating-its-own-tail if you went down that route?

InfluxDB 2 uses secrets (and therefore its vault backend) for connecting to other databases to scrape/aggregate data. No snakes here.

[maxcoulombe]

Hey @stevenleadbeater thanks for taking a jab at this. I wanted to jump in with some information if you or someone else wanted to continue working on the support of InfluxDB v2.

We had a brief internal discussion with the team on the topic and we'd recommend creating an external plugin in a dedicated repository instead a second version built-in directly in Vault. This is the recommended approach for any new plugin moving forward. You can look at the Elasticsearch or Redis projects for examples.

We'd be super happy to feature the repository in the community section of our plugin portal for visibility if you get something going.

I checked-out and tried what you have so far and you're definitely on the right track. If you don't mind however, I'd close this draft as we are unlikely to accept the contribution as a built-in plugin.

[lkubb]

If anyone is interested, I wrote an early (but mostly working) PoC for InfluxDB v2 token auth here: https://github.com/lkubb/vault-plugin-database-influxdb2

This plugin uses the deprecated v4 SDK because, as was mentioned here, the v5 interface does not fit with how InfluxDB v2 operates. Let's hope there will be a workaround before support for v4 plugins is dropped entirely, otherwise the secret backend might be the only option.

[Note that I'm not too fluent with Go and this is my first Vault plugin, there might be a lot of rough edges.]