Couchbase Vault database plugin

Vault
1

Hi,
I have written a database plugin for Couchbase DB. It supports credential rotation and dynamic and static roles. Would anyone in the Vault engineering team be interested in taking a look at it to see what could be improved?
Regards, Francis.

2

Hi @fhitchen,

Yes, please open a PR and we’ll have a look.

Best,
Nick

3

Hi, I am having problems getting my PR past circleci compilation. I need a couchbase package which normally you pull with

go get github.com/couchbase/gocb/v2

Any pointers how to add this to the build? I added the module to the toplevel go.mod but to no avail.

regards, Francis.

4

What happens when you run the go get command you listed? That should be the correct way to update go.mod and go.sum.

5

Works fine, but how do I add that to the circleci build scripts…

I see this in the go.mod I checked in.

github.com/couchbase/gocb/v2 v2.1.1 // indirect
6

Why would the circleci build config need to be updated? I should’ve mentioned that you need to go mod vendor for the CI to build, maybe that’s what you’re seeing?

7

Thanks that tip helped.

I managed to get the go mod vendor command to add the couchbase client code to the vendor path and the build succeeded.

Now I guess I have to figure out how to spin up a couchbase DB container, initialize it and get it ready to accept the tests I have defined.

I hope it won’t be beyond me. I have a test db image already that contains test accounts and data buckets to grant access to, docker.io/fhitchen/vault-couchbase, no reason why I can’t pull that is there?

8

Here’s what we’re doing to test the physical storage backend currently: https://github.com/hashicorp/vault/blob/master/physical/couchdb/couchdb_test.go#L57

Ideally they would share the same code, i.e. move prepareCouchdbDBTestContainer from that package to a new package helper/testhelpers/couchdb and make it public. Similar to what we do with ldap, consul, mysql, etc. You can see an example of what that might look like in my PR here: https://github.com/hashicorp/vault/pull/8948/files

We’d prefer to use standard upstream images where possible, rather than someone’s personal image that might not be updated as often.

9

I have modified what was done for cassandra, couchbase requires many more ports to be mapped for plain text and TLS communications though. I am still using a custom docker image as the out of the box one from couchbase requires configuration steps to create an administrator account as well as selecting a subset of the services (most are not needed and will just waste resources) and a test data bucket to test creating accounts that can access that bucket. I will see if there is a way to package all of that and automate creation of the test couchbase db. Don’t really want to waste more time doing a docker build though?

10

The database init test passed…

PASS physical/raft.TestRaft_Snapshot_Restart (2.48s)
PASS physical/raft.TestRaft_Snapshot_ErrorRecovery (7.82s)
PASS plugins/database/couchbase.TestCouchbaseDB_Initialize (62.27s)
PASS plugins/database/couchbase
PASS plugins/database/hana
PASS plugins/database/influxdb

but the test-go-race step failed on something that I do not think is related to my changes…

=== FAIL: helper/testhelpers/teststorage TestReusableStorage/raft (59.32s)
=== PAUSE TestReusableStorage/raft
=== CONT TestReusableStorage/raft
— FAIL: TestReusableStorage/raft (59.32s)
teststorage_reusable.go:119: raft dir: /tmp/vault-raft-005687069
teststorage_reusable.go:119: raft dir: /tmp/vault-raft-217546200
teststorage_reusable.go:119: raft dir: /tmp/vault-raft-157801303
teststorage_reusable.go:119: raft dir: /tmp/vault-raft-581860042
teststorage_reusable.go:119: raft dir: /tmp/vault-raft-014599585
teststorage_reusable_test.go:183: Get https://127.0.0.1:54003/v1/secret/foo: redirects not allowed in these tests
=== FAIL: helper/testhelpers/teststorage TestReusableStorage (0.00s)

Please advise. What should I do next.?