p.NamespacePath undefined (v1.12.7 and v1.13.3)

Vault
1

Hi,

I have a Go application that use Vault, importing the following packages:

  • github.com/hashicorp/vault/api
  • github.com/hashicorp/vault/sdk

In addition, for the unit tests the following are also imported to create a test cluster:

  • github.com/hashicorp/vault
  • github.com/hashicorp/vault-plugin-secrets-kv

There seems to be an issue should I try to perform either of the following:

  • 1.12.6 → 1.12.7
  • 1.13.2 → 1.13.3

For both of these, I am getting the following errors show up:

# github.com/hashicorp/vault/audit
../go/pkg/mod/github.com/hashicorp/vault@v1.12.7/audit/format.go:148:22: p.NamespacePath undefined (type logical.PolicyInfo has no field or method NamespacePath)
../go/pkg/mod/github.com/hashicorp/vault@v1.12.7/audit/format.go:339:22: p.NamespacePath undefined (type logical.PolicyInfo has no field or method NamespacePath)

This appears to be a backport:

commit 8e96ea16982ac926dbfd574506b41dc0e4a1d041
Author: hc-github-team-secure-vault-core <82990506+hc-github-team-secure-vault-core@users.noreply.github.com>
Date:   Fri May 5 10:13:23 2023 -0400

    backport of commit dee7fd839e5db8956d7c08495d8bd96feb062ac8 (#20526)
    
    Co-authored-by: Pratyoy Mukhopadhyay <35388175+pmmukh@users.noreply.github.com>

diff --git a/audit/format.go b/audit/format.go
index 88e264f14b..6cafc1b4f3 100644
--- a/audit/format.go
+++ b/audit/format.go
@@ -143,9 +143,10 @@ func (f *AuditFormatter) FormatRequest(ctx context.Context, w io.Writer, config
 
                for _, p := range auth.PolicyResults.GrantingPolicies {
                        reqEntry.Auth.PolicyResults.GrantingPolicies = append(reqEntry.Auth.PolicyResults.GrantingPolicies, PolicyInfo{
-                               Name:        p.Name,
-                               NamespaceId: p.NamespaceId,
-                               Type:        p.Type,
+                               Name:          p.Name,
+                               NamespaceId:   p.NamespaceId,
+                               NamespacePath: p.NamespacePath,
+                               Type:          p.Type,
                        })
                }
        }
@@ -333,9 +334,10 @@ func (f *AuditFormatter) FormatResponse(ctx context.Context, w io.Writer, config
 
                for _, p := range auth.PolicyResults.GrantingPolicies {
                        respEntry.Auth.PolicyResults.GrantingPolicies = append(respEntry.Auth.PolicyResults.GrantingPolicies, PolicyInfo{
-                               Name:        p.Name,
-                               NamespaceId: p.NamespaceId,
-                               Type:        p.Type,
+                               Name:          p.Name,
+                               NamespaceId:   p.NamespaceId,
+                               NamespacePath: p.NamespacePath,
+                               Type:          p.Type,
                        })
                }
        }
@@ -433,9 +435,10 @@ type AuditPolicyResults struct {
 }
 
 type PolicyInfo struct {
-       Name        string `json:"name,omitempty"`
-       NamespaceId string `json:"namespace_id,omitempty"`
-       Type        string `json:"type"`
+       Name          string `json:"name,omitempty"`
+       NamespaceId   string `json:"namespace_id,omitempty"`
+       NamespacePath string `json:"namespace_path,omitempty"`
+       Type          string `json:"type"`
 }
 
 type AuditSecret struct {

Considering that namespaces are in the enterprise version, can anyone shed any light on why this appeared and what might be causing the error using the opensource code?

Cheers

2

Although namespaces are an Enterprise feature, and large portions of the supporting code are closed source, you’ll find small bits of code that accomodate the namespace feature in many places throughout open source Vault. They serve no useful purpose in the open source edition, as there is only one namespace with a constant ID (root) and path (the empty string) but (I assume, since I don’t work for HashiCorp) it’s easier for HashiCorp to allow smaller parts of the logic, which are deeply intertwined with other code, to be visible, than to have to maintain two different versions of such files.

It appears highly likely that most/all code in the open source edition is kept in sync with the Enterprise edition at the file level.

This isn’t supported. It often works, but there’s no committent from HashiCorp that the root of the Vault repo is importable as a Go module. This is somewhat implied by the use of replace directives in its go.mod.

In the event you want to do this, you’d need to also ensure you use a compatible version of the nested Go modules, github.com/hashicorp/vault/api, github.com/hashicorp/vault/sdk, and any github.com/hashicorp/vault/api/auth/... modules used.

A version which would be compatible, would be the Git hash of the commit used for the github.com/hashicorp/vault module. The most recent tagged release of the nested modules will not necessarily be compatible.

This is because the code in the api and sdk subdirectories is effectively independently released under two versioning schemes:

  • Embedded directly within Vault server binaries, implicitly sharing the main Vault version.
  • As standalone libraries for other projects to import, using version numbers not tied to the Vault server release process.
3

Thanks for the reply, @maxb !

It appears highly likely that most/all code in the open source edition is kept in sync with the Enterprise edition at the file level.

Yes, that is my presumption too. The question was really, why did it suddenly appear as a problem, as a quick scan through the diff didn’t seem to show up anything that introduced a change elsewhere, although …

This isn’t supported. It often works, but there’s no committent from HashiCorp that the root of the Vault repo is importable as a Go module. This is somewhat implied by the use of replace directives in its go.mod .

… is the most likely explanation.

This is because the code in the api and sdk subdirectories is effectively independently released under two versioning schemes

I am doing a go get for the Vault version, so that related updates also appear, in this case I get a version change for the SDK when asking for the version that hits the issue, so that is presumably failing to acquire suitable versions for a given Vault version. It is a real shame that the testing code isn’t written so that it works separately, as that allows debugging in tests, similarly that the version of Vault seems to have moved from the SDK to the main code.

4

So, whilst upgrading SDK and API independently did remove the error with the 1.13.3 upgrade, other errors appeared. Whilst not what I wanted to do, but since the main vault code is only used in testing, upgrading to 1.14.0 removed all errors.

I am seriously considering removing the main vault code, as it seems to be responsible for almost all of dependabot’s attention (via itself or indirect deps) and is producing issues such as this on a regular enough basis.