Register private internal s3 object store volume

We have an internal AWS compatible S3 object store that I want to set up using CSI plugin. I assume the first step is registering the volume. Here is HCL:
id = “ebs0”
name = “my name”
type = “csi”
external_id = “externalid_1234”
plugin_id = “aws-ebs0”
access_mode = “single-node-writer”
attachment_mode = “file-system”
mount_options {
fs_type = “ext4”
mount_flags = [“rw”]
}
secrets {
aws_access_key_id = “mykeyid”
aws_secret_access_key = “mysecretkey1234”
}
context {
endpoint = “http://ipaddress:port”
}

Response is:
Error registering volume: Unexpected response code: 500 (rpc error: rpc error: no CSI plugin named: aws-ebs0 could be found)

The first step is running a CSI plugin job. The plugin_id field you’ve provided is pointing to a plugin named aws-ebs0, so if you haven’t run a plugin named that, then there’s nothing to register the volume to.

Thanks for the reply. When I run the CSI plugin job like the one in the docs, I get this response:

could not get metadata from AWS: EC2 instance metadata is not available

This isn’t an AWS EC2 instance. It is an on premise, AWS compatible s3 object store. How to I tell the plugin to go to my internal endpoint and not to AWS?

@tomrwillis the arguments you need to pass to the CSI plugin are going to be entirely controlled by that plugin and not Nomad. I might be able to give you some pointers, but I’m not familiar with any CSI plugins that claim to be S3-compatible. Can you provide a link to the plugin you’re using?

@tomrwillis interesting… I don’t see that error message anywhere in the code for that csi-powerscale plugin. Can you provide the full jobspec you’re using for the CSI plugin, and any logs you’re getting from the plugin allocation (via nomad alloc logs)?

That error looks suspiciously like it’s trying to fetch it’s own metadata to provide information back to the driver. That would imply that it’s looking for either input variables or otherwise-supplied runtime parameters (eg: perhaps interface id, hw addr, etc) and reverting to an instance metadata check in their absence.

Given that it’s using aws-ebs, I’m guessing the driver is making the assumption that you’re running in ec2.

I don’t think this is object-store compatible. It follows the CSI v1.1 spec:

It looks like you’re limited to NFS via the Powerscale implementation.

Given that it’s using aws-ebs , I’m guessing the driver is making the assumption that you’re running in ec2.

Yeah that’s why I was asking for the jobspec for the plugin. If the plugin is the AWS EBS CSI driver, that only works with AWS block storage, not S3 or any other storage types.