Context: I am using the oracle/oci provider. I am trying to dynamically generate userdata using a template file. Within the template file, I am iterating through a map of block volume objects. This map of block volumes is attached to the bigger compute_instances var that gets iterated in the compute instance resource. This is so that more than 1 block volume can get attached to different device paths on the same instance, if the device_path is defined. I also have to allow the mounting of an optional mount target (which is nfs) within the userdata as well. I have tried combining both of the functionality within the user_data.tftpl tempatefile.
compute_instances var:
compute_instances = {
key1 = {
instance1 = {
instance_display_name = "name1"
fs_name = "fs_1"
mt_name = "mt1"
block_volumes = {
bv1 = {
device_path = "/dev/oracleoci/oraclevdb"
backup_policy_id = null
size = 100
mount_path = "/mnt/u01"
}
bv2 = {
device_path = "/dev/oracleoci/oraclevdc"
backup_policy_id = null
size = 75
mount_path = "/mnt/u02"
}
}
},
}
}
snippet of user_data being used in the compute instance resource with templatefile:
resource "oci_core_instance" "instance" {
for_each = var.compute_instances
...<bunch of code here>...
user_data = base64encode(templatefile("${path.module}/user_data.tftpl", {
fs_bool = local.fs_bool, private_ip = local.private_ip, fs_path = local.fs_path, fs_name = local.fs_name,
bvs = each.value.block_volumes
}))
}
the user_data.tftpl tempate using cloud-init:
#cloud-config
${yamlencode({
mounts = fs_bool == true ? [[ "${private_ip}:${fs_path}", "/mnt/${fs_name}", "nfs", "defaults,nofail", "0", "0" ]] : null
runcmd = [
for bv in bvs : [
["sudo", "mkdir", "${bv.mount_path}", "-p"],
["sudo", "mkfs", "-t", "xfs", "${bv.device_path}"],
] if bv.device_path != null && bv.mount_path != null
]
})}
The first mounts = fs_bool == true ? [[ "${private_ip}:${fs_path}", "/mnt/${fs_name}", "nfs", line is for mounting the filesystem to the mount target (nfs). Because I am already using the cloudinit mounts module for the nfs mount target (and want to avoid merging two different userdata together), I decided to use the runcmd for handling the block volume device paths.
The above runcmd does not work at all. I have tried MANY different versions of syntax for the looping in this runcmd like below:
runcmd = [
for bv in bvs : [
["sudo mkdir ${bv.mount_path} -p"],
["sudo mkfs -t xfs ${bv.device_path}"],
] if bv.device_path != null && bv.mount_path != null
]
or this
runcmd = [
for bv in bvs : [
"mkdir ${bv.mount_path} -p",
"mkfs -t xfs ${bv.device_path}",
] if bv.device_path != null && bv.mount_path != null
]
This last one is the closest I got. I validated the schemas with sudo cloud-init schema --system and shows no errors.
I also need to add the following commands into the runcmd to fully mount the device_path, but left them off for now so I could test things out (the syntax was getting tricky with the echo statement in the runcmd):
- echo '/dev/oracleoci/oraclevdb /mnt/u01 xfs defaults,_netdev,nofail 0 2' | tee -a /etc/fstab > /dev/null
- mount -a
So the runcmd part will eventually need to look something like this, but keeping it simpler for now (I do not think the syntax is correct here for the echo statement. echo statement is an additional syntactical complexity on top of this):
runcmd = [
for bv in bvs : [
"mkdir ${bv.mount_path} -p",
"mkfs -t xfs ${bv.device_path}",
"echo '${bv.device_path} ${bv.mount_path} xfs defaults,_netdev,nofail 0 2' | tee -a /etc/fstab > /dev/null"
"mount -a"
] if bv.device_path != null && bv.mount_path != null
]
When I looked at the runcmd script via sudo cat /var/lib/cloud/instance/scripts/runcmd
it showed:
#!/bin/sh
'mkdir /mnt/u01 -p' 'mkfs -t xfs /dev/oracleoci/oraclevdb'
When I tried runnig this script manually on the instance, it would give me a No such file or directory error. I am pretty sure the runcmd is supposed to look like this in the end:
#!/bin/sh
mkdir /mnt/u01 -p
mkfs -t xfs /dev/oracleoci/oraclevdb
but I do not understand how I can do it when I am using loops as it adds additional [] and quotes which I believe is messing up the syntax leading me to believe this may not be possible with the runcmd. I could not find anything else online and in this forum about this.
I have tried adding additonal layers of [], removing them, changing the quotes around, etc. Using loops with runcmd was not working so I tried to use the fs_setup cloudinit module. This would require me to use an additional mounts module so I know I will eventually need to use the data.cloudinit_config from the cloudinit provider to merge two different user data templates together (since I am already using the mounts module to mount the nfs mount target).
I was trying to avoid adding this additonal layer of complexity (and needing to create another template file for something so small) which was why I wanted to use runcmd together with mounts module in one templatefile. This way, I don’t need to use data.cloudinit_config. I may also in the future need to use multiple lines of runcmd iteratively as I’m doing now so I’d like to be prepared for that.
The fs_setup module unfortunately never worked for me. I could not see that it ever ran or any errors or anything related to it in the /var/log/cloud-init.log, so I decided to instead use bootcmd like this:
#cloud-config
${yamlencode({
bootcmd = [
for bv in bvs : [
"mkfs", "-t", "xfs", bv.device_path,
] if bv.device_path != null && bv.mount_path != null
]
mounts = [
for bv in bvs : [
bv.device_path, bv.mount_path, "xfs", "defaults,_netdev,nofail", "0", "2"
] if bv.device_path != null && bv.mount_path != null
]
})}
The above works, but it limits me to only having 1 full command in the loop, because the syntax starts breaking/not working once I try to add additional commands to the list like I was initially trying to do with runcmd.
Is there is any way for the runcmd to work like this below?:
runcmd = [
for bv in bvs : [
"mkdir ${bv.mount_path} -p",
"mkfs -t xfs ${bv.device_path}",
] if bv.device_path != null && bv.mount_path != null
or eventually… something like this (echo statement is an additional syntactical complexity on top of this, which is also why I was trying to use mounts module instead):
runcmd = [
for bv in bvs : [
"mkdir ${bv.mount_path} -p",
"mkfs -t xfs ${bv.device_path}",
"echo '${bv.device_path} ${bv.mount_path} xfs defaults,_netdev,nofail 0 2' | tee -a /etc/fstab > /dev/null"
"mount -a"
] if bv.device_path != null && bv.mount_path != null
]