In Automating VCF Deployments with GitHub Actions and Terraform, the pipeline stops when Terraform finishes: a VM exists in VCF Automation, ready to use. This post is the next hop. I will walk through how the VCFA template hands that VM to AWX, what cloud-init does on first boot, the lab gotchas that bit me, and the playbook that finishes day-two configuration.
If you have not set up the GitHub Actions and Terraform side yet, start with that post first. Everything here assumes a successful deploy from the same example repo: vcfa-terraform-example.
AWX + Ansible Integration
In this example, the VCF Automation template does the work in two passes. cloud-init handles first-boot plumbing, and then VCF Automation hands the finished VM to AWX for configuration. Here’s who does what:
- cloud-init – Creates the
ansibleuser with its SSH key, sets the hostname and static IP, and preps the box - VCF Automation – Adds the new VM to the AWX inventory and launches the job template
- AWX – Installs packages and nginx via Ansible, and optionally reboots the VM after an upgrade
How the Template Hands Off to AWX
No ABX actions or event subscriptions are needed here. The template uses the built-in Ansible Tower integration in VCF Automation, which works with AWX, and a Cloud.Ansible.Tower resource on the VM Apps canvas. Here’s the trimmed template, with the cloud-init part pulled out for the next step:
resources:
Cloud_Ansible_Tower_1:
type: Cloud.Ansible.Tower
properties:
host: ${resource.Cloud_vSphere_Machine_1.*}
account: awx # Must match the name of the Ansible Tower integration in VCF Automation
templates:
provision:
- name: nginx web server install # Exact name of the AWX job template
hostVariables:
ansible_host: ${input.ipAddress}
ansible_user: ansible
groups:
- test
Cloud_vSphere_Machine_1:
type: Cloud.vSphere.Machine
properties:
image: ubuntu-24-04
flavor: small
storage:
constraints:
- tag: stor:nfs
networks:
- network: ${resource.Cloud_vSphere_Network_1.id}
cloudConfig: |
# ... cloud-init, see the next excerpt ...
Cloud_vSphere_Network_1:
type: Cloud.vSphere.Network
properties:
networkType: existing
constraints:
- tag: net:nsx
The Ansible resource is where the handoff happens, and every field has a job:
- host –
${resource.Cloud_vSphere_Machine_1.*}binds the Ansible resource to the machine, so VCF Automation provisions the VM first and only then calls AWX - account – Must match the name of the Ansible Tower/AWX integration in VCF Automation exactly. Mine is named
awx - templates.provision – The AWX job template to launch on provisioning. It’s matched by name, character for character, so
nginx web server installhas to be spelled exactly like it is in AWX - hostVariables –
ansible_hostcomes from theipAddressinput, so AWX connects by IP and doesn’t care whether DNS has caught up yet.ansible_user: ansibleis the account cloud-init creates below - groups – VCF Automation drops the host into the
testgroup in the AWX inventory, which the playbook can target
The machine and network resources are the usual basics: an Ubuntu 24.04 image, a small flavor, and tag constraints that steer storage to NFS and the NIC onto an existing NSX network.
The cloud-init Side
AWX can only configure a VM it can log in to, so cloud-init sets the table before Ansible serves the main course. This is the trimmed cloudConfig (in the template it sits indented under cloudConfig: |):
#cloud-config
ssh_pwauth: yes
users:
- name: ansible
password: ${secret.awx-userPassword}
groups: sudo
shell: /bin/bash
sudo: ALL=(ALL) NOPASSWD:ALL
lock_passwd: false
ssh_authorized_keys:
- ${secret.awx-user-publicKey}
chpasswd:
list: |
ansible:${secret.awx-userPassword}
expire: false
hostname: ${input.hostname}
fqdn: ${input.hostname}.example.com
write_files:
- path: /etc/cloud/cloud.cfg.d/99-disable-network-config.cfg
content: |
network: {config: disabled}
- path: /etc/netplan/99-installer-config.yaml
content: |
network:
version: 2
renderer: networkd
ethernets:
ens33:
dhcp4: false
addresses:
- ${input.ipAddress}/${input.prefixLength}
routes:
- to: 0.0.0.0/0
via: ${input.gateway}
on-link: true
nameservers:
search:
- example.com
addresses: ${input.dns}
runcmd:
- hostnamectl set-hostname ${input.hostname}
- rm -f /etc/netplan/50-cloud-init.yaml
- netplan apply >> /var/log/cloud-init-custom.log 2>&1
# ... the rest of runcmd is in the gotchas below ...
Three things make the handoff work:
- The ansible user – Gets its password and public key from VCF Automation secrets (
${secret.awx-userPassword}and${secret.awx-user-publicKey}), so neither one lives in the template. Passwordless sudo lets the playbook usebecomewithout a prompt - A static netplan – The IP, prefix length, gateway, and DNS all come from the template inputs, which are the same inputs Terraform passes in. The
dnsarray renders as a list like["192.168.10.10"], which happens to be valid YAML for netplan - cloud-init keeps its hands off the network –
99-disable-network-config.cfgstops cloud-init from generating its own network config, and removing50-cloud-init.yamlleaves my netplan file as the only one beforenetplan apply
One check before you copy this: ens33 is the NIC name in my image. Run ip link on a VM from your image and use whatever name it shows.
Lab Gotchas from the Template
The rest of runcmd clones a Git repo as my regular user, which needs a private key. The key lives in a VCF Automation secret, and that secret stores the newlines as literal \n text, so writing it straight to a file with write_files gives you a one-line key that SSH rejects. printf '%b' turns the escapes back into real newlines:
# Rebuild the private key from a secret with literal \n escapes
- bash -lc 'printf '%b' "${secret.git-key}" > ~devuser/.ssh/id_ed25519'
- bash -lc 'chown devuser:devuser ~devuser/.ssh/id_ed25519 && chmod 600 ~devuser/.ssh/id_ed25519'
Before the clone, the script waits for the network to actually show up, and the very last step turns cloud-init off for future boots:
# Wait for DNS and TCP/22 to be reachable
- bash -lc 'until getent hosts github.com >/dev/null; do sleep 2; done'
- bash -lc 'until nc -z -w 3 github.com 22; do sleep 2; done || true'
# ... pre-seed GitHub host keys, then clone ...
- bash -lc 'sudo -u devuser git clone git@github.com:<owner>/<repo>.git /home/devuser/app'
# ... VCF CLI install, kubectl alias ...
- echo "Cloud-init finished at $(date)" >> /var/log/cloud-init-custom.log
- touch /etc/cloud/cloud-init.disabled
A few things I learned the hard way, or would change outside the lab:
- Wait for DNS and port 22 before cloning – Right after
netplan apply, the VM can’t resolve or reachgithub.comyet, and the clone fails. The twountilloops wait for both. They loop forever if the network never comes up, so wrap them intimeout 300if you’d rather fail than hang - Root login and password auth are lab-only – The full template also runs
sedto setPermitRootLogin yesandPasswordAuthentication yes, and setsssh_pwauth: yes. That’s handy when you’re poking at a lab VM, but anywhere else drop those lines, setssh_pwauth: false, and let AWX use key-only auth with theansibleuser’s key - Secrets end up on the VM – VCF Automation fills in
${secret.*}before the VM boots, so the values land in the instance’s user-data on disk. Treat the image and the VM as holding those secrets, and use a deploy key that can only read the one repo - Disable cloud-init when you’re done –
touch /etc/cloud/cloud-init.disabledkeeps cloud-init from running again on the next boot and rewriting the network or users. Because of that marker,cloud-init statusisn’t a reliable wait in the playbook. Waiting on/var/lib/cloud/instance/boot-finishedis safer - Your regular user comes from the image – The
users:list only createsansible, so thedevuseraccount the key and clone use has to already exist in the template image
The Playbook
The playbook behind the nginx web server install job template is ubuntu-install-webserver.yml. That’s what the NSX Ubuntu Webserver (AWX) catalog item launches once VCF Automation has added the new VM to the inventory. The job template uses that inventory (the host lands in the test group) and connects with a machine credential that holds the ansible user’s private key, the same key cloud-init plants on first boot.
It doesn’t wait on /var/lib/cloud/instance/boot-finished. The first task waits for the apt lock instead, so Ansible doesn’t collide with a package update still running from first boot. After that it dist-upgrades, installs nginx, and starts the service. It also adds the Kubernetes apt repo and installs kubectl, pinned to the 1.35 minor in my lab so it stays within one minor of the clusters I manage. The last task reboots the VM so those upgrades actually take effect.
As I mentioned earlier, touch /etc/cloud/cloud-init.disabled makes cloud-init status a poor wait. A first task on /var/lib/cloud/instance/boot-finished would still be safer than relying on the apt lock alone, because that lock only covers the package-manager race, not the rest of first-boot work.
- hosts: all
name: Ansible playbook to install a web server
become: true
vars:
kubectl_minor: "v1.35" # keep within one minor version of the clusters you manage
tasks:
- name: Wait for any other apt/dpkg process to finish
shell: |
timeout 600 bash -c 'while fuser /var/lib/dpkg/lock-frontend >/dev/null 2>&1; do
echo "Waiting for apt lock to be released..."; sleep 5;
done'
changed_when: false
- name: Upgrade all packages
apt:
upgrade: dist
update_cache: yes
cache_valid_time: 3600
- name: Install nginx (latest version)
apt:
name: nginx
state: latest
update_cache: yes
- name: Ensure nginx is running
service:
name: nginx
state: started
enabled: yes
- name: Ensure apt keyring directory exists
file:
path: /etc/apt/keyrings
state: directory
mode: "0755"
- name: Add Kubernetes apt signing key
get_url:
url: "https://pkgs.k8s.io/core:/stable:/{{ kubectl_minor }}/deb/Release.key"
dest: /etc/apt/keyrings/kubernetes-apt-keyring.asc
mode: "0644"
timeout: 30
register: k8s_key
retries: 3
delay: 10
until: k8s_key is succeeded
- name: Add Kubernetes apt repository
apt_repository:
repo: "deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.asc] https://pkgs.k8s.io/core:/stable:/{{ kubectl_minor }}/deb/ /"
filename: kubernetes
state: present
- name: Install kubectl from the Kubernetes apt repo
apt:
name: kubectl
state: present
update_cache: yes
register: kubectl_pkg
retries: 3
delay: 15
until: kubectl_pkg is succeeded
- name: Verify kubectl
command: kubectl version --client
register: kubectl_ver
changed_when: false
- name: Reboot the VM after updates
reboot:
msg: "Rebooting the VM to apply updates"
connect_timeout: 5
reboot_timeout: 600
pre_reboot_delay: 5
post_reboot_delay: 30
test_command: whoami
Successful Run
That’s job 449, the nginx web server install playbook, finishing successfully in a little over six minutes. After the reboot at the end of the playbook, the packages are upgraded, nginx is running, and kubectl is on the box. The VM is no longer just deployed; it is configured and ready to use.
Looking Forward
Together with the GitHub Actions and Terraform post, you now have a full path from a workflow_dispatch click to a configured VM. From here, the interesting work is hardening: fewer secrets on the guest, a cleaner handoff to AWX, and bringing All Apps organizations onto the same pattern with the vmware/vcfa provider.
If you try this in your own lab, I would love to hear which gotcha showed up first.
Leave a Reply