I want to share my experience using Palo Alto FW in AWS ( using Terraform ).
Palo Alto has some good examples out there of how their stuff works with Terraform and AWS, such as this github repo for thier two-tier implementation using Terraform. The two tier design here is a small starting point upon which to code out your platform. However, it does not take into account how the PA’s work with multiple AZs or Load Balancers, both of which will most likely be needed in your implementation.
Multiple AZ’s are not hard to do, but if you want Elastic Load Balancers in the mix, there is a Management Interface swap that needs to happen in order for the Palo Alto VM FW to work with AWS ELB.
This seemingly tiny, insignificant detail has been challenging to integrate into Terraform, and although PaloAlto does provide some documentation, it is not enough. Here is what I learned:
- An EIP association is also needed for eth1 ( what will become mgmt. interface ) if you are not using a Bastion host to connect initially.
- Another EIP association is also needed in Terraform for FW eth0 so device can read and get bootstrap from S3 Bucket. ( this EIP can later be torn down – but yes, you need TWO EIPs at boot time to make this work! ) [ UPDATE: a VPC endpoint w/ routing can replace the need for this second EIP ]
- FW Initial Ruleset must permit ssh / web to the IP of Ethernet 1/1 new mgmt ( that’s obvious but included here so not be be over looked )
- Ethernet 1/1 was not assigned to a Zone in PAN-OS initial config bootstrap when I got it working.
- Terraform interface templates for Palo Alto need to have mgmt. interface associated to device_index = 1 ( instead of zero ) in initial config. Configuring for Terraform, thus AWS, the actual interface assignments are abstracted, so you configure them as you will use them AFTER the swap.
Also of note, the PAN-OS does not “see” this change in the GUI. You will still implement your rules and zones as though there was not an interface change. Eth 1/1 shows up in Network interfaces… The only place to see this is in the CLI with this lengthy command:
Fig 1. Normal boot: admin@PA-VM> debug show vm-series interfaces all Interface_name Base-OS_port Base-OS_MAC PCI-ID Driver mgt eth0 06:82:4e:66:99:9a 0000:00:03.0 ixgbevf Ethernet1/1 eth1 06:1a:e7:12:01:e0 0000:00:04.0 ixgbevf Ethernet1/2 eth2 06:39:13:1b:e9:d4 0000:00:05.0 ixgbevf Fig 2. Booting with the command ‘op-command-modes=mgmt-interface-swap’ in the init.cfg admin@sample-cft-fw> debug show vm-series interfaces all Interface_name Base-OS_port Base-OS_MAC PCI-ID Driver mgt (interface-swap) eth0 06:db:de:02:e5:22 0000:00:04.0 ixgbevf Ethernet1/1 eth1 06:9f:0e:8b:de:ec 0000:00:03.0 ixgbevf Ethernet1/2 eth2 06:5d:86:02:b1:4e 0000:00:05.0 ixgbevf admin@sample-cft-fw>
It’s hard to notice a difference, but if it worked, you’ll see the (interface-swap) after mgt. ( above ) . Palo Alto provides this graphic to explain it, but it does not really line up well with the output above since the VM interfaces are extracted into AWS.
In Terraform, it lines up as so:
device_index = 0 will be eth0 in AWS, which is initial mgmt in Palo Alto (before swap), communication to the S3 bucket for bootstrap happens from this interface.
device_index = 1 will be eth1 in AWS, (which would be new mgmt if swapped)
device_index = 2 will be eth2 in AWS, not affected by swap
Also, before getting it to work in Terraform, I tried the command to swap interfaces on a FW VM series running n AWS that had booted normally; using Palo’s CLI:
set system setting mgmt-interface-swap enable yes
It responded with:
Reboot system to take effect new changes. After reboot use IP address of eth1 (external to VM) for management
After it booted again, I had trouble getting back into the Firewall with ssh. Got connection refused. Could have been various reasons, but at this stage, I think it was that I did not wait long enough. Another Engineer I am working with said that this worked for him but he had to wait a long time after boot.
This Management interface swap feels very much like the Palo Alto VM Firewall has been ‘shoe-horned’ to work in and with AWS. I have always loved Palo Alto, but not really a fan of this fix.
I’ll be writing more as I learn. I hope this helps you! Oh, before I go, here are some handy PAN-OS CLI commands you will use if you are doing this same thing:
op-command-modes=mgmt-interface-swap # for bootstrap set system setting mgmt-interface-swap enable yes # PA cli debug show vm-series interfaces all # show your stuff set mgt-config users admin password #you'll need this to get to the web GUI save config