Problem: Untangle 8.1 OVF and ESX 4.1 in Transparent Mode

By admin, April 29, 2011 14:55

I’ve read the Wiki page for installing OVF on ESX and failed once already because I bridged the vNICs on the same vSwitch without using VLAN, so eventually I crashed my network as looping started to occur.

So this is my second attemp, please refer to the picture I’ve attached, hope you can give me some advice and suggestions.

1. So basically, I have ONE vSwitch with TWO physical NICs binded together to have Load balancing and failover.

2. Within, I have THREE Port Groups:
- External (no VLAN) connects to Internet
- Untangle DMZ (VLAN 21) – Useless as I will use Transparent Mode
- Untangle Internal (VLAN 22) – where I put VMs that I want to protect behind the Untangle.

Is the following concept correct?
1. When I use Untangle in Bridge or Transparent Mode, I will ONLY utlilize two interfaces External (no VLAN) and Untangle Internal (VLAN 22), so these are the two vNICs Untangle VM will connect to. This leaves Untangle DMZ useless, so I can remove it from VMX or VM configuraiton GUI?

2. I Understand I need to enable Promiscuous Mode in order to have Untangle to scan the network in transparent mode (ie, a sniffer that is), (side topic: Do I need to have Promiscuous Mode if I am using Route Mode?)

I understand I need to enable Promiscuous Mode on Virtual Switch Level (ie, toppest level), which I DON’T WANT to due to security reasons (ie, VM behind Untangle can sniff the whole network right?), Can I enable Promiscuous Mode in individual Port Group Instead?

If Yes, the ONLY Port Group need to have Promiscuous Mode enabled is Untangle Internal (VLAN 22) right? Where it is the Port Group all the VMs are going to connect to. I do not need to enable Promiscuous Mode in External (No VLAN), is this correct?

Or I HAVE TO ENABLE IT on vSwitch level? but why? I thought individual Port Group will OVERWRITE the default setting, NO?

But wait, no matter where I enabled the Promiscuous Mode (ie, vSwitch level or Port Group level), the risk is still here, can I say I am allowing all the VM to have the capability to sniff traffic on the network? If yes, this is absolutely NO GOOD in using Untangle as enabling Promiscuous Mode will open a big security hole in L2 (ie, enabling Promiscuous Mode will render my switch to a hub)

3. FYI, the TWO PHYSICAL NICs (ie, vmnic8 and vmnic0) are connected to the same physcial L2 switch. VLAN 21 AND VLAN22 have been configured on this physical switch as well, also VMware VST VLAN tagging is used on the Port Group. I wonder if my current configuration will STILL create a loop that will crash my network again? (I don’t see how it can, but really want to double make sure and confirm with you guys)

4. Where is the management interface for Untangle going to be in this case? Do I need to create a new port group say Untangle – Management VLAN 23, and also add a new vNIC (probably just use the one for DMZ) and then connect to this Untangle – Management port group.


Update May-1-2011

Now I understood Port Group with VLAN VST Mode won’t work with Untangle and confirmed again with what’s on the Wiki, the document said it clearly “Each vSwitch should be connected to it’s own Physical NIC, or at least be separated by VLAN tagging at the physical NIC level. ” (ie, on the Physical NIC level and the picture attached above also confirmed this).

So does this mean Promiscuous Mode for bridge mode can ONLY work on Virtual Switch Level, but not Port Group Level?

Um…it’s quite disapointed as I gradually found out Untangle on ESX has so many limitation (ie, no VLAN tagging, must enable Promiscuous Mode for vNic connecting VMs, must have Promiscuous Mode on vSwitch but not on Port Group).

So I have decided to use Route Mode now to aovid the above limitation now.

Since I don’t have any more physical NIC to spare, can I create an internal vSwitch (ie, WITHOUT NIC) for Untangle VM ?

ie, External > Untangle External > Untangle Internal (which is on the internal vSwitch without NIC) and all the VM will be on this same internal vSwitch, which will be all protected by Untangle that is.

This will work right? Anyone Please?


Update May-19-2011

Finally, I’ve got Untangle 8.1 OVF working under ESX 4.1 in route mode, the solution is very simple:

1. Simply remove the last NIC in VM configuration, this will get rid of the DMZ NIC, leaving only External and Internal NICs. These two NICs are exactly what Route Mode requires.

2. Assign External NIC to your external connectivity to the Internet, and Internal to a seperate Port Group (in my case it’s VLAN 20 – Untangle)

3. Reboot Untangle, now, you won’t be able to use the default admin/passwd to login, it’s ok, just reset it, after successfully login to the console, configure the statics IP for both External and Internal.

That’s all you need, simple and neat! and I am really starting to fall in love with Untangle’s GUI, they do look so much cooler than my dull Netscreen’s GUI.

Leave a Reply