VMadmin.co.uk

Ok so your server probably has more than one physical NIC, by default most have two built in.
You may have been wondering what is the best network configuration for ESX/ESXi Servers.
How many physical NICs to use, which vSwitch to assigned them to and what vSwitch to use for certain traffic.
If you have more than one physical switch that supports link aggregation 802.3ad then you may want to take advantage of that too.

The number of network cards you have and what your requirements are may differ but I belive the 4 NIC setup is the best for performance, reliability and functionality. You can have more if you like but 4 will certainly suffice.

Why would you want to go to the trouble of setting this up and annoying the network team...

1. To present trunk ports to all ESX network interfaces. This is to segregate traffic to different VLANs for improved security (802.1q). Meaning you will be able to put virtual machines onto different VLANs (Maybe you have dev, test and production VLANs).
2. To load balance across NICs and physical switches using the IP hash (802.3ad). This will improve bandwidth usage and increase reliability.

Setup the network configuration as follows on each ESX/ESXi Server:

Virtual Switch Layout:
Virtual Switch              Used For
vSwitch0                    Virtual Machines
vSwitch1                    Service Console and VMotion
vSwitch2                    Internal


Virtual Switch to Physical NIC to Physical Switch Mapping:
Physical Network Interface        Used by Virtual Switch        Connected to Physical Switch        Type
vmknic0                                            vSwitch0                                 Switch2                                Trunk
vmknic1                                            vSwitch1                                 Switch2                                Trunk
vmknic2                                            vSwitch0                                 Switch1                                Trunk
vmknic3                                            vSwitch1                                 Switch1                                Trunk


Virtual Networks:
Used For                    Port Group Type                VLAN ID
Server Network         Virtual Machines                 11
VMotion                       Vmkernel                             12
Service Console       Service Console                 12


Failover and Load Balancing Settings for vSwitchX:
Load Balancing:                        IP Hash
Network Failure Detection:      Link Status Only
Notify Switches:                         Yes
Failback:                                     Yes
Active Adaptors:                         vmnicX, vmnicX
Standby Adaptors:                     None
Unused Adaptors:                     None


NOTE: You cannot use VLAN 1 in virtual switches with trunking because the traffic is not tagged.


Switch Channel-Group Configuration for 802.3ad link aggregation
Create a port channel for each server.

interface Port-channel2
description ESX Server 1
switchport
switchport trunk encapsulation dot1q
switchport mode trunk
switchport nonegotiate
spanning-tree portfast trunk
end

Switch Port Configuration for 802.1q trunking
Make the port a trunk port and assign to that servers channel group.

interface GigabitEthernet2/10
description ESX Server 1
switchport trunk encapsulation dot1q
switchport mode trunk
switchport nonegotiate
spanning-tree portfast trunk
channel-group 2 mode on
end