When configuring networking on an ESX/ESXi server it is important that you plan and configure it correctly.
Doing so will ensure you get the best reliability, performance and security within your virtual network.
Depending on your requirements the configuration will vary slightly. For example if network storage such and iSCSI and NFS will be used the setup will need to plan for this.
I will explain the different setups for these needs.
The following setup is the best for a server with 8 physical networks cards. This setup would be implemented if network based IP storage is being used to ensure reliability, performance and security.
Physical NIC Virtual Switch Port Group(s)
vmnic0 vSwitch0 Service Console
vmnic1 vSwitch0 Service Console
vmnic2 vSwitch1 VMkernel (IP Storage)
vmnic3 vSwitch1 VMkernel (IP Storage)
vmnic4 vSwitch2 VM Network
vmnic5 vSwitch2 VM Network
vmnic6 vSwitch3 vMotion
vmnic7 vSwitch3 vMotion
Ideally VLANs should be used to seperate these networks.