ESX4 performance troubleshooting PDF

Hal Rosenberg, a performance engineer at VMware has just released a new Community Document entitled “Performance Troubleshooting for VMware vSphere 4“.

It uses numerous flow charts to help identify the cause of performance related issues.  It focuses on ESX 4 hosts, but Hal explains that he intends to update it and “add more detailed performance information, including troubleshooting information for more advanced problems and multi-host vSphere deployments”.  So keep checking back.

Go and grab it here.

Hello Planet

If all those interwebz tubes are correctly aligned, then I should be reaching out to you on Planet V12n. To quickly introduce myself, my name is Forbes Guthrie and I muse over at www.vReference.com.  I’m probably best known for my VI3 reference card and newly released vSphere4 reference card and notes.

I like to blog about virtualization things which are new to me. I try not regurgitate industry news I read on other sites; they do it better than I ever could. I like to concentrate on little snippets I discover along the way. If you’re new to my traffic, you might want to check back on some of my recent posts.

VMworld: Networking Deep Dive, Storage Log Analysis, VMFS3, HA, Fault Tolerance, Host Profiles, Reception Party

VMworld 2010 dates
Firewall PDF
VI3 and vSphere Posters
VMware document nerds
MSCS confusion
Hidden GUI disk policy

VMworld: Networking Deep Dive

At VMworld I attended the excellent “Networking Deep Dive” (TA2525) by Srinivas Neginhal.  Here are just a few things he discussed that jumped out me as either new information (not in the standard documentation) or something that expanded on my current understanding.

ESX hosts keep a local cache of the vDS and DVPort information, to use when vCenter is unavailable (both are non-editable binary files):
Host DVPort state:    /etc/vmware/dvsdata.db
VM DVPort state:      /vmfs/volumes/<storage>/.dvsData

DVPort binding types:

  • Static – assigned to the virtual adapter and immediately pushed to the host. Written to the host’s cache and written in the VM’s vmx file.
  • Dynamic – assigned when the VM is powered on, and then pushed to the host. There is no guarantee that the VM will get the same DVPort on the next  power on.  However it uses a concept similar to DHCP in that if the same port is available then it will renew that one.
  • Ephemeral – a new port on every power-on. The port is destroyed when the VM disconnects from the port. Essentially this is what happened on all vSwitch ports on ESX 3.x and lower.  This is the only option if you are connecting the client directly to the host, vCenter is required to create the other 2 types of binding.

He also explained the Traffic Shaping terminology, which seems obvious in retrospect but was something that I hadn’t “got”.  Simply put, its all relative to the switch.  So Ingress traffic is stuff coming in, whether from a VMNIC (physical adapter) or a vNIC (VM’s adapter).  Egress traffic is leaving the switch.

And one last thing which stood out was a table he used to quickly show some of the advanced vNIC driver capabilities.  I’d not seen this laid out in this way, and is much more specific than the standard documentation (in fact if this is correct, which I’m sure it is, then the current documentation is downright misleading – N.B I need to update my vSphere reference card with this).  It also makes it much easier to remember.

vNIC_driver

TSO – TCP Segmentation Offloading
RSS – Receive Side Scaling http://www.microsoft.com/whdc/device/network/NDIS_RSS.mspx
MSI/MSI-X – Message Signaled Interrupts http://blogs.sun.com/gnunu/entry/how_does_msi_x_work

Update: VMware has just released this performance whitepaper discussing the merits of the new vmxnet3 driver.

VMworld: Storage Log Analysis

I also hit “vSphere4 Storage Troubleshooting and Log Analysis” (TA1394) which was presented by Mostafa Khalil at VMworld this year.  Wow, he was a wealth of information.  Unfortunately he was pushed for time, and I had to leave this one early to get to my next session.  If a video of this ever popped up on youtube I think I’d watch it at least 5 times through.

Anyway, one thing he introduced which I thought was rather nifty was a new element to the Event Handling.  Now in vCenter, when certain events occur, the event’s description givess a hyperlink which takes you directly to an associated KB article explained what happened, why and what you might need to do to resolve things.  Currently this is limited to the 16 most common events, but they intend to add more links along with vSphere updates.