Virtual Machine Performance – CPU Ready

I have had feedback that navigation of my blog to find past posts is difficult, so I am aiming to solve this by creating new sections which will hopefully help with navigation.

This section is dedicated to posts I have written relating to CPU ready.

I am still seeing environments on a regular basis where virtual machines are not being sized appropriated during initial deployment and tools such as vCenter Operations (even where it has been deployed) not being used to optimize performance of virtual machines and vSphere cluster/s.

I have customers buying new hardware, where it is simply not required. The goal of this section is to make sure people get the best return on investment (ROI) out of their hardware and VMware licensing.

CPU Ready

1. VM Right Sizing – An Example of the benefits

2. How Much CPU Ready is OK?

3. Common Mistake – Using CPU Reservations to solve CPU Ready

4. High CPU Ready with Low CPU Utilization

5. More Coming soon

Related Articles

1. Determining if multiple vCPUs are causing performance problem (VMware KB)

vmware_monster_vm

How much CPU ready is OK?

I have noticed a lot of search results hitting my blog asking

Question: How much CPU ready is OK?

so I thought I would address this question with a quick post.

Of course the answer is it depends, for example Server workloads have a lower tolerance to CPU ready than desktop workloads but as a rule of thumb, here is my thoughts.

For Production server workloads

<2.5% CPU Ready
Generally No Problem!

2.5%-5% CPU Ready
Minimal contention that should be monitored during peak times

5%-10% CPU Ready
Significant Contention that should be investigated & addressed

>10% CPU Ready
Serious Contention to be investigated & addressed ASAP!

In my experience, the above have been good for a rule of thumb.

However, applications which are latency sensitive may be severely impacted even with low levels of CPU ready, these types of VMs should be on clusters with lower CPU overcommitment, leverage DRS rules to separate the contending workloads or in extreme cases, dedicated clusters.

On the flip side, Some servers are much more tolerant to CPU ready, and 5%-10% CPU ready or higher may not noticeably impact performance.

Keep in mind that setting CPU Reservations does not solve CPU Ready, see my post on the topic for more details.

VMware vCenter Operations is a tool which can help easily identify contention (including CPU) within your vSphere environment.

For Virtual Desktop workloads, what level of CPU ready is acceptable will largely depend on the individual user (ie: Power User verses Task Worker). Keep in mind virtual desktop deployments generally have high CPU consolidation ratios of  around 6:1 all the way to >12:1.

I would suggest the following , again as a rule of thumb

<5% CPU Ready
Generally No Problem!

5%-10% CPU Ready
Minimal contention that should be monitored during peak times

>10% CPU Ready
Contention to be investigated & addressed where the end user experience is being impacted.

Any Higher CPU ready will likely be impacting your users, and should be investigated.

VMware have recently released vCenter Operations for View, which you could use to monitor your VMware View environment.

VM Right Sizing – An example of the benefits

I thought this example may be useful to show the benefits of Right sizing a virtual machine.

The VM is an SQL Database server with 4 vCPUs on a cluster which is highly overcommitted with lots of oversized VMs.

As we can see by the below graph, the CPU ready was more or less averaging 10% and on the 24th of July most vCPUs spiked to greater than 30% CPU ready each. ie: 30% of the time the server is waiting to be scheduled onto the pCPU cores.

The performance of applications using databases hosted on the server were suffering serious issues during this time.

On the  24th the VM was dropped from 4 vCPUs, down to 2 vCPUs and the results are obvious.

CPU ready dropped immediately (even in a heavily over-committed environment) to around 1% and CPU utilization remained at around the same levels. Performance also improved for applications (for example vCenter) using the database server.

TIP: Right Sizing not only helps the VM you right size, but it helps relieve the contention on the ESXi host (and cluster), which will improve performance for all VMs.

It is also important to point out this VM is the first VM to be right sized, so as more VMs are right sized in the cluster, Ready time will drop further and performance will continue to improve.

This also results in opportunities for greater consolidation within the environment without compromising performance or redundancy.

I would like to point out that I believe this server may benefit from 4 vCPUs, but definitely not in this highly CPU contended environment.

As more virtual machines are Right Sized, then this environment would likely have the opportunity to consider increasing vCPUs in suitable VMs after monitoring performance for a suitable period of time. Products like VMware vCenter Operations is excellent for reporting on Oversized and undersized VMs.

Do you believe in right sizing now?