Ask the Community
Groups
Backing up Virtual Machines in ESXi 6.0.x with Changed Block Tracking enabled returns incorrect changed sectors - Connect IT Community | Kaseya
<main> <article class="userContent"> <h2 data-id="summary"><strong>SUMMARY</strong></h2> <p>VMware has announced a bug related to ESXi 6.0.x whereby backing up a changed block tracking enabled virtual machine in ESXi 6.0.x returns incorrect changed sectors.</p> <h2 data-id="issue"><strong>ISSUE</strong></h2> <p>The Issue:<br>Virtual machine backups utilizing Changed Block Tracking (CBT) in ESXi 6.0 may experience the following symptoms: </p> <ul><li>The CBT API call QueryDiskChangedAreas() API call can sometimes return incorrect changed sectors, which results in inconsistent incremental virtual machine backups.</li> <li>Inconsistent virtual machine backups</li> </ul><br>This affects all backup vendors backing up ESXi 6.0.x via snapshots or incremental backups. End users may experience corrupted or inconsistent backups, causing data loss on restore. VMware has announced a bug related to ESXi 6.0.x: <a rel="nofollow" href="/home/leaving?allowTrusted=1&target=http%3A%2F%2Fkb.vmware.com%2Fselfservice%2Fmicrosites%2Fsearch.do%3Flanguage%3Den_US%26cmd%3DdisplayKC%26externalId%3D2136854">Backing up a Changed Block Tracking enabled virtual machine in ESXi 6.0.x returns incorrect changed sectors (2136854)</a>.<br><br>Resolution:<br>This issue has been resolved in VMware ESXi 6.0 patch ESXi600-201511001. See <a rel="nofollow" href="/home/leaving?allowTrusted=1&target=http%3A%2F%2Fkb.vmware.com%2Fselfservice%2Fsearch.do%3Fcmd%3DdisplayKC%26docType%3Dkc%26docTypeID%3DDT_KB_1_1%26externalId%3D2137545">VMware ESXi 6.0, Patch Release ESXi600-201511001 (2137545)</a> for further details. Questions relating to this patch should be directed to VMware Support (see <a rel="nofollow" href="/home/leaving?allowTrusted=1&target=http%3A%2F%2Fkb.vmware.com%2Fselfservice%2Fsearch.do%3Fcmd%3DdisplayKC%26docType%3Dkc%26docTypeID%3DDT_KB_1_1%26externalId%3D2006985">How to file a Support Request in My VMware</a>).<br> <blockquote class="blockquote"> Caution:<br>After applying the patch, a host reboot is required. In addition, you must reset CBT on your virtual machines (see: <a rel="nofollow" href="/home/leaving?allowTrusted=1&target=https%3A%2F%2Funitrends-support.zendesk.com%2Fhc%2Fen-us%2Farticles%2F360013269678">How to Reset Change Block Tracking (CBT) for VMware Backups</a>) and run a full backup to ensure data is consistent going forward.</blockquote> <br>Optional Work Arounds:<br>In the event upgrading to the patch above is not possible in the near future, VMware has suggested using one of the following workarounds. <ul><li>Downgrade the affected ESXi hosts to version 5.5, and downgrade the virtual Hardware Version from 11 to 10, if necessary. For more information, see VMware Knowledge Base articles <a rel="nofollow" href="/home/leaving?allowTrusted=1&target=http%3A%2F%2Fkb.vmware.com%2Fselfservice%2Fsearch.do%3Fcmd%3DdisplayKC%26docType%3Dkc%26docTypeID%3DDT_KB_1_1%26externalId%3D1033604">Reverting to a previous version of ESXi (1033604)</a> and <a rel="nofollow" href="/home/leaving?allowTrusted=1&target=http%3A%2F%2Fkb.vmware.com%2Fselfservice%2Fsearch.do%3Fcmd%3DdisplayKC%26docType%3Dkc%26docTypeID%3DDT_KB_1_1%26externalId%3D1028019">Downgrading the virtual machine hardware version in ESX/ESXi (1028019)</a>.</li> <li>Shutdown the virtual machine before doing an incremental backup.</li> <li>Do a full virtual machine backup rather than an incremental backup.</li> <li>Additionally, customers may utilize a Unitrends agent for backups within the VMs.</li> </ul><br>Important Considerations: <ul><li>If you are using Eager Zeroed thick virtual disks, just a full virtual machine backup is sufficient to resume incremental virtual machine backups.</li> <li>If you are using other types of virtual disks, a full reset of CBT data is required, see VMware KB 2139574, <a rel="nofollow" href="/home/leaving?allowTrusted=1&target=http%3A%2F%2Fkb.vmware.com%2Fkb%2F2139574">Resetting Changed Block Tracking for VMware vSphere virtual machines</a>.</li> </ul><br>Additional Resources:<br>Unitrends has created a script which may be used to reset CBT, enable/disable CBT, and get current status across multiple VMs. See <a rel="nofollow" href="/home/leaving?allowTrusted=1&target=https%3A%2F%2Funitrends-support.zendesk.com%2Fhc%2Fen-us%2Farticles%2F360013262198">VMware CBT Reset PowerShell Cmdlet</a> for instructions and to download the script. </article> </main>