Jun 26

Hardware and Software Requirements: Data Protection for VMware Version 7.1.2

Info, VE Comments Off on Hardware and Software Requirements: Data Protection for VMware Version 7.1.2

Abstract

This document details the Hardware and Software requirements for Tivoli Storage Manager for Virtual Environments: Data Protection for VMware 7.1.2.

Content

The Data Protection for VMware requirements vary by high level functional component. Data Protection for VMware has the following high level functional components:

  • Data Protection for VMware GUI
  • Data Mover
  • Recovery Agent

Note:
All Data Protection for VMware functional components must be at the same Fix Pack level. Within a given Fix Pack level, the interim fix level can differ.

Examples:

  • When upgrading the Data Protection for VMware GUI from 7.1.1 to 7.1.2, all Data Movers and recovery agents must be upgraded to 7.1.2.
  • When upgrading a Data Mover from 7.1.1.1 to 7.1.1.2, it is not required that all other components be upgraded to 7.1.0.2 (but they all must be at 7.1.1.x).

This document is divided into linked sections for ease of navigation. You may use the links below to jump to the desired section of the document.

Supported VMware vSphere versions

Supported VMware vCloud Director versions

Data Protection for VMware GUI supported operating systems

Data Mover supported operating systems

Recovery Agent & Recovery Agent Command Line Interface supported operating systems

Mount proxy supported operating systems

Additional software requirements and limitations

Hardware requirements for Data Protection for VMware

Where to download Tivoli Storage Manager for Virtual Environments 7.1.2

Hardware and Software requirements for other releases of Data Protection for VMware

Supported VMware vSphere versions

The following VMware vSphere versions are supported:

  • vSphere 5.1
  • vSphere 5.5
  • vSphere 6.0

Attention: ESXi 5.1 hosts must be at Update 3 or newer patch or update level. ESXi 5.5 hosts must be at Patch 4 or newer patch or update level. These prerequisite levels address a critical VMware Changed Block Tracking (CBT) integrity issue described in VMware KB 2090639.

Note:

  1. The Data Protection for VMware GUI “IBM Data Protection” extension requires vSphere Web Client version 5.5 update 1 (or later). Update 1 is included in the vSphere 5.5 package.
  2. To view supported vSphere and ESXi host versions, please refer to the following tech note: “Tivoli Storage Manager for Virtual Environments: Data Protection for VMware Data Mover compatibility guide with VMware’s Virtual Disk Development Kit”http://www.ibm.com/support/docview.wss?uid=swg21672351

Supported VMware vCloud Director versions

The following VMware vCloud Director versions are supported:

  • vCloud Director 5.1
  • vCloud Director 5.5

Data Protection for VMware GUI supported operating systems

This functional component integrates the product with the VMware vSphere client in order to back up, restore, and manage virtual machines in the VMware vCenter and includes the Data Protection for VMware command-line interface.

The supported operating systems by platform are detailed below.

Linux x86_64 platform

The following operating systems are supported for the Data Protection for VMware GUI on the Linux x86_64 platform.

  • Red Hat Enterprise Linux (RHEL) 5 Update 6 and later maintenance levels and mod levels: Desktop, Advanced Platform and Server Editions
  • Red Hat Enterprise Linux (RHEL) 6, and later maintenance levels and mod levels: Server Edition
  • Red Hat Enterprise Linux (RHEL) 7, and later maintenance levels and mod levels: Server Edition
  • SUSE Enterprise Linux 11, and later mod levels and Fix Packs: Server Edition

Windows x64 platform

The following operating systems are supported for the Data Protection for VMware GUI on the Windows x64 platform:

  • Windows Server 2008, and later maintenance levels: Standard, Enterprise, Datacenter Editions
  • Windows Server 2008 R2, and later maintenance levels: Standard, Enterprise, Datacenter Editions
  • Windows Server 2012 and later maintenance levels: Essentials, Standard, and Datacenter Editions
  • Windows Server 2012 R2 and later maintenance levels: Essentials, Standard, and Datacenter Editions

Note: Data Protection for VMware GUI provides support for Windows Storage Server 2012 for functionality that is also available and supported by the Data Protection for VMware GUI on the Windows Server 2012 Editions.

Data Mover supported operating systems

The Data Mover is installed on the vStorage Backup Server and enables off-host block level incremental backups. This function is enabled by the installation of the Tivoli Storage Manager (TSM) Backup-Archive Client 7.1.2, and later maintenance levels and Fix Packs. The TSM Backup-Archive Client is a separate prerequisite and is included as a convenience in the Data Protection for VMware 7.1.2 installation package. For Windows it may optionally be installed during TSM for Virtual Environments installation. The TSM Backup-Archive Client may also be downloaded independently – see below for details on additional software requirements and limitations.

Though the TSM Backup-Archive Client supports a wide variety of operating systems, only a subset of those operating systems are supported with TSM for Virtual Environments.

  • The 7.1.2 level of the TSM Backup-Archive client, and later Interim fixes of this Fix Pack, is required with Data Protection for VMware 7.1.2.

WARNING :
The TSM Client API 7.1.2 is also a prerequisite of Data Protection for VMware 7.1.2 and on Windows it is automatically installed as a part of the Data Protection for VMware 7.1.2 installation process. The TSM Backup-Archive Client 7.1.2 is a prerequisite to using the Data Mover function for off-host block level incremental backups. The TSM Backup-Archive Client and TSM Client API levels must remain in sync in your complete environment. Therefore, future updates to either the TSM Backup-Archive Client or the TSM Client API require the other component to be updated at the same time. I.e. if you install a new Fix Pack level of the TSM Backup-Archive client, you need to upgrade the TSM Client API to the exact same level on your Data Protection for VMware systems and vice versa. See also ” Upgrading to Tivoli Storage Manager for Virtual Environments Version 7.1.2

The VMware Virtual Disk Development Kit (VDDK) version 6.0 runtime files are used for backing up virtual machines by the Data Mover. Data Mover / TSM Backup-Archive Client with Data Protection for VMware supports the operating systems listed below, as long as they are also supported by VDDK version 6.0. For updates on supported OSes, known issues and limitations regarding VDDK 6.0 see the VMware web site.

The Data Protection for VMware Data Mover utilizes VMware’s Virtual Disk Development Kit (VDDK) API’s to open, close, read and write virtual disk data (VMDKs). To determine the level of VDDK used in each level of Data Protection for VMware, as well as compatible ESX/ESXi builds supported by the VDDK level, please refer to the tech note:
Tivoli Storage Manager for Virtual Environments: Data Protection for VMware Data Mover compatibility guide with VMware’s Virtual Disk Development Kit
located here http://www.ibm.com/support/docview.wss?uid=swg21672351

The supported operating systems by platform for the Data Mover / TSM Backup-Archive Client with Data Protection for VMware are detailed below.

Linux x86_64 platform

The following operating systems are supported for the Data Mover on the Linux x86_64 platform:

  • Red Hat Enterprise Linux (RHEL) 6 Update 6: Client and Server Editions
  • Red Hat Enterprise Linux (RHEL) 7.0: Client and Server Editions
  • SUSE Enterprise Linux 11 SP3: Desktop and Server Editions
  • SUSE Enterprise Linux 12: Desktop and Server Editions

Windows x64 platform

The following operating systems are supported for the Data Mover on the Windows x64 platform:

  • Windows Server 2008 R2: Standard, Enterprise, Datacenter, Storage Server, Storage Server R2, Small Business Server, Essential Business Server, and Web Editions
  • Windows Server 2012 Essentials, Standard, and Datacenter Editions
  • Windows Server 2012 R2 Essentials, Standard, and Datacenter Editions

Note: Data Mover provides support for Windows Storage Server 2012 for functionality that is also available and supported by Data Mover on the Windows Server 2012 Editions.

Recovery Agent supported op erating systems

This function provides virtual mount and instant restore capabilities. The supported operating systems by platform are detailed below.

Linux x86_64 Platform

The following operating systems are supported for the Recovery Agent on the Linux x86_64 platform:

  • Red Hat Enterprise Linux (RHEL) 51 Update 2, and later maintenance levels and mod levels: Desktop and Advanced Platform Editions
  • Red Hat Enterprise Linux (RHEL) 61, and later maintenance levels and mod levels: Client and Server Editions
  • Red Hat Enterprise Linux (RHEL) 7: Client and Server Editions
  • SUSE Enterprise Linux 11, and later maintenance levels and mod levels: Desktop and Server Editions
  • SUSE Enterprise Linux 12, and later maintenance levels and mod levels: Desktop and Server Editions


Note:

  1. Requires the following minimum packages and their dependencies:
  • compat-db-4.6.21-15
  • libXp-1.0.0-15.1
  • libXmu-1.0.5-1
  • libXtst-1.0.99.2-3
  • pam-1.1.1-4
  • libXft-2.1.13-4.1
  • gtk2-2.18.9-4
  • gtk2-engines-2.18.4-5

The Recovery Agent Command Line Interface (CLI) is not supported on the Linux x86_64 platform.

Windows x86 Platform

The following operating systems are supported for the Recovery Agent on the Windows x86 platform:

  • Windows Server 2008 SP1, and later maintenance levels: Standard, Enterprise, Datacenter, Storage Server, Storage Server R2, Small Business Server, Essential Business Server, and Web Editions
  • Windows 7, and later maintenance levels: Enterprise, Home Premium, Professional, Starter, and Ultimate Editions
  • Windows 8, Professional and Enterprise Editions
  • Windows 8.1, Professional and Enterprise Editions


Note:
If you use Active Directory with Microsoft Windows 2008 or Microsoft Windows 2008 R2, install the hot fix associated with Microsoft Knowledge Base article 970770 at:
http://support.microsoft.com/default.aspx?scid=kb;EN-US;970770

Windows x64 Platform

The following operating systems are supported for the Recovery Agent on the Windows x64 platform:

  • Windows Server 2008 SP11, and later maintenance levels: Standard, Enterprise, Datacenter, Storage Server, Storage Server R2, Small Business Server, Essential Business Server, and Web Editions
  • Windows Server 2008 R21, and later maintenance levels: Standard, Enterprise, Datacenter, Storage Server, Storage Server R2, Small Business Server, Essential Business Server, and Web Editions
  • Windows 7, and later maintenance levels: Enterprise, Home Premium, Professional, and Ultimate Editions
  • Windows 8, Professional and Enterprise Editions
  • Windows 8.1, Professional and Enterprise Editions
  • Windows Server 20122 Essentials, Standard, and Datacenter Editions
  • Windows Server 2012 R22 Essentials, Standard, and Datacenter Editions


Notes:

  1. If you use Active Directory with Microsoft Windows 2008 or Microsoft Windows 2008 R2, install the hot fix associated with Microsoft Knowledge Base article 970770 at: http://support.microsoft.com/default.aspx?scid=kb;EN-US;970770
  2. Recovery Agent & Recovery Agent Command Line Interface plug-in provides support for Windows Storage Server 2012 for functionality that is also available and supported by Recovery Agent & Recovery Agent Command Line Interface on the Windows Server 2012 Editions.

Mount proxy supported operating systems

A mount proxy represents the Linux or Windows system that provides access to virtual machine disks and requires an iSCSI connection. Mount proxy systems are required when using the Data Protection for VMware vSphere GUI Mount wizard (introduced in Version 7.1.1) to mount a backed up virtual machine disk. The user has the option to automatically create shares (either CIFS or NFS) of the mounted volumes for a file-level restore operation.

When mounting a Windows virtual machine, a Windows mount proxy system is required. When mounting a Linux virtual machine, a pair of Windows and Linux mount proxy systems is required.

For a Linux mount proxy system, the following applications must be installed and configured:

  • Tivoli Storage Manager Backup-Archive client Version 7.1.2.

Note: The iSCSI service must be configured and running during the virtual machine disk mount operation and the file-level restore operation.
By default, the iSCSI service is started manually. If the system where this service is running restarts, the iSCSI service must be restarted manually.

For a Microsoft Windows mount proxy system, the following applications must be installed and configured:

  • Tivoli® Storage Manager Backup-Archive client Version 7.1.2.
  • Tivoli Storage Manager for Virtual Environments Version 7.1.2: Tivoli Storage Manager recovery agent.

Note: The iSCSI Initiator service must be running during the virtual machine disk mount operation and the file-level restore operation.
By default, the iSCSI service is started manually. If the system where this service is running restarts, the iSCSI service must be restarted manually.

Data Protection for VMware requires the following operating system levels as mount proxy systems:

Linux x86_64 platform

  • Red Hat Enterprise Linux (RHEL) 6 Update 3 and Update 4: Client and Server Editions
  • Red Hat Enterprise Linux (RHEL) 7: Client and Server Editions
  • SUSE Enterprise Linux 11 SP3: Desktop and Server Editions
  • SUSE Enterprise Linux 12 : Desktop and Server Editions

Windows x64 platform

  • Windows Server 2008 R2: all Editions
  • Windows Server 2012 all Editions
  • Windows Server 2012 R2 all Editions

Additional software requirements and limitations

The following additional software is required to use Data Protection for VMware:

  • Tivoli Storage Manager (TSM Server) 6.2 and later interim fixes and Fix Packs
  • Tivoli Storage Manager (TSM Server) 6.3 and later interim fixes and Fix Packs
  • Tivoli Storage Manager (TSM Server) 7.1 and later interim fixes and Fix Packs

TIP: TSM Client/API and Server compatibility information is located in the IBM Tivoli Storage Manager Server/Client Compatibility and Upgrade Considerations tech note located here:
http://www.ibm.com/support/docview.wss?uid=swg21053218

  • Tivoli Storage Manager (TSM) Server 6.3.4, and later maintenance levels, Fix Packs, releases, and versions is required for use of following features that are new to Data Protection for VMware 7.1:
    • Show VMs with a backup report
    • Show failures from the most recent backup report
    • Show all backup failures report
    • Show backup history view
    • Show all history view
    • Show failed VMs view
    • Show all VMs view
  • Tivoli Storage Manager (TSM) Server 7.1.0, and later maintenance levels, Fix Packs, releases, and versions is required for use of the following features that are new to Data Protection for VMware 7.1:
    • Application protection reporting

Data Protection for VMware components support or require the following applications:

  • The Recovery Agent and Data Protection for VMware GUI require the following supporting applications on Linux:
    • Perl version 5
    • mdadm 2.6 tool for managing Linux Software RAID arrays
    • iSCSI Initiator 6.2 for Linux package
    • lsscsi command version 0.23
    • iscsiadm utility 0.10
    • Secure Shell (SSH) client for Linux 0.1
  • The following browsers are supported with the Data Protection for VMware GUI
    • Microsoft Internet Explorer version 9, 10 or 11 on Windows
      If launching the Data Protection for VMware GUI as a plug-in from the VMware vSphere Desktop Client, Microsoft Internet Explorer version 9, 10 or 11 must be installed on the system where the desktop client is installed.
      Note: vSphere Desktop Client plugin is not supported with vSphere 6.0
    • Google Chrome 13 or later on Windows and Linux
    • Mozilla Firefox 10 Extended Service Release or later on Windows and Linux
  • Application protection for VM guests is supported:
    • on the vStorage Backup server for the following platforms:
      • Microsoft Windows 2008 (64-bit)
      • Microsoft Windows 2008 R2 (64-bit)
      • Microsoft Windows 2012 (64-bit)
      • Linux 64-bit platforms
      • VMware ESX 5.x or later (Windows, Linux)
    • for the following applications:
      • Microsoft Exchange Server 2010
      • Microsoft Exchange Server 2013
      • Microsoft SQL Server 2008 (64-bit)
      • Microsoft SQL Server 2008 R2
      • Microsoft SQL Server 2012
      • Microsoft SQL Server 2014

Hardware requirements for Data Protection for VMware

Hardware requirements vary and depend on the following items:

  • Number of protected servers
  • Number of protected volumes
  • Data set sizes
  • LAN and SAN connectivity

The following table describes the hardware requirements that are needed to install Data Protection for VMware.

Component Minimal requirement Preferred
System IntelPentium D 3 GHz Dual Core processor or compatible Not applicable
Memory 2 GB RAM, 2 GB virtual
address space
Not applicable
Available hard disk 200 MB for ‘Documents and
Settings’ folder
2 GB
NIC Card 1 NIC – 100 Mbps 1 NIC – 1 Gbps

Where to download Tivoli Storage Manager for Virtual Environments 7.1.2

The Tivoli Storage Manager for Virtual Environments download document provides information about how and where to obtain the Tivoli Storage Manager for Virtual Environments 7.1.2 package. The information is located at: http://www.ibm.com/support/docview.wss?uid=swg24039450

TIP: Data Protection for VMware 7.1.2 is a component of Tivoli Storage Manager for Virtual Environments 7.1.2

Hardware and Software requirements for other releases of Data Protection for VMware

The Hardware and Software requirements for other releases and Fix Packs for all Tivoli Storage Manager Data Protection for Virtual Environment components are located in the “Tivoli Storage Manager for Virtual Environments – All Requirements Doc” located at:
http://www.ibm.com/support/docview.wss?uid=swg21505139

Related information

All Requirements Doc
Download Tivoli Storage Manager for Virtual Environment

written by Bosse

May 21

VMCLI node connects every 20 minutes to the Tivoli Storage Manager server

VE Comments Off on VMCLI node connects every 20 minutes to the Tivoli Storage Manager server

Problem(Abstract)

In an environment configured with Tivoli Storage Manager for Virtual Environments software, the Tivoli Storage Manager server shows a client connection every 20 minutes for the Data Protection for VMware VMCLI node.

Resolving the problem

By default, reconciliation operations run every 20 minutes on the Derby database to delete metadata for backups that are no longer available. This action ensures that the Derby database remains synchronized with the Data Protection for VMware repository. When reconciliation runs, the VMCLI node connects to the Tivoli Storage Manager server.

The interval between reconciliation operations on the Derby database with Data Protection for VMware is specified by the VMCLI_RECON_INTERVAL_TSM parameter, which is one of the Data Protection for VMware command-line interface profile parameters. The default value for the parameter is 1200 seconds (20 minutes).

You can adjust the value of the parameter to reduce the frequency of connections made by the VMCLI node. For example, to have the VMCLI node connect to the Tivoli Storage Manager server once a day use the following parameter :

VMCLI_RECON_INTERVAL_TSM 86400

Note that the VMCLI node also establishes a connection to the Tivoli Storage Manager server when the Tivoli Storage Manager for VMware UI is started, regardless of the value set with the VMCLI_RECON_INTERVAL_TSM parameter.

written by Bosse

Feb 03

Understanding metadata for TSM DP for VMware VM backups

VE Comments Off on Understanding metadata for TSM DP for VMware VM backups

Question

How can the size needed for metadata of VM backups be estimated to size the storage pool defined as destination by the VMCTLMC option ?

Answer

The VMCTLMC management class and storage pool destination is required when backing up virtual machines to tape or Virtual Tape using TSM for Virtual Environments. These “control” files (or “metadata”) provide a fast index to locate the virtual disk blocks backup data in TSM. The storage pool destination for these control files, associated with the VMCTLMC management class, should be on a random disk storage device type.
The easiest method to estimate the required amount storage for the VMCTLMC storage pool is to multiply the total retained VM backup data by 0.2% (0.002 decimal). The 0.2% value is an approximation and may not be appropriate for all environments, but has been shown to be valid in a sample of actual customer installations. To estimate the size requirement for the VMCTLMC storage pool, you must first estimate the total amount of retained backup storage for VMware backups. The total retained storage is total amount of data that is backed up for virtual machines, including all versions.

As an example let’s consider a virtual machine with a single vmdk of 100GB, with an average 10% daily change rate of data, with 15 days retention policy.

The total retained data is the original source data (100GB) plus the daily amount of data changed (10% * 100GB = 10GB) times the number of days retained (15). The total is (100 GB) + (10GB * 15 days) = 100GB + 150GB = 250GB. To estimate the amount of VMCTLMC data required multiple the retained data by 0.2%: 250GB * 0.2% = 0.5GB or 512 MB of disk storage.

In summary:

  • Original source data = 100 GB
  • Total retained data = 100 GB + (100 GB * .10 change rate * 15 days) = 100 GB + 150 GB = 250 GB
  • VMCTLMC data required = 250 GB * 0.002 VMCTLMC estimate = 0.5 GB

Remember that this is an approximation, thus you may want to increase or round this value up to ensure that you allocate sufficient storage pool, for example to 600GB. As with any TSM storage pool, you should monitor and set alerts for the percent utilization so you can increase the size of the storage pool if the utilization reaches a critical level, such as over 90%. Backups will fail if the VMCTLMC storage pool runs out of space. Control files associated with expired data will be removed from the VMCTLMC storage pool destination. However the timing actual expiration cycle may result in the VMCTLMC storage pool retaining some expired data and in the short term exceeding the estimate of 0.2%.

The following sections describe the analysis of VMCTLMC capacity estimation in greater detail. However, this information is generally not required if you use the simplified method of estimation using 0.2% as the guideline.

Detailed Analysis of VMCTLMC storage pool estimation method:

The VMCTLMC control files contain index information for each block of a virtual disk (vmdk) in VMware. These blocks are 16KB each. TSM stores the 16KB blocks in larger objects called “Megablocks” that are 128MB. A single control file is used for each TSM Megablock. For a 100GB vmdk, there are 800 megablocks [(100GB * 1024)/128MB]. For an initial full backup, 800 megablocks will be stored, as well as 800 control files. Each control file is a fixed size of 73KB. The total amount of space required for the control files with the initial full is 800 * 73KB = 58400KB. As a percent of retained data, the control files in this case (single full backup only) will be (58400KB)/(100GB * 1024 * 1024) = 0.000557 or 0.06% rounded. This represents the best case, or minimum, amount of VMCTL MC data as a percent of retained data.

Best case calculation (minimum distribution of changed vmdk blocks):

When incremental backups occur which affect individual 16KB vmdk blocks and not an entire 128MB TSM Megablock, only the changed data is backed up to TSM. A new control file is generated when as little as a single 16KB vmdk block is changed. The best case scenario, where the fewest megablocks are affected and the least number of new control file are created occurs when all of the changed data is consolidated in as few megablocks as possible. For example, with an incremental backup with 10% changed data on a 100GB vmdk, there would be a minimum of (10GB * 1024)/128MB = 80 changed megablocks. The amount of control file data for just this one incremental backup is 80 * 73KB = 5840KB. As a percent of the incremental backup data, the control files require (5840KB)/(10*1024*1024) = 0.000557 or 0.06% rounded of the backup data. Since the best case percentage is the same as the initial full backup, subsequent incremental backups will not change the percent calculation when calculating control file space as a percent of total retained data.

The 0.006% value calculated about represents the theoretical absolute minimum storage required for the VMCTLMC, and therefore is not a good value to use for estimation for an actual deployment.

Worst case calculation (minimum distribution of changed vmdk blocks):

It is most likely that the changed vmdk blocks are distributed to some degree throughout the virtual disk. The result of this fragmentation is to distribute the changed data over a greater number of megablocks, and results in a greater number of control files that are created. The worst case is that the changed vmdk blocks are distributed over the maximum number of megablocks possible, which in this example is 800. (There are only 800 megablocks for the 100GB vmdk, therefore this represents the maximum.) In this worst case scenario, the maximum amount of control file data that will be generated is 800 * 73KB = 58400. The associated amount of backup data for these control files is only 10GB (the 10% changed data), rather than 100GB in the case of the full backup. Therefore, the percentage of control file data (VMCTLMC) required is (58400KB)/(10GB * 1024 * 1024) = 0.00557 or 0.6% rounded. Note that this is 10 times the percentage calculated for the best case. As the number of retained incremental backups increases, the calculation of control file space as a percentage of retained backup data will approach 0.6%.

Estimation for most likely scenario

In an actual TSM for Virtual Environments deployment, neither the theoretical best case nor the worst case are likely, and the most likely scenario will be somewhere between. Based on a sampling of actual deployments, the steady state value of the ratio of control file space required compared with total retained data is approximately 0.2%. At any particular point in time the percentage may be higher such as due to delays in expiring old backup data, smaller change rates (which can result in the same amount of control file data for a smaller amount of backup data), or a wider distribution of changed data in vmdk data blocks. Best practice is to monitor the percent utilization of the VMCTLMC storage pool to determine if additional space is allocated.

written by Bosse

Jan 30

IT05265: “BACKUP VM” MAY FAIL TO PASS THE DISK VERIFICATION CHECK WITH ERRORS ANS9921E/ANS9919E

VE Comments Off on IT05265: “BACKUP VM” MAY FAIL TO PASS THE DISK VERIFICATION CHECK WITH ERRORS ANS9921E/ANS9919E
  • Tivoli Storage Manager ifincr "backup vm" may be completed with
    status failure (RC12) if client fails the virtual machine disk
    verification check.
    
    The client's console shows the following errors:
    
    ANS9921E Virtual machine disk, <VMNAME> (Hard Disk 1),
    verification check failed (1234567890/1234567890).
    ANS9919E Failed to find the expected control files for <VMNAME>.
    
    The problem occurs if  the "size on disk" value doesn't match a
    multiple of 16kb of the sector size declared in the vm control
    file.
    
    Customer/L2 diagnostic:
    
    Example:
    ***********************************
    VmVerifyIfSingleDisk():
         Found disk: Hard Disk 2
         Verifying disk backup ctls: checking size on disk vs ctl
    size
                                     coverage: Hard Disk 2.
         Num of CTLs = 425; type = IFFULL
         Skipping BITMAP.DAT file
         VM / Disk         : <VMNAME>/ Hard Disk 2
         capacity          : 32226647040
         size on disk      : 28151880704
         ctl coverage size : 28151889920
         disk included     : Yes
         prev backup ifincr: No
         ctl matches size  : No
         ctl found         : Yes
         bitmap found      : Yes
         disk used         : Yes
         result            : OK; Prev backup full; will recheck next
    ifincr
    ***********************************
    
    Generally the "ctl coverage size" is equal to the "size on
    disk". In this case the "ctl coverage size" matches the next
    16KB multiple and it should be enough to allow for a successful
    backup, however the check on the "size on disk" fails and makes
    the "backup vm" command exit with a failure.
    
    The problem occurs if the value of the "size on disk" is not a
    multiple of 16KB (16384 bytes). In the example given above the
    value "28151880704" is not a multiple of 16K.
    
    A trace gathered for an earlier VM backup belonging to the same
    ifincr chain may also show the following message:
    
    ************************
    ..\..\common\vm\vmbackvddk.cpp(6163): VmProcessExtent(): Extent
    not multiple of numSectorsPerRec 32; Last block partial read of
    14 sectors
    ************************
    
    The fullvm backup feature is not affected by this defect because
    the check is performed only during the incremental vm backup.
    
    | MDVREGR 6.4.2.001-TIV_5698MCL |
    | MDVREGR 7.1.1.000-TIV_5698MCL |
    
    Tivoli Storage Manager Versions Affected:
    All supported Tivoli Storage Manager Client on all supported
    platforms.
    
    Initial Impact:
    Medium
    
    Additional Keywords:
    vm backup fail zz62 zz63 zz64 zz71 disk vmdk ctl
    

 

Local fix

  • The disk check can be disabled, but the check was put into place
    to detect unrecoverable backups described by Flash (Alert)
    number 1684022,
    TSM data mover backups of VMware guests might be unrecoverable:
    http://www.ibm.com/support/docview.wss?uid=swg21684022
    
    If, after reading the flash and determining its applicability,
    you decide to temporarily suspend the check so that the VM can
    be backed up, then disable disk size verification by adding the
    following line in the dsm.opt file:
    VMVERIFYIFLATEST NO
    
    The VMVERIFYIFLATEST option should be re-enabled afterwards.
    

Problem summary

  • ****************************************************************
    * USERS AFFECTED:                                              *
    * Tivoli Storage Manager for Virtual Environments version 7.1  *
    * running on all Microsoft Windows and Linux x86_64 platforms. *
    ****************************************************************
    * PROBLEM DESCRIPTION:                                         *
    * See ERROR DESCRIPTION                                        *
    ****************************************************************
    * RECOMMENDATION:                                              *
    * This issue is projected to be fixed in the Tivoli Storage    *
    * Manager Client on Windows and Linux x86_64  in level 7.1.2   *
    * Note 1: The Tivoli Storage Manager (TSM) Client is a         *
    * prerequisite to using the Data Protection for VMware. In     *
    * Data Protection for VMware environments, the TSM Client is   *
    * also known as the data mover.                                *
    * Note 2: This is subject to change at the discretion of IBM.  *
    ****************************************************************
    

Problem conclusion

  • Disk verification is successful, if ctl coverage is greater then
    disk size on disk, disk size on disk is not multiple of 16kb and
    the difference is less then 16kb.
    

written by Bosse

Jun 24

Backup of VMware Windows MS SQL guest fails

VE Comments Off on Backup of VMware Windows MS SQL guest fails

Problem(Abstract)

Tivoli Storage Manager for Virtual Environments is used to back up a Windows guest that runs Microsoft SQL. The backup of the VM fails with the ANS9365E and ANS5250E errors.

Symptom

The following errors are logged during the backup :


04/30/2014 23:18:46 ANS9365E VMware vStorage API error for virtual machine 'myguest'.
TSM function name : visdkCreateVmSnapshotMoRef
TSM file : vmvisdk.cpp (5416)
API return code : 67
API error message : An error occurred while quiescing the virtual machine. See the virtual machine's event log for details.
04/30/2014 23:18:46 ANS5250E An unexpected error was encountered.
TSM function name : vmVddkFullVMPrePareToOpenVMDKs
TSM function : snapshot targetMoRefP is null
TSM return code : 115
TSM file : ..\..\common\vm\vmbackvddk.cpp (12439)
04/30/2014 23:18:46 ANS1228E Sending of object 'myguest' failed.

 

Diagnosing the problem

Examine the vmware.log of the virtual machine, located in the folder of the virtual machine in the datastore. The following errors are logged :

2014-05-05T02:32:18.259Z| vcpu-1| I120: [msg.snapshot.quiesce.vmerr] The guest OS has reported an error during quiescing.
2014-05-05T02:32:18.259Z| vcpu-1| I120+ The error code was: 5
2014-05-05T02:32:18.259Z| vcpu-1| I120+ The error message was: 'VssSyncStart' operation failed: IDispatch error #8449 (0x80042301)

Examine the Windows event log of the Windows guest. The following errors are logged :

4/30/2014 11:18:30 PM VSS Error None 6008 N/A MYGUEST
Sqllib error: Failed to create VDS object. hr = 0x80040154.
4/30/2014 11:18:30 PM VSS Error None 20 N/A MYGUEST
Volume Shadow Copy Service error: A critical component required by the Volume Shadow Copy service is not registered. This might happened if an error occurred during Windows setup or during
installation of a Shadow Copy provider. The error returned from CoCreateInstance on class with CLSID
{40700425-0080-11d2-851f-00c04fc21759} and Name MSSQL_ClientVirtualDeviceSet is [0x80040154].

 

Resolving the problem

In this case, the backup fails is caused by a VSS problem on the Windows guest. Follow the steps below to resolve the error :

  1. Stop SQL Server.
  2. Click Start/Run and enter the following command in the Open box, and then click OK.

    Regsvr32 Path\SQLVDI.DLL

    The default path of the Sqlvdi.dll file is C:\Program Files\Microsoft SQL Server\80\COM.
  3. Restart SQL Server.
  4. Run the backup of the VM

Refer to the following Microsoft article for more information about the VSS SQL error.
http://support.microsoft.com/kb/830575

written by Bosse

Apr 16

Troubleshooting Volume Shadow Copy (VSS) quiesce related issues

VE Comments Off on Troubleshooting Volume Shadow Copy (VSS) quiesce related issues

Details

•When quiescing a guest operating system using the Microsoft Volume Shadow Copy Service (VSS) prior to a snapshot, the operation fails.

•You see one of these errors:

Cannot create a quiesced snapshot because the snapshot operation exceeded the time limit for holding off I/O in the frozen virtual machine

An error occurred while quiescing the virtual machine. The error code was: 4 The error was: Quiesce aborted

•In the Event Viewer Application logs of the Windows Virtual Machine, you see errors similar to this:

◦Event ID 11
◦Event ID 12292
◦Event ID 12032
◦Event ID 12298

•The Event ID errors contain descriptions similar to this:

Volume Shadow Copy Service information: The COM Server with CLSID {} and name cannot be started. [].

Solution

Background Information

VMware products may require file systems within a guest operating system to be quiesced prior to a snapshot operation for the purposes of backup and data integrity.

VMware products which use quiesced snapshots include, but are not limited to, VMware Consolidated Backup and VMware Data Recovery.

As of ESX 3.5 Update 2, quiescing can be done by Microsoft Volume Shadow Copy Service (VSS), which is available in Windows Server 2003.

Operating systems which do not have VSS make use of the SYNC driver for quiescing operations. When VSS is invoked, all VSS providers must be running. If there is an issue with any third-party providers or the VSS service itself, the snapshot operation may fail.

 

Before verifying a VSS quiescing issue, ensure that you are able to create a manual non-quiesced snapshot using the vSphere Snapshot Manager. For more information see Troubleshooting issues when creating or committing snapshots in ESX/ESXi (1038963).

 

Troubleshooting Steps

Note: These steps may vary depending on the operating system.
  1. Ensure that you meet all of the prerequisites of VSS when quiescing guest operating systems:
    • Verify that you are running ESX 3.5.0 Update 2 or higher.
    • Verify that the latest version of VMware Tools is installed on the virtual machine. VSS components must be explicitly specified during the VMware Tools upgrade process. VSS is not installed in non-interactive mode. For more information, see Verifying a VMware Tools build version (1003947).
    • Ensure that you are using Windows Server 2003 or higher. Previous versions of Windows, such as Windows XP and Windows 2000, do not include VSS and rely on the SYNC driver.
  2. Ensure that all the appropriate services are running and the startup types are listed correctly:
    Note: With the exception of the VMware Snapshot Provider service, all services listed below are Microsoft services bundled with the operating system. If you are having difficulty starting these services, consult Microsoft support before proceeding with the troubleshooting steps.

    • During the installation of VMware Tools (this is required to install the VMware Snapshot Provider Service):
      • Ensure that the COM+ System Application service is listed as Started and that the startup type is listed as Manual.
      • Ensure that the COM+ Event System service is listed as Started and that the startup type is listed as Automatic.
    • While Idle:
      • Ensure that the COM+ System Application is listed as Started and that the startup type is listed as Manual.
      • Ensure that the COM+ Event System service is listed as Started and that the startup type is listed as Automatic.
      • Ensure that the Volume Shadow Copy service is not running and the startup type is listed as Manual.
      • The Microsoft Software Shadow Copy Provider service may or may not be started. Ensure that the startup type is listed as Manual.
      • Ensure that the VMware Snapshot Provider is not running and that the startup type is listed as Manual.
    • During the backup:
      • Ensure that the COM+ System Application is listed as Started and that the startup type is listed as Manual.
      • Ensure that the COM+ Event System service is listed as Started and that the startup type is listed as Automatic.
      • The Volume Shadow Copy service may or may not be started. Ensure that the startup type is listed as Manual.
      • Ensure that the Microsoft Software Shadow Copy Provider service startup type is listed as Manual.
      • Ensure that the VMware Snapshot Provider is listed as Started and that the startup type is listed as Manual.
      • Ensure that the Virtual Disk is listed as Started and that the startup type is listed as Manual.
  3. Make sure that you are using the Microsoft Software Shadow Copy provider:
    1. Click Start > Run.
    2. Type cmd and press Enter to open a command prompt.
      Note: You may need to run this as administrator.
    3. Check the VSS Providers with this command:
      C:\Users\Workstation> vssadmin list providers
      The output appears similar to this:
      Provider name: ‘Microsoft Software Shadow Copy provider 1.0’
      Provider type: System
      Provider Id: {b5946137-7b9f-4925-af80-51abd60b20d5}
      Version: 1.0.0.7

      Check the permissions on the VSS and on any third party VSS providers and ensure that the account is valid. For more information, see the Microsoft Knowledge Base article 259733.
      If you have a third-party provider, it may interfere with the quiescing operation. Try uninstalling any third-party VSS providers.
      Note: The vssadmin utility and VSS are bundled with the Microsoft operating system. If the vssadmin utility is reporting errors, this may indicate that you may have a pre-existing issue with the VSS. Consult Microsoft support before proceeding with the next troubleshooting steps.
  4. Ensure that all of the VSS writers are stable and that they are not reporting an error. Run this command:
    C:\Users\Workstation> vssadmin list writers
    The output for each writer appears similar to this:
    Writer name: ‘VSS Metadata Store Writer’
    Writer Id: {75dfb225-e2e4-4d39-9ac9-ffaff65ddf06}
    Writer Instance Id: {088e7a7d-09a8-4cc6-a609-ad90e75ddc93}
    State: [1] Stable
    Last error: No error

    Ensure that the State is Stable and that the last line of the output is Last error: No error.
  5. If the issue persists, perform these troubleshooting steps:
    1. Run NTBackup and see if there are VSS errors when NTBackup attempts to run. For more information, see the Windows NT Backup – Restore Utility page.
    2. Try reinstalling VMware Tools to re-register VSS.
    3. If you are having difficulty backing up information from specific applications such as Microsoft Exchange, Microsoft SQL, and Active Directory, ensure that all necessary VSS writers are also installed with these components.
    4. Engage Microsoft support to ensure that there are no known issues with VSS. Visit http://support.microsoft.com/ to see if there are any patches and updates for VSS.
    5. If errors such as Error: 0x8000FFFF or Event ID 12302 persist, see the Microsoft Knowledge Base article 940184.
    6. Ensure that the time is accurate in the guest operating system and validate that it is being synchronized via NTP or VMware Tools. Please see Timekeeping in VMware Virtual Machines for more information.

Notes:

  • In Windows 2000, XP, and 2003, you can reinstall VMware Tools in custom mode and choose not to install VSS. This results in the SYNC driver being used.
  • The Distributed Transaction Coordinator service must be running while installing VMware Tools. Otherwise, VSS fails to quiesce Windows 2008 R2.

written by Bosse

Sep 23

Amount of data backed up increases significantly after VM disk resizing

VE Comments Off on Amount of data backed up increases significantly after VM disk resizing

Problem(Abstract)

After a disk of a VM guest is resized, the amount of data backed up by a full VM incremental of the guest machine increases significantly.

Symptom

Example of change follows.

Before: Disk size = 1.17TB Amount of data backed up by full vm incremental backup = 12GB
After: Disk size = 1.46TB Amount of data backed up by full vm incremental backup = 363GB

Cause

Change block tracking corrupted.

 

Resolving the problem

Change Block Tracking needs to be reset on the VMWare guest machine. Follow the steps below.

 

  1. Shutdown the VMware guest.
  2. Disable change block tracking for the guest
  3. Start the guest
  4. Shutdown the guest again
  5. Enable change block tracking
  6. Start the guest.

Refer to VMware article    1031873 for more information on enabling Changed Block Tracking (CBT) on virtual machines.

written by Bosse

Sep 23

Expiration of IFFull and IFincremental VM backups

VE Comments Off on Expiration of IFFull and IFincremental VM backups

The first incremental-forever-incremental backup of a VM guest will be an an incremental-forever-full backup. The “query vm” command will shows the type as IFFULL. For example :

dsmc q vm my-test-vm -ina ...  #      Backup Date       Mgmt Class  Size         Type   A/I Virtual Machine ---  -------------------  ----------  -----------  ------ --- ---------------   1  08/01/2013 11:13:10  VEMGMT         11.71 GB  IFFULL  A  MY-TEST-VM
Subsequent incremental-forever-incremental backups will then be incremental. The “query vm” command will shows the type as IFINCR. For example :
dsmc q vm my-test-vm -ina ...  #      Backup Date       Mgmt Class  Size         Type   A/I Virtual Machine ---  -------------------  ----------  -----------  ------ --- ---------------   1  08/01/2013 11:13:10  VEMGMT         11.71 GB  IFFULL  I  MY-TEST-VM   2  08/02/2013 11:57:13  VEMGMT         11.71 GB  IFINCR  A  MY-TEST-VM
Note that the IFFULL backup becomes inactive (I) and the IFINCR backup is then the active version (A). In a case where VEMGMT managament class is configured with verexists=5, each subsequent IFINCR backup inactivates the previous one. After the 5th backup, the “query vm” will show one IFFULL backup and 4 IFINCR backups. For example :
dsmc q vm my-test-vm -ina ...  #      Backup Date       Mgmt Class  Size         Type   A/I Virtual Machine ---  -------------------  ----------  -----------  ------ --- ---------------   1  08/01/2013 11:13:10  VEMGMT         11.71 GB  IFFULL  I  MY-TEST-VM   2  08/02/2013 11:57:13  VEMGMT         11.71 GB  IFINCR  I  MY-TEST-VM   3  08/03/2013 14:17:18  VEMGMT         11.71 GB  IFINCR  I  MY-TEST-VM   4  08/04/2013 11:26:31  VEMGMT         11.71 GB  IFINCR  I  MY-TEST-VM   5  08/05/2013 11:32:26  VEMGMT         11.71 GB  IFINCR  A  MY-TEST-VM
Note that the last IFINCR backup is now the active version and all previous IFFULL and IFINCR backups are inactive versions. Continuing with this example, and still using the VEMGMT managament class configured with verexists=5, any subsequent backup will expire the oldest backup as only 5 versions should be kept. For example, if another IFINCR backup is done, the IFFULL (oldest backup) is expired. After this backup, the “query vm” will show 5 IFINCR backups. For example :
dsmc q vm my-test-vm -ina ...  #      Backup Date       Mgmt Class  Size         Type   A/I Virtual Machine ---  -------------------  ----------  -----------  ------ --- ---------------   1  08/02/2013 11:57:13  VEMGMT         11.71 GB  IFINCR  I  MY-TEST-VM   2  08/03/2013 14:17:18  VEMGMT         11.71 GB  IFINCR  I  MY-TEST-VM   3  08/04/2013 11:26:31  VEMGMT         11.71 GB  IFINCR  I  MY-TEST-VM   4  08/05/2013 11:32:26  VEMGMT         11.71 GB  IFINCR  I  MY-TEST-VM   5  08/06/2013 11:15:47  VEMGMT         11.71 GB  IFINCR  A  MY-TEST-VM
Notes :

  • Despite the IFFULL backup being expired, it is still possible to restore the VM guest using the IFINCR backups due to the nature of the Incremental forever backup strategy.
  • The space occupied by the VM backups in an active-data storage pool will be equivalent to the amount of in-use blocks on the Virtual Machine, even when the IFFULL backup is inactive or expired.
  • If an activedata storage pool is used and backups of virtual machines are stored in the activedata storage pool, adequate space to the active-data pool should be allocated. The space allocation can be estimated based on the size of the original IFFULL backup.

Related information

Query amount of data backup up by IFINCR Query amount of data backup up by IFFULL

written by Bosse

Jan 21

Steps to unregister and register TSMVEplugin to vCenter Server

VE Comments Off on Steps to unregister and register TSMVEplugin to vCenter Server

Question

Unable to connect to TDP for VMware plugin from vCenter Server after IP address was changed of the server TDP VEplugin is installed. >>> The following error occurred while downloading the script plugin from http://(old IP address of TDPVE plugin server):9080/TsmVMwareUI/plugin/config.xml: Unable to connect to the remote server <<<
How can I re-register new IP address to vCenter Server?

Answer

1. From the machine where the TSM for VE plugin is installed, click Start > Control Panel > Administrative Tools > Services.

2. Right-click IBM WebSphere Application Server V7.0 – TSMVEplugin and click Stop.
3. Close the vCenter client
4. From the machine where the TSM for VE plugin is installed, in the command line go to the following directory: C:\Program Files (x86)\Common Files\Tivoli\TDPVMware\VmwarePlugin
5. Run the unregister_vcenter.cmd using the following syntax:
unregister_vcenter.cmd <vcenter hostname> <vcenter user ID> <vcenter user pw> <old IP address>
6. Run the register_vcenter.cmd: register_vcenter.cmd <vcenter hostname> <vcenter user ID> <vcenter user pw> 9080
7. Restart the eWas service.
8. Open the vCenter client, check to see if the plugin can be opened.

written by Bosse