Difference between revisions of "Lab Manager"

From vwiki
Jump to navigation Jump to search
(→‎SOAP API: Added "Configuration IP's Lister" link)
(→‎Troubleshooting: Added "Unable to Export Configuration")
 
(7 intermediate revisions by the same user not shown)
Line 49: Line 49:
# Go to '''Manage | Resources | Physical Networks''' and add network bindings for each of the ESX's
# Go to '''Manage | Resources | Physical Networks''' and add network bindings for each of the ESX's
# Go to '''Manage | Resources | Hosts''' and '''Enable''' the hosts
# Go to '''Manage | Resources | Hosts''' and '''Enable''' the hosts
== Put Spanned ESX Host into Maintenance Mode ==
You can't put a Lab Manager enabled ESX directly into maintenance mode, if it's also enabled for host spanning.  The service VM (called something like <code>012345-VMwareLM-ServiceVM-I6-H78</code>) won't power-down or undeploy automatically (and it can't be migrated via vMotion as it needs to stay on its ESX to enable host-spanning).
# Deployed but powered off LM VM's can't be moved either, so any on the host have to be either powered on, or undeployed
# With DRS set to '''Fully Automated''', select '''Enter Maintenance Mode''' for your ESX host
# Once all deployed (non service) VM's have migrated off, put DRS to '''Partially Automated''' and '''''cancel'''''' the ''Enter Maintenance Mode'' task
# Then '''Disable''' the host in Lab Manager, and unselect the '''Enable host for Host Spanning''' to undeploy the service VM
#* In Lab Manager, go to Manage > Resources > Hosts tab to Disable, and go into the ESX's Properties to disable host spanning
# Finally, put the host in '''Maintenance Mode'''
Once your work is complete, take the host out of maintenance mode, enable for host spanning and as LM host, then put DRS back to Fully Automated.  Its worth manually vMotioning a Lab Manager deployed VM onto the host to double check everything is back in full working order.


== Change a Deployed VM's Resources ==
== Change a Deployed VM's Resources ==
In theory a deployed VM's config shouldn't be changed, in reality its bound to be required at some point as a one off, but its quite limited on what can be acheived.   
In theory a deployed VM's config shouldn't be changed, in reality its bound to be required at some point as a one off, but its quite limited on what can be achieved.   


Any changes will be lost when the template is undeployed and its state not saved.  If this isn't wanted, or if its a regular occurrence, the VM templates should be edited (CPU and memory can be edited with Lab Manager, the VM doesn't need to be re-imported).
Any changes will be lost when the template is undeployed and its state not saved.  If this isn't wanted, or if its a regular occurrence, the VM templates should be edited (CPU and memory can be edited with Lab Manager, the VM doesn't need to be re-imported).
Line 90: Line 102:


== SOAP API ==
== SOAP API ==
As of 2010, there is no [[Power_Shell|PowerShell]] or [[PowerCLI]] interface into Lab Manager, and there is unlikely to be one now that Lab Manager is going end of life.  There is an underlying SOAP interface (on which a PowerCLI interface would be built), which provides limited interaction.  This can be utilised via PowerShell scripts.
As of 2010, there is no [[:Category:PowerShell|PowerShell]] or [[:Category:PowerCLI|PowerCLI]] interface into Lab Manager, and there is unlikely to be one now that Lab Manager is going end of life.  There is an underlying SOAP interface (on which a PowerCLI interface would be built), which provides limited interaction.  This can be utilised via PowerShell scripts.


The functions below were taken from http://poshcode.org/753, note that whilst you can connect to the Lab manager web service without explicitly supplying a user/pass (assuming the logged in user has access), you do need credentials in order to be able to create the Authentication Header that's required for all API calls.  I tend to store a local copy of my user/pass (see [[Power_Shell#Store_Password_Securely|Store Password Securely]]) to make this less of a chore.
The functions below were taken from http://poshcode.org/753, note that whilst you can connect to the Lab manager web service without explicitly supplying a user/pass (assuming the logged in user has access), you do need credentials in order to be able to create the Authentication Header that's required for all API calls.  I tend to store a local copy of my user/pass (see [[Power_Shell#Store_Password_Securely|Store Password Securely]]) to make this less of a chore.
Line 156: Line 168:


=== Configuration IP's Lister ===
=== Configuration IP's Lister ===
* '''[[Lab Manager IP List]]'''
* '''[[Lab Manager Configuration IP List]]'''
The above script allows you to generate a email showing the external (NATed) addresses of a configuration.
The above script allows you to generate a email showing the external (NATed) addresses of a configuration.
=== Configuration IP's Lister and RDP Tester ===
* '''[[Lab Manager Config IP List and RDP Test]]'''
The above script is really a version 2 of [[#Configuration_IP.27s_Lister|Configuration IP's Lister]] which still allows you to generate a email showing the external (NATed) addresses of a configuration, but also allows you to test RDP access to all your VM's within the config.


== Troubleshooting ==
== Troubleshooting ==
Line 211: Line 227:


If Lab Manager keeps dropping the connection to the ESX, check activity in the Virtual Centre.  If an ESX is continually being instructed to complete tasks that fail you need to consider restarting that ESX.
If Lab Manager keeps dropping the connection to the ESX, check activity in the Virtual Centre.  If an ESX is continually being instructed to complete tasks that fail you need to consider restarting that ESX.
=== Unable to Export Configuration ===
Export of a configuration to full fat VM's fails...
* Lab Manager shows...
** '''VirtualMachine.clone task on CloneVM_Task failed'''...
* vCenter also shows...
** '''Incompatible device backing specified for device 'x'.''' for the affected VM's Clone virtual machine task
Suspect the problem is caused by a problem with the VM's config file (VMX) - to work around, create a clone of the configuration and then export from that clone.


=== Unable to Enable Datastore ===
=== Unable to Enable Datastore ===
Line 230: Line 255:


[[Category:VMware]]
[[Category:VMware]]
[[Category:Lab Manager]]

Latest revision as of 09:30, 19 July 2012

Import Machine into Lab Manager

Prerequisites

  • Source Virtual Machine
    • Will be cloned (or moved) into Lab Manager, can be a template or a VM (but must be powered off)
  • Destination info
    • Hostname
    • Description (optional)
    • Network (an existing Physical [lab] Network)
    • Datastore

Procedure

  1. Within Lab Manager, select the correct Organisation
  2. Go to Build and Deploy > VM Templates, and hit the Import VM Template... button (if the button doesn't exist then your account doesn't have sufficient rights)
  3. Enter the new Template's Name and Description, then locate the source Virtual Machine and hit Next
  4. Select the appropriate Network and Datastore, leave Make a Copy option selected unless you want the source VM to be lost/moved, then hit Import
  5. Once the import has completed, check/change the following options
    • Owner
    • Published

Export a Machine from Lab Manager

Note that if you can't satisfy the Prerequisites then you can always use VMware Converter...

  1. Unpublish the VM template
  2. Deploy a single VM from a template (powered off)
  3. Use Converter to create a clone in a new location
  4. Publish the VM tenplate

Prerequisites

  • Your source ESX's need to have SMB access to the UNC path your going to copy to, from an ESX test the following...
    1. SMB allowed through ESX firewall
      • esxcfg-firewall -q smbClient
    2. ESX can access server via SMB
      • smbmount <share> <mountpoint> -o username=<username>
      • EG smbmount //Dest_Win_Server/D$ /tmp -o username=Administrator
    3. If successful, remove the mount
      • umount //Dest_Win_Server/D$ /tmp
  • Lab Manager server must not have MS KB 956572 installed

Procedure

  1. Locate the machine within the VM Template screen
  2. Right-click over the VM and select Export...
  3. Enter a UNC path
    • Must be accessible via SMB from the ESX's
    • Must be a sub-folder, not a root
  4. Enter user/pass with Write rights on UNC share

Add ESX to Lab Manager

  1. Set up a new ESX as normal then within Lab Manager
  2. Go to Manage | Resources | Hosts and Prepare the hosts to install the Lab Manager agent on the ESX's
  3. Go to Manage | Resources | Physical Networks and add network bindings for each of the ESX's
  4. Go to Manage | Resources | Hosts and Enable the hosts

Put Spanned ESX Host into Maintenance Mode

You can't put a Lab Manager enabled ESX directly into maintenance mode, if it's also enabled for host spanning. The service VM (called something like 012345-VMwareLM-ServiceVM-I6-H78) won't power-down or undeploy automatically (and it can't be migrated via vMotion as it needs to stay on its ESX to enable host-spanning).

  1. Deployed but powered off LM VM's can't be moved either, so any on the host have to be either powered on, or undeployed
  2. With DRS set to Fully Automated, select Enter Maintenance Mode for your ESX host
  3. Once all deployed (non service) VM's have migrated off, put DRS to Partially Automated and cancel' the Enter Maintenance Mode task
  4. Then Disable the host in Lab Manager, and unselect the Enable host for Host Spanning to undeploy the service VM
    • In Lab Manager, go to Manage > Resources > Hosts tab to Disable, and go into the ESX's Properties to disable host spanning
  5. Finally, put the host in Maintenance Mode

Once your work is complete, take the host out of maintenance mode, enable for host spanning and as LM host, then put DRS back to Fully Automated. Its worth manually vMotioning a Lab Manager deployed VM onto the host to double check everything is back in full working order.

Change a Deployed VM's Resources

In theory a deployed VM's config shouldn't be changed, in reality its bound to be required at some point as a one off, but its quite limited on what can be achieved.

Any changes will be lost when the template is undeployed and its state not saved. If this isn't wanted, or if its a regular occurrence, the VM templates should be edited (CPU and memory can be edited with Lab Manager, the VM doesn't need to be re-imported).

  • CPU - Update the deployed VM's config through vCentre
  • Memory - Update the deployed VM's config through vCentre
  • Network - Update the deployed VM's config through vCentre
  • Disk - Can't be changed easily, a deployed template is a delta from the original template (think snapshotting), you must reconfigure the template. Additional disks can be added to an undeployed machine

Increasing Disk Size

  1. In the config, shutdown the VM
  2. Once shutdown, select Add to VM templates...
  3. Select Export... to copy the VM to a normal VM, then delete it
  4. Locate the VM in vCentre and modify as required
  5. Boot the VM up, extend the disk, and shutdown
  6. In Lab Manager, import the VM, and publish
  7. In the config, delete the VM, and then add the modified template in

Move to a new Datastore

If you need to move off an old datastore, you need to get your templates (and so also the machines deployed from them) off it. Therefore to do it in a nice, graceful (and gradual fashion). Templates would need to be re-provisioned on new storage, and workspaces re-deployed using the new templates, which is likely to take time. In order to get off old storage quickly (because its unstable maybe) you can...

  • Migrate templates and their deployed child VM's using VMware SSMove tool - VM KB 1006694 (not successfully tested)
  • Clone deployed child VM's to new storage (you take a full fat copy, through the Lab Manager interface)

To clone child VM's to new storage

  1. Action Undeploy - Save State on the entire workspace
  2. Once completed, go to Clone...
    • In Clone to:, select Existing Configuration and select the workspace you're currently in
    • In Create:, select Full Clones...
    • In Clone:, select Selected Virtual Machines:, tick the VM's you want to clone and select the destination Datastore
    • Click OK
  3. Once completed, go into the workspace, and rename VM's so that...
    • Source VM's are given new names
    • Clone VM's are given names of the machines they were cloned from
  4. For the entire workspace, select Move...
  5. Move all the old source VM's into a new (temporary) workspace
  6. Redeploy the original workspace
  7. Assuming the original workspace spins up OK, the new temporary workspace can be deleted


SOAP API

As of 2010, there is no PowerShell or PowerCLI interface into Lab Manager, and there is unlikely to be one now that Lab Manager is going end of life. There is an underlying SOAP interface (on which a PowerCLI interface would be built), which provides limited interaction. This can be utilised via PowerShell scripts.

The functions below were taken from http://poshcode.org/753, note that whilst you can connect to the Lab manager web service without explicitly supplying a user/pass (assuming the logged in user has access), you do need credentials in order to be able to create the Authentication Header that's required for all API calls. I tend to store a local copy of my user/pass (see Store Password Securely) to make this less of a chore.

function Ignore-SslErrors {
	# Create a compilation environment
	$Provider=New-Object Microsoft.CSharp.CSharpCodeProvider
	$Compiler=$Provider.CreateCompiler()
	$Params=New-Object System.CodeDom.Compiler.CompilerParameters
	$Params.GenerateExecutable=$False
	$Params.GenerateInMemory=$True
	$Params.IncludeDebugInformation=$False
	$Params.ReferencedAssemblies.Add("System.DLL") > $null
	$TASource=@'
	  namespace Local.ToolkitExtensions.Net.CertificatePolicy {
	    public class TrustAll : System.Net.ICertificatePolicy {
	      public TrustAll() { 
	      }
	      public bool CheckValidationResult(System.Net.ServicePoint sp,
	        System.Security.Cryptography.X509Certificates.X509Certificate cert, 
	        System.Net.WebRequest req, int problem) {
	        return true;
	      }
	    }
	  }
'@ 
	$TAResults=$Provider.CompileAssemblyFromSource($Params,$TASource)
	$TAAssembly=$TAResults.CompiledAssembly

	## We now create an instance of the TrustAll and attach it to the ServicePointManager
	$TrustAll=$TAAssembly.CreateInstance("Local.ToolkitExtensions.Net.CertificatePolicy.TrustAll")
	[System.Net.ServicePointManager]::CertificatePolicy=$TrustAll
}

function New-ObjectFromProxy {
	param($proxy, $proxyAttributeName, $typeName)

	# Locate the assembly for $proxy
	$attribute = $proxy | gm | where { $_.Name -eq $proxyAttributeName }
	$str = "`$assembly = [" + $attribute.TypeName + "].assembly"
	invoke-expression $str

	# Instantiate an AuthenticationHeaderValue object.
	$type = $assembly.getTypes() | where { $_.Name -eq $typeName }
	return $assembly.CreateInstance($type)
}

function Connect-LabManager {
	param($server, $credential)

	# Log in to Lab Manager's web service.
	$server = "https://" + $server + "/"
	$endpoint = $server + "LabManager/SOAP/LabManager.asmx"
	$proxy = new-webserviceproxy -uri $endpoint -cred $credential

	# Before continuing we need to add an Authentication Header to $proxy.
	$authHeader = New-ObjectFromProxy -proxy $proxy -proxyAttributeName "AuthenticationHeaderValue" -typeName "AuthenticationHeader"
	$authHeader.username = $credential.GetNetworkCredential().UserName
	$authHeader.password = $credential.GetNetworkCredential().Password
	$proxy.AuthenticationHeaderValue = $authHeader
	return $proxy
}

Configuration IP's Lister

The above script allows you to generate a email showing the external (NATed) addresses of a configuration.

Configuration IP's Lister and RDP Tester

The above script is really a version 2 of Configuration IP's Lister which still allows you to generate a email showing the external (NATed) addresses of a configuration, but also allows you to test RDP access to all your VM's within the config.

Troubleshooting

Unable to Deploy Virtual Machine

External network PortGroup is not deployed on host "ESX"
Caused by a Physical Network not being correctly configured on the ESX property

  1. Go to Manage > Resources
  2. Locate the problem network and go to Properties
  3. Update the Physical Network Bindings for the relevant ESX


Unable to deploy VM VM in VirtualCenter. The session is not authenticated.
OR Failed to collect requirements for virtual machine VM. The session is not authenticated.
Caused by logon from Lab Manager to Virtual Centre being unable to authenticate, assuming the user/pass on the VC hasn't been changed, this can be corrected by...

  1. Go to System > Settings
  2. Go to the VirtualCentre tab
  3. Click on OK to re-apply the user/pass


ESX does not support virtual machine hardware version "0"
Caused by config data corruption, potentially from the machine's VMX file

  1. Locate the VM's config and confirm the hardware version is set correctly (eg to v7 for ESX4)
    • virtualHW.version = "7"
  2. Go into the SQL database on the Lab Manager server, perform the following queries to confirm the VM's affected and correct
    • SELECT dir_id, name, hw_version FROM VirtualServer WHERE hw_version=0
    • UPDATE VirtualServer SET hw_version=7 WHERE hw_version=0
    • UPDATE VirtualServer SET hw_version=<correctNumber> WHERE hw_version=0 AND name='VirtualMachineName'
  3. The VM then needs to be cloned to template and then redeployed (clone required to prevent reoccurrance)


vCenter Lab Manager timed out trying to read the virtual machine's file on the remote host
Caused by either of...

  • The ESX experiencing problems reading the files of the VM (and its parents) on VMFS storage
    1. Check out VMFS storage!
  • The VM is still partially running on an ESX (so its files are locked)
    1. Run the following command on the likely ESX(s) to running VM processes
      • ps auxwww | grep -i vmx
    2. Kill the relevant processes, eg
      • 7135117 7179577 mks:001876-VM-Server-A /bin/vmx
      • 7180174 7179577 vcpu-0:001876-VM-Server-A /bin/vmx
      • 7180175 7179577 vcpu-1:001876-VM-Server-A1 /bin/vmx
      • Run kill 7179577

Lab Manager Disconnected from ESX

Check/restart the Lab Manager agent service that runs on the ESX...

  • ESX3
    • service vsla-agent status
    • service vsla-agent restart
  • ESX4i
    • /etc/opt/init.d/vslad status
    • /etc/opt/init.d/vslad restart

If Lab Manager keeps dropping the connection to the ESX, check activity in the Virtual Centre. If an ESX is continually being instructed to complete tasks that fail you need to consider restarting that ESX.

Unable to Export Configuration

Export of a configuration to full fat VM's fails...

  • Lab Manager shows...
    • VirtualMachine.clone task on CloneVM_Task failed...
  • vCenter also shows...
    • Incompatible device backing specified for device 'x'. for the affected VM's Clone virtual machine task

Suspect the problem is caused by a problem with the VM's config file (VMX) - to work around, create a clone of the configuration and then export from that clone.

Unable to Enable Datastore

This datastore has missing or invalid directories

The Lab Manager server knows which folders to expect under its own folder within a given datastore. If what's recorded in its database doesn't match the file system, this error is thrown.

  1. Confirm which the (un)expected folders by looking at error message detail
  2. Create new (dummy/empty) folders for anything that's expected
  3. Delete/move folders that shouldn't be there, to confirm of they're safe to delete...
    1. Confirm the VM isn't visible via Virtual Centre
    2. Confirm the VM doesn't exist in the Lab Manager database
      1. From a command prompt open the SQL command line to get into the db
        • LM3: osql -S localhost\vlm -E
        • LM4: osql -S localhost\labmanager -E
    • To move a VM folder up a level use mv 1234 ../

INCOMPLETE !!