VMware, knife-vsphere, rbvmomi and NicSettingMismatch

The error is pretty obvious … we need to select the correct pool. I’m pretty sure, this example will run if you have only 1 ESX Server or 1 ESX Cluster, because then is 1 single pool defined (by default), Having multiple datacenters / clusters the application is not able to choose a pool, mainly because I don’t have permissions to choose my datacenter directly.

To get pools from my ESX Clusters I run:

$knife vsphere pool list
Folder:
ClusterComputeResource: PROD
ClusterComputeResource: STG
ClusterComputeResource: TEST


Now we need to modify our code to add pool location. It’s not very obvious to add pool location if you are not connected to the correct datacenter, so we need to search after it. Next example show how to search after the correct pool in all clusters and clone.

#
# 
# Simple example to clone a machine from an template with spec created from pool
# Author:: Adrian Stanila
#
#
require 'rbvmomi'
 
#Code from knife-vsphere BaseVsphereCommand.rb, modified for current needs. 
#Code also can be simplified to look only for poolName directly, but this is not the point in this example
def find_pool(vim,poolName)
	dcname = nil
	dc = vim.serviceInstance.find_datacenter(dcname) or abort "datacenter not found"
	baseEntity = dc.hostFolder
	entityArray = poolName.split('/')
	entityArray.each do |entityArrItem|
		if entityArrItem != ''
			if baseEntity.is_a? RbVmomi::VIM::Folder
				baseEntity = baseEntity.childEntity.find { |f| f.name == entityArrItem } or abort "no such pool #{poolName} while looking for #{entityArrItem}"
			elsif baseEntity.is_a? RbVmomi::VIM::ClusterComputeResource
				baseEntity = baseEntity.resourcePool.resourcePool.find { |f| f.name == entityArrItem } or abort "no such pool #{poolName} while looking for #{entityArrItem}"
			elsif baseEntity.is_a? RbVmomi::VIM::ResourcePool
				baseEntity = baseEntity.resourcePool.find { |f| f.name == entityArrItem } or abort "no such pool #{poolName} while looking for #{entityArrItem}"
			else
				abort "Unexpected Object type encountered #{baseEntity.type} while finding resourcePool"
			end
		end
	end
 
        baseEntity = baseEntity.resourcePool if not baseEntity.is_a?(RbVmomi::VIM::ResourcePool) and baseEntity.respond_to?(:resourcePool)
        baseEntity
end
 
 
vim = RbVmomi::VIM.connect host: 'hostname',user: 'user', password: 'password', insecure: true
vm=vim.serviceInstance.find_datacenter.find_vm("Folder/tmpl-debian-6-64") or abort ("VM Not Found!")
 
relocateSpec = RbVmomi::VIM.VirtualMachineRelocateSpec(:pool => find_pool(vim,'TEST'))
spec = RbVmomi::VIM.VirtualMachineCloneSpec(:location => relocateSpec, :powerOn => false, :template => false)
 
task = vm.CloneVM_Task(:folder => vm.parent, :name => "test-clone", :spec => spec)
print "Cloning ..."
task.wait_for_completion

And success! I create a machine from a template. Now is time to see how can we do that on knife-vsphere. The main problems appear to be in generate_clone_spec. I hacked a little bit on vsphere_vm_clone.rb and succeeded to clone a machine. In fact I skipped entire generate_clone_spec function and create a simple function based VirtualMachineRelocateSpec and find_pool.

I was pretty sure something from generate_clone_spec function is creating entire “problem”, but I didn’t want to skip entire generate_clone_spec function, because there you also have machine specification settings (CPU, RAM, VLAN,) and this are very useful. Looking further in source code after nicSettings, I realized what vmware machine expect from us: nicSettingsMap. Ok I added cips and cgw parameters to my command to see what happen.

$ knife vsphere vm clone test-clone --template tmpl-debian-6-64 -f Folder --datastore ESX1-TF-8 --dest-folder Int\ Test/ --cips 10.11.0.187 --cgw 10.11.0.1
Cloning template tmpl-debian-6-64 to new VM test-clone
ERROR: RbVmomi::Fault: UncustomizableGuest: Customization of the guest operating system 'debian6_64Guest' is not supported in this configuration. Microsoft Vista (TM) and Linux guests with Logical Volume Manager are supported only for recent ESX host and VMware Tools versions. Refer to vCenter documentation for supported configurations.


OK, I said. One step closer. Let’s install the latest VMware Tools. I already had open-vm-tools installed. I uninstalled open-vm-tools and I tried to install VMWare-Tools-8.6.5 (which came with ESX). Bad luck. vmtoolsd is crashing with Segmentation Fault. Let’s install open-vm-tools the newest version … no luck. It needs glib >= 2.6 and my systems come with 2.24.

So we need to skip identity customization settings (OS Settings like ip, gw, hostname, domainname) to fix the problem. I just comment out line 289 from vsphere_vm_clone.rb (knife-vsphere package)

clone_spec.customization = cust_spec

The guys who create knife-vsphere have this idea to add hostname and domain name for each new cloned machine, so when it boots, at least you will have the domain and hostname changed. But this doesn’t work if you have a combination of old vmware tools, LVM, and not a very new OS.

Next step was to create a patch to add –no-customization option and skip the section with default customization for new machine. Unlucky me, again:

mixlib-cli was recently changed to allow for options using the –[no]-whatever format provided by ruby’s OptionParser.

http://tickets.opscode.com/browse/CHEF-2641

And that means my option is always false … so I rewrite –no-customization in –disable-customization and everything is ok now.

The patch is here: https://github.com/sacx/patches and the files with rbvmomi tests are here: https://github.com/sacx/rbvmomi-test

If you think this article worth it, you can buy me a beer in return.