VMware, knife-vsphere, rbvmomi and NicSettingMismatch
I finished my Chef Recipes and obviously next step is to create and bootstrap VMware virtual machines directly from scripts. I started to test 2 knife plugins knife-esx and knife-vsphere. Quickly I dropped knife-esx solution, because was based on ssh access to the ESX host and we, “the common people”, don’t have rights on that machine. A little bit about my ESX environment: Is an environment with multiple datacenters, multiple clusters, hundreds of virtual machines. On this environment I don’t have full access and requesting one, it takes weeks/months (a lot of bureaucracy).
I read the documentation about knife-vsphere and after 10 minutes I was ready to start experimenting. But very fast I was stopped by an error.
$ knife vsphere vm clone test-clone -d ESX1 --template tmpl-debian-6-64 -f Folder --datastore ESX1-TF-8 --dest-folder Int\ Test/
ERROR: RbVmomi::Fault: NoPermission: Permission to perform this operation was denied.
I found how to solve the problem here: http://lists.opscode.com/sympa/arc/chef/2012-04/msg00268.html
If I comment out the vsphere_dc option in my knife.rb and run knife vsphere vm list again, it *works*, but selects the first datacenter it finds on the vsphere host, which of course is not the datacenter I want.
If I’m using a knife-vsphere with -d or –vsdc I will get this error, so don’t use any datacenter as parameter or option in your knife.rb (delete or comment knife[:vsphere_dc])
Next, I tried again, but this time I get another error:
$ knife vsphere vm clone test-clone --template tmpl-debian-6-64 -f Folder --datastore ESX1-TF-8 --dest-folder Int\ Test/
Cloning template tmpl-debian-6-64 to new VM test-clone
ERROR: RbVmomi::Fault: NicSettingMismatch: fault.NicSettingMismatch.summary
The problem exists knife-vsphere bug tracking system and have 1 workaround:
create a customization spec in Vsphere then specifying it using the –cspec switch
But was my unlucky day, I don’t have access to create my customized spec. I was almost going crazy. I tried to force knife-vsphere to use rbvmomi 1.6.0 (customizing /var/lib/gems/1.9.1/specifications/knife-vsphere-0.3.0.gemspec and removing rbvmomi 1.5.1), but no luck, same errors. I tried to use rvc (Ruby vSphere Console) and everything appears to be ok there.
So I have only 1 choice! Is RUBY time. Let’s learn rbvmomi.
Most simple example is provided by Patrick Debois (DevOPS Days father): https://gist.github.com/jedi4ever/1001978
I create my example based on Patrick testing notes
require 'rbvmomi' vim = RbVmomi::VIM.connect host: 'host',user: 'user', password: 'password', insecure: true vm=vim.serviceInstance.find_datacenter.find_vm("Folder/tmpl-debian-6-64") or abort ("VM Not Found!") VIM = RbVmomi::VIM relocateSpec = RbVmomi::VIM.VirtualMachineRelocateSpec spec = VIM.VirtualMachineCloneSpec(:location => relocateSpec, :powerOn => false, :template => false) task = vm.CloneVM_Task(:folder => vm.parent, :name => "test-clone", :spec => spec) print "Cloning ..." task.wait_for_completion |
Running it I get the following error:
Cloning .../var/lib/gems/1.9.1/gems/rbvmomi-1.5.1/lib/rbvmomi/vim/Task.rb:11:in `wait_for_completion': InvalidArgument: A specified parameter was not correct. (RbVmomi::Fault)
spec.location.pool
from test1.rb:14:in `
The error is pretty obvious … we need to select the correct pool. I’m pretty sure, this example will run if you have only 1 ESX Server or 1 ESX Cluster, because then is 1 single pool defined (by default), Having multiple datacenters / clusters the application is not able to choose a pool, mainly because I don’t have permissions to choose my datacenter directly.
To get pools from my ESX Clusters I run:
$knife vsphere pool list
Folder:
ClusterComputeResource: PROD
ClusterComputeResource: STG
ClusterComputeResource: TEST
Now we need to modify our code to add pool location. It’s not very obvious to add pool location if you are not connected to the correct datacenter, so we need to search after it. Next example show how to search after the correct pool in all clusters and clone.
# # # Simple example to clone a machine from an template with spec created from pool # Author:: Adrian Stanila # # require 'rbvmomi' #Code from knife-vsphere BaseVsphereCommand.rb, modified for current needs. #Code also can be simplified to look only for poolName directly, but this is not the point in this example def find_pool(vim,poolName) dcname = nil dc = vim.serviceInstance.find_datacenter(dcname) or abort "datacenter not found" baseEntity = dc.hostFolder entityArray = poolName.split('/') entityArray.each do |entityArrItem| if entityArrItem != '' if baseEntity.is_a? RbVmomi::VIM::Folder baseEntity = baseEntity.childEntity.find { |f| f.name == entityArrItem } or abort "no such pool #{poolName} while looking for #{entityArrItem}" elsif baseEntity.is_a? RbVmomi::VIM::ClusterComputeResource baseEntity = baseEntity.resourcePool.resourcePool.find { |f| f.name == entityArrItem } or abort "no such pool #{poolName} while looking for #{entityArrItem}" elsif baseEntity.is_a? RbVmomi::VIM::ResourcePool baseEntity = baseEntity.resourcePool.find { |f| f.name == entityArrItem } or abort "no such pool #{poolName} while looking for #{entityArrItem}" else abort "Unexpected Object type encountered #{baseEntity.type} while finding resourcePool" end end end baseEntity = baseEntity.resourcePool if not baseEntity.is_a?(RbVmomi::VIM::ResourcePool) and baseEntity.respond_to?(:resourcePool) baseEntity end vim = RbVmomi::VIM.connect host: 'hostname',user: 'user', password: 'password', insecure: true vm=vim.serviceInstance.find_datacenter.find_vm("Folder/tmpl-debian-6-64") or abort ("VM Not Found!") relocateSpec = RbVmomi::VIM.VirtualMachineRelocateSpec(:pool => find_pool(vim,'TEST')) spec = RbVmomi::VIM.VirtualMachineCloneSpec(:location => relocateSpec, :powerOn => false, :template => false) task = vm.CloneVM_Task(:folder => vm.parent, :name => "test-clone", :spec => spec) print "Cloning ..." task.wait_for_completion |
And success! I create a machine from a template. Now is time to see how can we do that on knife-vsphere. The main problems appear to be in generate_clone_spec. I hacked a little bit on vsphere_vm_clone.rb and succeeded to clone a machine. In fact I skipped entire generate_clone_spec function and create a simple function based VirtualMachineRelocateSpec and find_pool.
I was pretty sure something from generate_clone_spec function is creating entire “problem”, but I didn’t want to skip entire generate_clone_spec function, because there you also have machine specification settings (CPU, RAM, VLAN,) and this are very useful. Looking further in source code after nicSettings, I realized what vmware machine expect from us: nicSettingsMap. Ok I added cips and cgw parameters to my command to see what happen.
$ knife vsphere vm clone test-clone --template tmpl-debian-6-64 -f Folder --datastore ESX1-TF-8 --dest-folder Int\ Test/ --cips 10.11.0.187 --cgw 10.11.0.1
Cloning template tmpl-debian-6-64 to new VM test-clone
ERROR: RbVmomi::Fault: UncustomizableGuest: Customization of the guest operating system 'debian6_64Guest' is not supported in this configuration. Microsoft Vista (TM) and Linux guests with Logical Volume Manager are supported only for recent ESX host and VMware Tools versions. Refer to vCenter documentation for supported configurations.
OK, I said. One step closer. Let’s install the latest VMware Tools. I already had open-vm-tools installed. I uninstalled open-vm-tools and I tried to install VMWare-Tools-8.6.5 (which came with ESX). Bad luck. vmtoolsd is crashing with Segmentation Fault. Let’s install open-vm-tools the newest version … no luck. It needs glib >= 2.6 and my systems come with 2.24.
So we need to skip identity customization settings (OS Settings like ip, gw, hostname, domainname) to fix the problem. I just comment out line 289 from vsphere_vm_clone.rb (knife-vsphere package)
clone_spec.customization = cust_spec |
The guys who create knife-vsphere have this idea to add hostname and domain name for each new cloned machine, so when it boots, at least you will have the domain and hostname changed. But this doesn’t work if you have a combination of old vmware tools, LVM, and not a very new OS.
Next step was to create a patch to add –no-customization option and skip the section with default customization for new machine. Unlucky me, again:
mixlib-cli was recently changed to allow for options using the –[no]-whatever format provided by ruby’s OptionParser.
http://tickets.opscode.com/browse/CHEF-2641
And that means my option is always false … so I rewrite –no-customization in –disable-customization and everything is ok now.
The patch is here: https://github.com/sacx/patches and the files with rbvmomi tests are here: https://github.com/sacx/rbvmomi-test
If you think this article worth it, you can buy me a beer in return.
Hi, I have been trying to use RVC to manage templates.
I have cloned a VM from a template with RVC, but the
limitation is that I cannot specify a Customization to
override the IP address, DNS name etc for Linux deployments.
So, the cloning of the VM works successfully, but I have to
manually intervene to set the DNS and IP details for the Guest. I am not at all skilled with Ruby and am still finding my way around RVC, so I am not sure what to do,
but surely it must be possible to set an IP/DNS etc during
the vm.clone job from within RVC?
Any help, code-snippets etc or just commiserations would
be gratefully appreciated!
rachel
What distro do you have ? If is debian like You can set the distribution on Ubuntu and everything will work well …
Regards
Leave your response!
Ads
Bitcoin/Litecoin/Dogecoin Donations
Send your donations to:
Bitcoin:
1SacxZjTR4NsF2W9TryVx3zDPMzRupsQB
Litecoin:
LXpVCTTsiGoQ3L1Qgr4Kk4wTDx5nqAfpwP
Dogecoin:
DR5W4rWQgGxrhUDaLiYY6vzwLW3NKqJ6kc
Thanks!
Tags
Categories