No more insane clicking in ESXi - setup your testbed in a minute

2013-02-19

Hello all!

Very often, I find myself in the situation where I have to quickly setup a test environment. In my case that usually means that I will quickly setup a:

You might have your own favorite setup, this is mine. The goodies on top of a minimal server installation is:

I've done this so many times by now, that I just cannot bear the thought of doing it again.

Why? Well, doing it by hand is summarised by the following:

I mean, that is 14 steps that I could live without. So, I started looking at ways to do this from a terminal window.

To this story, you need to know that my office environment I've got the following setup:

In my environment, I can browse my datastores when logged into "guran", since all my virtual machines are located in the datastores on NFS. Look here:

malu@kmg-guran-0001:/mnt/synology02/files/vmware/datastores $ls -la
total 24
drwxr-xr-x 6 malu malu 4096 2012-10-02 12:49 .
drwxr-xr-t 9 malu malu 4096 2012-12-05 14:58 ..
drwxr-xr-x 19 malu malu 4096 2013-02-19 20:38 dev
drwxr-xr-x 8 malu malu 4096 2013-02-04 18:42 prd

malu@kmg-guran-0001:/mnt/synology02/files/vmware/datastores $ls -la dev
total 76
drwxr-xr-x 19 malu malu 4096 2013-02-19 20:38 .
drwxr-xr-x 6 malu malu 4096 2012-10-02 12:49 ..
drwxr-xr-x 2 root root 4096 2013-02-04 17:27 jira-v001fry
drwxr-xr-x 2 root root 4096 2013-01-04 11:11 kmg-buildbox-0001
drwxr-xr-x 2 malu malu 4096 2012-11-24 15:12 kmg-op5-0003
drwxr-xr-x 2 root root 4096 2013-01-04 11:10 kmg-op5-0004
drwxr-xr-x 2 root root 4096 2012-12-09 21:56 kmg-sandbox-0005
drwxr-xr-x 2 root root 4096 2012-12-09 19:49 kmg-sandbox-0005.save
drwxr-xr-x 2 root root 4096 2013-01-04 11:11 kmg-web-0001
drwxr-xr-x 2 root root 4096 2013-01-04 11:11 kmg-web-0002
drwxr-xr-x 2 root root 4096 2012-12-28 13:24 kmg-zenLoadbalancer-0001
drwxr-xr-x 2 malu malu 4096 2013-02-07 14:27 nexenta-v001test
drwxr-xr-x 2 root root 4096 2013-01-16 16:24 op5-v001test
drwxr-xr-x 2 root root 4096 2013-01-04 11:10 openstack-v001fry

This is very useful, I must say. The goal for me is to be able to create new virtual machines without even thinking of starting up my Windows machine. I accomplished this by doing the following:

The basic recipe in my environment is:

My template.vmx file looks like this:

.encoding = "UTF-8"
config.version = "8"
virtualHW.version = "8"
pciBridge0.present = "TRUE"
pciBridge4.present = "TRUE"
pciBridge4.virtualDev = "pcieRootPort"
pciBridge4.functions = "8"
pciBridge5.present = "TRUE"
pciBridge5.virtualDev = "pcieRootPort"
pciBridge5.functions = "8"
pciBridge6.present = "TRUE"
pciBridge6.virtualDev = "pcieRootPort"
pciBridge6.functions = "8"
pciBridge7.present = "TRUE"
pciBridge7.virtualDev = "pcieRootPort"
pciBridge7.functions = "8"
vmci0.present = "TRUE"
hpet0.present = "TRUE"
nvram = "XXX_HOST_NAME_XXX.nvram"
virtualHW.productCompatibility = "hosted"
powerType.powerOff = "default"
powerType.powerOn = "hard"
powerType.suspend = "default"
powerType.reset = "default"
displayName = "XXX_HOST_NAME_XXX"
extendedConfigFile = "XXX_HOST_NAME_XXX.vmxf"
floppy0.present = "TRUE"
scsi0.present = "TRUE"
scsi0.sharedBus = "none"
scsi0.virtualDev = "lsilogic"
memsize = "1024"
scsi0:0.present = "TRUE"
scsi0:0.fileName = "XXX_HOST_NAME_XXX.vmdk"
scsi0:0.deviceType = "scsi-hardDisk"
ide1:0.present = "TRUE"
ide1:0.fileName = "/vmfs/volumes/c262ee3b-00d1a1ed/images/kmg-ubuntu-12.04.2.LTS.iso"
ide1:0.deviceType = "cdrom-image"
floppy0.startConnected = "FALSE"
floppy0.fileName = ""
floppy0.clientDevice = "TRUE"
ethernet0.present = "TRUE"
ethernet0.virtualDev = "e1000"
ethernet0.networkName = "VM Network"
ethernet0.addressType = "generated"
chipset.onlineStandby = "FALSE"
guestOS = "ubuntu-64"
uuid.location = "56 4d 5d 9a c5 dc 8e a1-45 76 3d 90 34 83 82 d1"
uuid.bios = "56 4d 5d 9a c5 dc 8e a1-45 76 3d 90 34 83 82 d1"
vc.uuid = "52 75 89 2c 80 59 17 93-b9 0b 33 49 04 8c c8 a3"
snapshot.action = "keep"
sched.cpu.min = "0"
sched.cpu.units = "mhz"
sched.cpu.shares = "normal"
sched.mem.min = "0"
sched.mem.shares = "normal"

Notice all the entries of "XXX_HOST_NAME_XXX"? I picked that string, since it is very unlikely that it is used by VMWare, and it is easy to replace using "sed".

To make things a bit easier, I first setup my ESXi host to accept my public key to login as root:

ssh root@myESXiHost

vi /etc/ssh/keys-root/authorized_keys

After this, the recipe is easy:

newHostName=testbox-v003fry
templateDir=/mnt/synology02/files/vmware/datastores/dev/template
datastoreDir=/mnt/synology02/files/vmware/datastores/dev
cd $datastoreDir
mkdir $newHostName
cat $templateDir/template.vmx | sed -e 's/XXX_HOST_NAME_XXX/'$newHostName'/' > $datastoreDir/$newHostName/$newHostName.vmx
ssh root@192.168.2.204 vmkfstools -c 16g /vmfs/volumes/NFSDev/$newHostName/$newHostName.vmdk -a lsilogic
vmID=$(ssh root@192.168.2.204 vim-cmd solo/registervm /vmfs/volumes/NFSDev/$newHostName/$newHostName.vmx $newHostName pool0)
#-- turn on VM
ssh root@192.168.2.204 vim-cmd vmsvc/power.on $vmID &
sleep 1
#-- check there is a message and choose 2 (default, moved it)
[ -z "`ssh root@192.168.2.204 vim-cmd vmsvc/message $vmID _vmx1 | grep 'No message'`" ] && ssh root@192.168.2.204 vim-cmd vmsvc/message $vmID _vmx1 2

The ampersand (&) is there after the power.on, as the command will hang if there is already (most likely) a VM with the same uuid (defined in the vmx file) in the system. After this we need to sleep for one second, since there will be no message in the message buffer until the ESXi host realizes the there is a conflict, after which I first check if there is a message. If there is a message, pick it up and tell ESXi to use the default (2 - I copied the VM) alternative.

P.S I found out the "pool0" reference to my NFSDev resource pool by browsing this page: http://communities.vmware.com/message/1114467

cat /etc/vmware/hostd/pools.xml | grep "YOUR-RESOURCE-POOL-NAME" -A1 | grep "" | sed 's///;s/</objID>//g' | sed -e 's/^[[:blank:]]*//;s/[[:blank:]]*$//'

Done. All for today.