Virtualization, Cloud, Infrastructure and all that stuff in-between

My ramblings on the stuff that holds it all together

Cohesity at vRetreat

Last month I had the luck to be invited to the vRetreat event put together by Patrick Redknap, this was a day to get some bloggers face to face with some presenters looking at some cool, new tech.

In the interests of disclosure I should point out that there was a Porsche track day, food and a hotel involved. On the day we were very privileged to have 1:1 instruction around the purpose built track at Silverstone race course (you can pay a visit for this yourself There is no directive on what content I have to write or pressure to write something positive. Evidence of this is that it was over a month ago and work commitments have meant I’ve not had chance to write a blog post about the event until now, Patrick was very cool about it.

This delay did, however give me some time to give some serious thought to the info we received during the day, Veeam, Zerto and Cohesity presented on the day. I’ve worked hands-on with Zerto and Veeam before but Cohesity were new to me and it piqued my interest – the delay in writing this blog post meant I thought about some very cool real-world use-cases.

As a side note; Zerto also demonstrated something very cool, they have a ready to use appliance in Azure, I’ve been doing a lot of experimentation with Azure recently and whilst they were explaining some of the finer points I managed to deploy it before the end of the session! Now that’s the real power of the cloud! A little disappointing that the Azure ready to run appliance at the time was a Windows VM with some links, but I understand this has was due to some license constraints and the full appliance will be ready to download from Azure soon.

Anyways, back to Cohesity, who bill themselves as a Hyperconverged platform for secondary storage. I have to say I rolled my eyes a little at the mention of “another” hyperconverved platform, it’s secondary storage – everyone de-dupes etc. Jam in more capacity and shove some data on it to forget about it, or let it rot. We spent a lot of time talking about storage functionality that to be honest is plain storage, But I have to say regardless of how their magic is served up software or hardware offering – the last 5mins were the coolest part; it has a very cool plug-in architecture to allow Java apps to run on the appliance itself, just think about that analytics, search, applications themselves running on the storage itself. Now that is the cool bit, and it’s USP as far as I’m concerned, it’s new and still being developed, but bear with me..

A long time ago I worked for a company that did video on demand solutions, about 10 years ago one of the most interesting (but now defunct) vendors I worked with had scale-out storage solution (think cheap, x86 pizza boxes with very clever software way ahead of its time) that could store large amounts of video content but also transcode it to different formats at the same (or near-realtime) something that is very computationally expensive; storage is boring (sorry, but it is..) but if you can make it do something with the data it holds at the same time then to me, it’s very clever.

Cohesity offers this with its plugin and analytics plug-ins, what if you have a compliance use-case and you need to prove to regulators that all your data doesn’t contain credit-card numbers or other Personally Identifiable Information (PII); scanning large volumes of data with an application can only run a) periodically on a schedule, and b) at the speed of NFS/SMB etc. as data has to be read off the array, scanned, and in some cases written back. in this case the storage can do it efficiently on the array itself using all that spare CPU power.

Plug that together with an API on the Cohesity array or via the plug-in application and you not only have a really powerful scale-out storage device, but you have an application with vast amounts of data adjacent and on-tap (sorry, rubbish NetApp joke).

I like to think I maintain an even, unbiased professional line and I guess like me you’ll eye start-up storage vendors with the evil-eye as there is a risk that they go belly-up leaving you with an un-supportable storage headache that you need to replace at massive cost and migrate off, but I think this one has legs and something that nobody else on the market has, and to me that’s a reasonable bet they’ll get snapped up or live on their own.

Anyways, back to the Porsche bit. As a life-long Porsche fan the Porsche Experience is a great day out and you get to do some cool stuff like skidding about and a wet skidpan to test your driving skills (verdict: needs work!). I’ve always been a sceptic of those new fangled J pretend automatic/sort of manual gearboxes in sports cars after a bad investment in a BMW e46 M3 with an SMG gearbox (don’t do it kids). But a couple of hours of proper driving with a PDK gearbox is starting to convert me. My wife said specifically to me when I left the house for this event to not come back wanting to change my car, err..

Also got a ride in Joe Baguley’s Tesla Model X… those things are like rocket ships!

Sad post-note: Several days of very creative man-maths have not yet resulted in me being able to buy one, back to the grind-stone Gallagher.

if you want to get a flavour of what went on, check out this video


London and UK VMUG Dates 2016 and 2017

Dates for your diary for our future events, keep an eye on for details of the agenda – Also feel free to join our LinkedIn group or follow our London VMUG Twitter feed and UKVMUG Feed to be kept up to date, we will post/tweet when the agenda and registration link is live as well as any logistical info.

23 June at TechUK in London – followed by Luxury vBeers at a brewery
November 17th National Motorcycle Museum for the national UKVMUG


19 January TechUK, London
6 April TechUK, London
22 June at TechUK, London
#UKVMUG 16th November at the National Motorcycle Museum, Solihull (Near Birmingham)

We (the committee) pride ourselves on promoting community content, we’re all about the U in VMUG – if you have an idea for a session at one of these events – you can use our handy call for papers form

Look forward to seeing you at a future meeting


How much do VMUG leaders contribute to VMUG globally

I’ve just attended the 1st annual VMware User Group Leader Summit, a day and a half event hosted at the VMware campus in Palo Alto to share best practice amongst the various groups around the world to better the organisation which was an excellent event.

The event was an impressive showing of commitment from VMware to the community in terms of the focus it has, but was more impressive was the scale of effort that the VMUG leaders put into their events – there aren’t many real rewards for being a VMUG leader other than some kudos and a pat on the back and it’s a very clear sign of people’s passion for the technology that they give this time freely.

Some leaders are self-employed or work full-time for an employer – but generally the time they give is their own personal, unpaid time as vacation time or work time that has to be made-up in personal time.

The London VMUG group of which I’m a leader in is currently going through a transition to a new team of leaders (I’m staying on but 3 leaders are stepping down after many years of service) and we’ve spent some time trying to quantify how much effort is required to run a VMUG group so we can set expectations appropriately for our new incoming leaders;

This is based on our experiences running 3 London (~100 attendees) and 1 UK national event (~600 attendees) each year.

Disclaimer this is very finger in the air analysis (and a little bit of fun) – but I do think it’s interesting to look at the opportunity cost of such activities (info on opportunity cost here) and other interesting {honest!} economics stuff here

Between the 4 of us we have 4 full-day meetings, so 4 man-days** contributed per meeting which we attend*, plus on average 2hrs of calls per month = 24hrs = 3 man-days/yr. (@8hrs/day) – so individually each leader contributes 7 man-days per year of effort to manage and run our events.

*I’ve not managed to sit through and enjoy a session at the London or UK VMUG meetings since I became a leader, because there is always something that needs doing, cats to herd, things to organise – not complaining, but – that’s the truth!

Our leadership team consists of 4 people, if we said the average group is 3 leaders (some have 7+, some have just 1!).

I don’t have access to all the details of the global VMUG chapters, but if you work on the basis that there was 1 leader invited from each active VMUG globally to the summit, there were 93 leaders in-attendance so let’s base our numbers on 93 ‘active’ groups – although I appreciate there are probably more as not everyone would be able to attend.

if we said an average of 3 leaders per ‘active’ group, each contributing 7 man-days per annum that’s 1,953 man-days per annum contributed by leaders to the VMUG community events. (3 x (4+3) ) x 93 = 1,953 man-days

Given there is an average of 251 working days per year that’s 7.7 man-years

Now, to make this more interesting, if we said the average salary of a VMware administrator was $80k USD (sort-of based on this article, and assuming that an VMUG leader will generally have more than 2 years of experience under their belt and will generally be in a senior-type role, the majority of VMUG leaders are in the US and salaries outside the US will obviously differ, but most VMUGs exist in well-developed 1st-word countries, rather than 2nd/3rd world emerging countries)

That would mean a VMUG leader globally earns an average of $318 per day before tax, multiply that out by the number of man-days given per year, that represents an opportunity cost that the VMUG leaders contribute to the VMware community & VMware itself of…..(drum-roll)

$622,470.12 USD.

Not too shabby 🙂 VMware, I hope you appreciate it 🙂

Anyways – just a bit of fun and not to be taken too seriously, but do go and hug a VMUG leader at your next meeting… (ok, don’t do that!)



**Yes, there are also many women who are VMUG leaders.. but man-days is an accepted term, and it’s shorter to type than person-years, apologies if it offends, it’s not meant to!


Making your OS X Terminal more useful for DOS refugees

I’m a DOS/Windows old-timer, but have been using a Mac for a number of years.. I find the OS X terminal (which is the *NIX bash shell) very flexible, but needs some tweaks to help me with my embedded DOS muscle memory – this is probably very basic and old-hat for *NIX types, but it’s here for my reference as I keep forgetting when I move to a new Mac.

If you look in your home dir “cd ~” you need to create (or edit, if it already exists) the “.bash_profile” file – you can do this with TextEdit, or use nano (“nano .bash_profile”)

Paste in the following contents


export CLICOLOR=1
export LSCOLORS=GxFxCxDxBxegedabagaced
alias dir=”ls -ahl”


Save the file (CTRL-X, yes, enter) in nano

then type “source .bash_profile” to load the changes (or start a new terminal session)

You now have a more DOS-like prompt with the full path in it, colour coding for different types of files and a “dir” command which shows the contents of the current directory by aliasing the “ls” command and adding some parameters to show it list-like.

Some really helpful references here

Also – in terminal, terminal->preferences/profiles and you can set the “pro” profile as default by hitting the “default” button at the bottom of the pick list – also remember to check “Antialias Text” for sharper text.

POSH1Liner: Find all hosts with less RAM than you expect

If you have a cluster where maybe there are some hosts with spared out RAM due to a fault or a non-standard amount of RAM you can quickly find them with this command

get-vmhost | where {$_.MemoryTotalGB -lt THE_AMOUNT_YOU_EXPECT}

For example; to find all hosts with less than 512GB of RAM

get-vmhost | where {$_.MemoryTotalGB -lt 512}

“-lt” is “less than” which is slightly less intuitive than the usual < <= operators you'd use in other languages – but handy reference here

As I go deeper with PowerShell (POSH) I like convenient things like the $_. syntax – makes it dead simple to come up with useful one-liners like this.

Joining an ESX host to the domain without rebooting

I found recently that despite the KB article saying no reboot is required that my ESX hosts would not authenticate AD users unless they were rebooted.

to work around this you can use the following PowerShell code to restart the relevant services without rebooting.

Get-VMHost $esx | Get-VMHostService | where {$_.Key -eq “lwiod”} | Restart-VMHostService -Confirm:$false
        Get-VMHost $esx | Get-VMHostService | where {$_.Key -eq “netlogond”} | Restart-VMHostService -Confirm:$false
        Get-VMHost $esx | Get-VMHostService | where {$_.Key -eq “lsassd”} | Restart-VMHostService -Confirm:$false
        Get-VMHost $esx | Get-VMHostService | where {$_.Key -eq “lbtd”} | Restart-VMHostService -Confirm:$false

Feel free to reuse the whole script, but do so at your own risk. (Download file (rename to .ps1)

# Simon Gallagher (@vinf_net)
# Script to join all ESX hosts in a vCenter to the domain, adding a specific group into a vSphere advanced setting to add the YOUR_AD_GROUP group to the local ESX admins group on the ESX host

#Version 1.0

function ESXDomainJoin ([STRING]$doVC)
connect-viserver $doVC -credential $vCenterAcct
#connect to vCenter using the credentials we stored earlier

$esxHosts = get-VMHost #list all the hosts in this vCenter, then do something with them

    foreach ($esx in $esxHosts) {

        Write-Host “Doing domain join on $esx” -ForegroundColor Green
        $esxParam = “Config.HostAgent.plugins.hostsvc.esxAdminsGroup” # the advanced setting we want to change to the AD group
        $esxValue = “YOUR_AD_GROUP” #the name of the group we want to add to the setting
        Get-VMHost $esx | Get-AdvancedSetting -Name $esxParam | Set-AdvancedSetting -value $esxValue -Confirm:$false #-WhatIf # set it and don’t ask 1st
        #set DNS domain name (required for domain join)
        Get-VMHostNetwork -VMHost $esx  | Set-VMHostNetwork -DomainName  #-WhatIf
        #join domain using build account
        Get-VMHostAuthentication -VMHost $esx | Set-VMHostAuthentication -domain -user $buildAcct.getNetworkCredential().Username     -password $buildAcct.getNetworkCredential().Password -JoinDomain -Confirm:$false  #-WhatIf
        #Restart services so that the YOUR_AD_GROUP group gets automatically ACLd on local host without a reboot
        # takes 2-5mins to apply from AD after services are restarted, but then you should be able to logon using VI client/SSH to an individual ESX host using your AD creds
        Write-Host “Restarting services on $esx..” -ForegroundColor Green
        Get-VMHost $esx | Get-VMHostService | where {$_.Key -eq “lwiod”} | Restart-VMHostService -Confirm:$false
        Get-VMHost $esx | Get-VMHostService | where {$_.Key -eq “netlogond”} | Restart-VMHostService -Confirm:$false
        Get-VMHost $esx | Get-VMHostService | where {$_.Key -eq “lsassd”} | Restart-VMHostService -Confirm:$false
        Get-VMHost $esx | Get-VMHostService | where {$_.Key -eq “lbtd”} | Restart-VMHostService -Confirm:$false
        write-host “Completed restarting services, domain logon should be available in 5mins on $esx” -ForegroundColor Green
disconnect-viserver * -Force #disconnect from all vCenters to be safe (get-VMhost connects to all vCenters you are connected to)
Write-Host “Done!” -ForegroundColor Green
} #end of function


write-host “Disconnecting from all current vCenter servers, just to be safe” –  -ForegroundColor Green
disconnect-viserver * -Force # disconnect from everything at the start, just to be safe

#build password list to work with
$vCenterAcct = Get-Credential -Message “Please enter credentials for vCenter administrator account”
$buildAcct = Get-Credential -Message “Please enter credentials to join machines to domain”

#now call the function for each vCenter in-turn

Find which SSL certificate is being used on an ESX host

If you have been through the pain of changing ESX host certificates from self-signed to real (CA signed certificates), you can check which certificate you are currently using for vCenter–>ESX host traffic by issuing the following command in an SSH session on the host

openssl x509 -noout -in /etc/vmware/ssl/rui.crt –text

You will see the details inside the SSL cert, signing authority etc. If the SSL cert contents make reference to VMware, you’re still using a self-signed certificate.

The vpxa (vCenter management) service on the ESX host is hard-coded to use /etc/vmware/ssl/rui.crt when the service is started (or restarted) so you can examine its properties using the above command to check
I can’t find any graphical way of checking this in the VI Client.

Cannot logon after setting the esxAdminsGroup advanced parameter

I found this issue recently – working on an environment where the ESX 5.5 hosts (build 1892794) had to be added to the domain. We needed to add a custom domain group in to grant it root access to the ESX host.
We followed this KB article to set the advanced parameter on each host (using PowerCLI) Config.HostAgent.plugins.hostsvc.esxAdminsGroup to be equal to the domain group we want to be granted admin access to the host e.g VMware_Admins so you can use AD credentials for SSH etc without relying on root all the time if you are a member of this group.

However – this did not work – if I connected the VI client directly to the host and logged on as root, I could not see the domain group on the permissions tab, I tried a lot of different things, combinations of DOMAIN\GROUPNAME and just GROUPNAME to no avail – the permission did not apply when I left & re-joined the domain.

It turns out that, in my environment despite the article saying no reboot required a reboot was actually required to enact the change and make it work (thanks to Julian Wood on twitter for sharing his identical experience).

I’ve provided some feedback to the KB article, but if you come across this issue – this is the cause! I did join/leave the domain a number of times with this host whilst testing so maybe this was a factor.

LonVMUG Prizes for community content aka vFACTOR London

As a fellow VMware community member I know how much I have benefited from sharing my experiences with my peers ; since I started doing so – I’ve given numerous presentations around the world on my work projects and my home lab projects I’ve met a lot of interesting people, I wrote a book about a product that VMware discontinued almost as soon as it was published  (ok, maybe gloss-over that last one!)

TL;DR version of this post

Come and present a 10min talk at London VMUG on 22nd Jan 2015 and win Apple shiny good or Amazon vouchers – submit abstract & rules here now

As a member of the London & UK VMUG steering committee I also know how hard it is for VMUGs to find people willing to talk about their experiences with their peers by giving a short talk or hosting a discussion group – so, in order to help encourage (well, ok bribe!) people to get them on their way to community stardom we are announcing EXCELLENT PRIZES for 5 first-time VMUG presenters who are end-users and are willing to deliver just a short 10-minute talk on a VMware related topic of their choosing at the next London VMUG meeting (Jan 22nd in central London)

And when I say FANTASTIC PRIZES – I mean…

1st Prize: MacBook Air
2nd Prize: iPad Air
3rd Prize: iPad Mini
4th Prize: Amazon voucher
5th Prize: Amazon voucher

You can submit your abstract using this online form here

We will pick 5 submissions from the entries to present at the event and the closing date is 19th December 2014 – everyone who presents will win something, and the audience will vote for their favourite to determine what loot you go home with from the above list?

Not convinced…? Read on.

Below are the most common reasons I hear not to do a talk… and are all my own personal opinion..

I’m nervous, I’m no Duncan Laverick, or Mike Dennemann  I can’t do public speaking

We all have been here, we’re a friendly crowd and it’s an excellent way to learn some important life & professional skills. To help with this the community at-wide offers the feed forward initiative, you can read more about this here – it can help you find a mentor to help you prepare for the day – one of our own members – Alex Galbraith did just this – read about his experiences on his blog here

I cant think of anything to talk about

We all have great experience from environments small and massive – I’ve personally found the best way to capitalise on this knowledge is to share it with others, having to distil it down to a presentation or blog post is a great way to help you focus on the important facts of what you did and how you did it – I have a tendency to whitter on (if you’ve read this blog for a while you already know this!) and doing this is an excellent tool, especially if you have to explain it to someone outside of your organisation

I work for a 3 man band company, nobody is interested in what I do

Wrong, I started my VMUGging talking about my 1 node home lab, you don’t have to have a data centre the size of the starship enterprise to have something interesting to say. Infact – I’d say from personal experience people from SMB type organisations have much more limited budgets and have to find far more creative ways to get things done

I work in a nuclear bunker, and I have access to your home address and browser history – I can’t talk about what I do

Now I know some of you maybe do work in nuclear bunkers, 3-letter govt. agencies or centres of commercial finance and are unable to talk about what you do for fear of exposing commercial secrets – I understand this, so do I.. My suggestion is that if you want to submit a talk – anonymise what you do – if you work for Goliath National Bank, just say you work for “a major bank”, or “an airline” – this is a VMUG, not Panorama. VMUG attendees are interested in the tech, what you did, how you did it & what could have gone better; not what impact it had on your Q4 financial results.

If you take this approach and remember to say you are not there as a company spokesperson, this is my own opinion etc. this is usually acceptable.

So, that’s it – submit here and I look forward to giving you your prize on 22nd January 2015.

..And to be honest – what’s the worst that could happen… 10mins of your time for a prize? …and we go to the pub afterwards anyway 🙂

If, even after all my objection handling you really can’t bring yourself to do it; that’s fine 🙂 you’re still welcome to come along and soak up the excellent content – you can register online at

What to see at UKVMUG 2014

The 4th annual UK-wide VMware User Group Conference is in a couple of weeks, on Tuesday 18th November several hundred (last year it was over 500!) of your peers will travel to the National Motorcycle Museum in Solihull.

The venue is just down the road from Birmingham airport, and a short taxi ride from the local train station – or easy to get to by road with free parking.

As part of the committee that puts this event on I’d like to point out some of the highlights of the agenda; as per-every year, we have the usual array of headlining technical super-stars from the virtualization community – Duncan Epping, Frank Denneman, Mike Laverick and Chris Wahl. But, over the years we’ve continued to build-up the community mezzanine area to deliver highly interactive face to face discussion groups and white board sessions – small groups of people talking brain-bendingly deep tech, or just introductions for n00bs – the guys staffing these tables are volunteers from all areas of industry and are the best brains in the biz – take advantage!

These are taking place on the mezzanine level and are noted as side-sessions on the main agenda. The mezzanine is the raised area at the back of the solutions-exchange & will be sign-posted on the day – there are way too many for me to detail on this blog post – 20 sessions in total!!!

    • VMware GSS are running a drop-in clinic – this is your chance to ask the tricky questions of VMware support, they are excellent supporters of VMUG and are sending two of their top level support & escalation engineers to field your questions face to face.
    • Designing real-world workflows for vRealize automation center (vCAC) with Jon Medd – if, like me – you’ve struggled to find practical examples & resources for this sort of project – Jon is your man; he’s done it for real – ask him the tricky questions and he’s an absolute wizz at PowerCLI!
    • Ask the CTO anything; this is your chance to have a face to face discussion with VMware EMEA CTO Joe Baguley – Joe is an avid VMUG supporter and general all-round expert at all things technical – he’s game for any questions – even non-VMware related ones (hint: ask him about fireworks or Robot Wars, or even SDDC) – how often do you get that opportunity! (he is also our keynote speaker).

We start early on the Tuesday, and as such we are repeating the popular vCurry the night before – check the box when you sign up to attend; we’ll be laying on curry and there will be a light-hearted quiz, this was a hoot last year – although we have already decided the guys from VMware GSS are excluded from winning 🙂 – although beware they may set some of the questions :).

Please note the event is on a Tuesday (not the typical Thursday) – but it’s early enough in the week to hopefully get your boss to agree that you can go to what is effectively a mini-VMworld in the UK.

You can see the full agenda online at and you can sign-up at