Virtualization, Cloud, Infrastructure and all that stuff in-between

My ramblings on the stuff that holds it all together

Cannot logon after setting the esxAdminsGroup advanced parameter

I found this issue recently – working on an environment where the ESX 5.5 hosts (build 1892794) had to be added to the domain. We needed to add a custom domain group in to grant it root access to the ESX host.
We followed this KB article to set the advanced parameter on each host (using PowerCLI) Config.HostAgent.plugins.hostsvc.esxAdminsGroup to be equal to the domain group we want to be granted admin access to the host e.g VMware_Admins so you can use AD credentials for SSH etc without relying on root all the time if you are a member of this group.

However – this did not work – if I connected the VI client directly to the host and logged on as root, I could not see the domain group on the permissions tab, I tried a lot of different things, combinations of DOMAIN\GROUPNAME and just GROUPNAME to no avail – the permission did not apply when I left & re-joined the domain.

It turns out that, in my environment despite the article saying no reboot required a reboot was actually required to enact the change and make it work (thanks to Julian Wood on twitter for sharing his identical experience).

I’ve provided some feedback to the KB article, but if you come across this issue – this is the cause! I did join/leave the domain a number of times with this host whilst testing so maybe this was a factor.


LonVMUG Prizes for community content aka vFACTOR London

As a fellow VMware community member I know how much I have benefited from sharing my experiences with my peers ; since I started doing so – I’ve given numerous presentations around the world on my work projects and my home lab projects I’ve met a lot of interesting people, I wrote a book about a product that VMware discontinued almost as soon as it was published  (ok, maybe gloss-over that last one!)

TL;DR version of this post

Come and present a 10min talk at London VMUG on 22nd Jan 2015 and win Apple shiny good or Amazon vouchers – submit abstract & rules here now

As a member of the London & UK VMUG steering committee I also know how hard it is for VMUGs to find people willing to talk about their experiences with their peers by giving a short talk or hosting a discussion group – so, in order to help encourage (well, ok bribe!) people to get them on their way to community stardom we are announcing EXCELLENT PRIZES for 5 first-time VMUG presenters who are end-users and are willing to deliver just a short 10-minute talk on a VMware related topic of their choosing at the next London VMUG meeting (Jan 22nd in central London)

And when I say FANTASTIC PRIZES – I mean…

1st Prize: MacBook Air
2nd Prize: iPad Air
3rd Prize: iPad Mini
4th Prize: Amazon voucher
5th Prize: Amazon voucher

You can submit your abstract using this online form here

We will pick 5 submissions from the entries to present at the event and the closing date is 19th December 2014 – everyone who presents will win something, and the audience will vote for their favourite to determine what loot you go home with from the above list?

Not convinced…? Read on.

Below are the most common reasons I hear not to do a talk… and are all my own personal opinion..

I’m nervous, I’m no Duncan Laverick, or Mike Dennemann  I can’t do public speaking

We all have been here, we’re a friendly crowd and it’s an excellent way to learn some important life & professional skills. To help with this the community at-wide offers the feed forward initiative, you can read more about this here – it can help you find a mentor to help you prepare for the day – one of our own members – Alex Galbraith did just this – read about his experiences on his blog here

I cant think of anything to talk about

We all have great experience from environments small and massive – I’ve personally found the best way to capitalise on this knowledge is to share it with others, having to distil it down to a presentation or blog post is a great way to help you focus on the important facts of what you did and how you did it – I have a tendency to whitter on (if you’ve read this blog for a while you already know this!) and doing this is an excellent tool, especially if you have to explain it to someone outside of your organisation

I work for a 3 man band company, nobody is interested in what I do

Wrong, I started my VMUGging talking about my 1 node home lab, you don’t have to have a data centre the size of the starship enterprise to have something interesting to say. Infact – I’d say from personal experience people from SMB type organisations have much more limited budgets and have to find far more creative ways to get things done

I work in a nuclear bunker, and I have access to your home address and browser history – I can’t talk about what I do

Now I know some of you maybe do work in nuclear bunkers, 3-letter govt. agencies or centres of commercial finance and are unable to talk about what you do for fear of exposing commercial secrets – I understand this, so do I.. My suggestion is that if you want to submit a talk – anonymise what you do – if you work for Goliath National Bank, just say you work for “a major bank”, or “an airline” – this is a VMUG, not Panorama. VMUG attendees are interested in the tech, what you did, how you did it & what could have gone better; not what impact it had on your Q4 financial results.

If you take this approach and remember to say you are not there as a company spokesperson, this is my own opinion etc. this is usually acceptable.

So, that’s it – submit here and I look forward to giving you your prize on 22nd January 2015.

..And to be honest – what’s the worst that could happen… 10mins of your time for a prize? …and we go to the pub afterwards anyway 🙂

If, even after all my objection handling you really can’t bring yourself to do it; that’s fine 🙂 you’re still welcome to come along and soak up the excellent content – you can register online at

What to see at UKVMUG 2014

The 4th annual UK-wide VMware User Group Conference is in a couple of weeks, on Tuesday 18th November several hundred (last year it was over 500!) of your peers will travel to the National Motorcycle Museum in Solihull.

The venue is just down the road from Birmingham airport, and a short taxi ride from the local train station – or easy to get to by road with free parking.

As part of the committee that puts this event on I’d like to point out some of the highlights of the agenda; as per-every year, we have the usual array of headlining technical super-stars from the virtualization community – Duncan Epping, Frank Denneman, Mike Laverick and Chris Wahl. But, over the years we’ve continued to build-up the community mezzanine area to deliver highly interactive face to face discussion groups and white board sessions – small groups of people talking brain-bendingly deep tech, or just introductions for n00bs – the guys staffing these tables are volunteers from all areas of industry and are the best brains in the biz – take advantage!

These are taking place on the mezzanine level and are noted as side-sessions on the main agenda. The mezzanine is the raised area at the back of the solutions-exchange & will be sign-posted on the day – there are way too many for me to detail on this blog post – 20 sessions in total!!!

    • VMware GSS are running a drop-in clinic – this is your chance to ask the tricky questions of VMware support, they are excellent supporters of VMUG and are sending two of their top level support & escalation engineers to field your questions face to face.
    • Designing real-world workflows for vRealize automation center (vCAC) with Jon Medd – if, like me – you’ve struggled to find practical examples & resources for this sort of project – Jon is your man; he’s done it for real – ask him the tricky questions and he’s an absolute wizz at PowerCLI!
    • Ask the CTO anything; this is your chance to have a face to face discussion with VMware EMEA CTO Joe Baguley – Joe is an avid VMUG supporter and general all-round expert at all things technical – he’s game for any questions – even non-VMware related ones (hint: ask him about fireworks or Robot Wars, or even SDDC) – how often do you get that opportunity! (he is also our keynote speaker).

We start early on the Tuesday, and as such we are repeating the popular vCurry the night before – check the box when you sign up to attend; we’ll be laying on curry and there will be a light-hearted quiz, this was a hoot last year – although we have already decided the guys from VMware GSS are excluded from winning 🙂 – although beware they may set some of the questions :).

Please note the event is on a Tuesday (not the typical Thursday) – but it’s early enough in the week to hopefully get your boss to agree that you can go to what is effectively a mini-VMworld in the UK.

You can see the full agenda online at and you can sign-up at

Press F8 to enter CIMC configuration does not work on Cisco c-class rack mount server

I encountered this today, on a system that is not managed by UCS manager, to setup the CIMC (HP iLo equivalent) you need to connect to the physical console with a screen and keyboard to set the initial IP address.

You do this by pressing F8 at the BIOS screen, however I couldn’t get this to work

the fix was simple, if a bit weird – I had accidentally cabled the management NIC to the serial console port on the back of the server, if you do this – it recognises you pressing F8 but then boots to a flashing prompt – I assume this is because it switches to some sort of serial console interface unstead of displaying the UI on the screen.

for reference – connect the Ethernet management cable to the correct NIC! as below


Move over vTARDIS here comes theBORG


Many of you will be familiar with my vTARDIS home lab project – this has run for a number of years and even won a couple of awards, making heavy use of nested ESX and virtual appliances for storage; however in the last 12 months RAM has been a massive problem – it’s hard to get SME type quiet hosts that can have more than 32GB of RAM.
If you consider most of VMware’s management stack consists of several 8GB virtual appliances and add in vCloud Director, vCAC etc. you can see my issue – I run out of capacity even before I start to run an actual workload.
In addition I’m doing a lot of work with the fantastic Atlantis ILIO and USX products, which can use RAM as storage infront of your storage array to de-dupe and deliver some very clever optimizations for storage (more on that in a later post).
So – I bit the bullet and purchased a blade chassis for home.
Did I read that right?
Are you rich/mad/deranged? (well definitely not rich but maybe the others..)

I was on the hunt via eBay for a DL380 or similar rackmount with 128GB of RAM – they were all pretty expensive, and for approx. the same money I managed to pick up a loaded up HP C7000 blade chassis with a total of >68 CPU cores and 318GB of RAM

Yes, it’s LOUD, yes it sucks a lot of power (this is with all the swich + FC modules installed, but only 1 full height blade powered on).



But – think of it this way – my total acquisition cost (including 4 x Cisco blade switches and 2 Cisco MDS FC blade switches was less than £2,000 GBP

It will cost (approx) £100 per month in electricity in the UK to run, I can handle that – consider it this way – purchased server cheap vs. spending £5,000 for a more modern set of kit to give the same capacity (and only ½ the running costs).

In my house we call this man-maths (and if you’re a Pistonheader – you may recognize the phrase) – it’s like when you buy an older, but very fast & cool car for a low (already depreciated) price but higher running costs (fuel, repairs etc.) but you enjoy it MUCH more than driving a more economical eurobox car for the same overall price(ish)


Anyway it makes sense to me and it currently runs about 40 nested ESX hosts, and I am only using 2 of the 14 blades ☺, it has a built in iLo-everything and it all comes in one box

(Disclaimer – the C7000 does not live in my house, as my wife would divorce me and the kids would have to wear ear-defenders).


More about what I’m using it for in future posts, yes that’s also an EMC VNXe for storage (again, subject of a future blog post – and to be honest it was the main reason for me needinf more compute capacity to push it – I blame Chad).

Do you want to know what VMware has planned for its next release?

Then you need to come to the London VMUG on July 17th (in London, UK), we are very lucky to be one of the few VMUGs that are able to deliver NDA content to VMware customers.

As this is NDA type content There are some caveats as to who can attend  – as listed below, but for those that aren’t eligible we have something else planned that will definitely be of interest.

And, we also have the usual array of awesome COMMUNITY, VMware and sponsor technical content. e.g no sales spin, no men in shiny suits and bonus schemes, real. interesting. relevant. technical content for YOU (ok, and me too!).

Sign up and view the full agenda here:

As I’m sure you understand VMware closely guard their roadmap and NDA content, we are very lucky to be able to deliver it at a VMUG, but there are some rules to prevent NDA material being leaked. If this happens VMware won’t be willing to help us deliver this content in future – so it’s in your interest to help safeguard the content – please respect this and the community spirit VMUG represents. You will have to sign a one-way personal NDA prior to being allowed entry to the NDA session.

All official VMware partners can request the content via their account manager at VMware, but to help make it clear we’ve put together an FAQ as follows.

  • I am an employee of a commercial or public sector organisation and we buy VMware products, or plan to, can I go the NDA session? Yes.
  • I’m a contractor and I work for a customer/consultancy, can I go to the NDA session? Yes
  • I work for a storage/network/server vendor, and am a VMware partner can I go to the NDA session? No. but you can request a private session via your VMware partner manager.
  • I work for a software vendor, and we are a VMware partner; can I go to the NDA session? No, but you can request a private session via your VMware partner manager.
  • I work for a reseller/consultancy who are a VMware partner, can I go the NDA session? No, but you can request a private session via your VMware partner manager.
  • I work for a cloud/managed/hosting service provider and we are VMware partners, can I go to the NDA session? No, but you can request a private session via your VMware partner manager.

This list isn’t exhaustive, if in-doubt of your status, bring a business card on the day andVMware can make a call, but please don’t be disappointed if we can’t admit you; we have an interesting update on VMware’s vision and strategy.

HP LaserJet 400 M475dw slow to connect to office365

I have one of these printers, it’s literally the best printer I have ever owned. it does everything, 1st time with no bother, aggro or hassle, it duplexes, copies, scans, prints very good colour pictures.


Granted, it’s not cheap or small but having purchased and thrown away literally tens of inkjet type printers over the years you get what you pay for with this printer.

The only small niggle is scanning to e-mail, I use Office365 for my personal and business email and I configured it to point at using this guide. however for some reason it took almost 3-5 minutes to connect and send to the SMTP server – I couldn’t find any obvious cause until I stumbled across this forum post, the solution? disable IPv6 – I tried this and it now connects & sends instantly.

For future reference, this is where you disable IPv6 on the web-UI.


VMworld accommodation on the cheap…

VMworld can be expensive, especially if, like me you are self-employed and don’t have a generous employer to cover the cost (or, even if your employer is cheap but won’t send you because of the cost).

For the last couple of VMworlds I have tried to save money on the accommodation as it can be a significant cost.

San Francisco, and Barcelona aren’t cheap cities, especially when they get wind of a conference – all the hotel prices go up as demand escalates.

My top tips..

1) Book early, before registration opens – early bird catches the worm, there are no last minute deals, ever.
2) Distance – whilst it’s nice to be near the venue, this can be very expensive – consider going further away and using the excellent public transport options in each city.
3) Lower your expectations, the Hilton is nice, but consider you’ll likely only be sleeping there between the conference and post-event parties – do you really care about the (premium) on-site spa facilities? and most cheaper places (even the hovels) have free WiFi – in your face $14.99/day Hilton!
4) Skip breakfast (at the hotel) and either eat locally to Moscone/Fira or there is usually breakfast laid on at VMworld (granted, not always the best food – WTF is a breakfast burrito for example, but still…)
5) Share, VMworld hotels are EXPENSIVE; especially if they are close to the venue; consider using AirBnB or look for a shared apartment on Expedia (easier in Barcelona than SF in my experience) & use the power of social media and gather up some like minded geeks + split the cost, there are significant savings here – provided you can avoid sharing with any serial-killers. On the latter point you can sort of vet would-be sharers via Social Media before saying yes (if they frequently post pics of shrunken heads or selfies with corpses, maybe think twice :)).

As evidence of this, I’ve stayed in some hovels in the past, stayed the other side of SF +commuted & this year I’m staying on a boat in the harbour with a mate via AirBnB + it’s 1/2 the cost of even a hovel hotel.

Good hunting!

Cannot logon to vCAC portal and you get Internal error when logging on as the root tenant

If you came to London VMUG last week and were one of the people who attended my vCAC real world session, you’ll have seen the demo gods not shining kindly on me as I couldn’t logon to my vCAC demo environment.

This was particularly annoying as I was testing it not 15mins before my session started and it was working 100%

With some quick on-stage debugging I found that I was unable to logon as any of the tenants on my vCAC deployment so was unable to show my cool demo 😦

After some further digging I found that I could logon to the root org using the administrator@vsphere.local account but nothing else and I couldn’t logon to any of my tenant organisations despite the credentials being 100% correct.

When logged onto the root org as administrator I could see the following error… which translates as “not good, your tenant has disapeared and I have an internal problem”


After some digging this seems this is a sort of time-bomb bug that has only been recently discovered, it occours after 90 days and will stop tenants being able to login using LDAP credentials.

When attempting to log into a tenant, a blank page is displayed with a Submit button in the upper left corner.
You receive a System Exception error when accessing the tenant identity store configuration page and the identity store configuration may have disappeared.
You cannot log in to a tenant using an LDAP account.
You are unable to add a new identity store configuration to the affected tenant.
The tenant identity store disappears from the SSO Administrator login.


This KB article details the issue and a temporary workaround

I assume a fix will follow in due-course.

In my case I worked out that it was exactly 90 days until about 15mins before my vCAC session started last Thursday – which is bad-timing taken to the extreme.

Once I have verified this fix in my environment and resumed access I will post a video of my intended real-world demo, where I’m using vCAC to deploy a VM and talk to a massive “enterprise system” via a REST API (e.g Twitter :)) – whilst an odd use-case this is an easy way of demonstrating the extensibility of vCAC (well, when it works anyway)

So, in summary if you have a vCAC implementation that you deployed since 6.0.1 came out you’re going to run into this install date +90 days.



How to Remove LinkedIn contact birthdays from your iPhone Calendar

Contacts of LinkedIn, I love you all. honestly I do – but there are a lot of you.

When I added the LinkedIn app to my iPhone it “helpfully” populated by iPhone calendar with everyone’s birthday – which is kind of a problem when you’re challenged for screen estate on the iPhone, and frankly as much as I would love to send you all a card and gift it’s not going to happen, so it’s a bit annoying.


You can remove these by tapping on Calendars at the bottom of the Calendar display.

Then scroll down past you usual calendar settings, look for “other” and uncheck Birthdays.


And presto, they’re gone.

*PS Kenny – I’ll buy you a bourbon next time I see you instead 🙂