Virtualization, Windows, Infrastructure and all that “stuff” in-between

Well over the weekend this blog turned over 50,000 visits since it started in November 2007 whilst I was on a break at Briforum 2007 in Amsterdam, so lots of thanks to everyone who has been reading, this blog is currently averaging 500 views/day which considering I just started it as a repository for my own useful work tidbits is quite amazing to me.

I’ve got a couple of interesting posts in the pipeline around unattended ESX installations and a step by step to build this under VMWare Workstation and my experiences with Platespin PowerConvert - particularly how it maps into my mini/dedicated cloud architectures to deliver a totally flexible and easy to manage infrastructure through the physical to physical conversion process (P2P) and the normal V2P/P2V features.

Similarly, if there is anything you’d like to see - comment here with your suggestions!

Happy reading

Useful Post on betanews from last year on details of new kernel level features in Windows Server 2008 as presented by Mark Russinovitch at WinHEC in May 2007

I like this excerpt from Mark’s keynote presentation… lol

“This slide…this being a keynote, the marketing people had to make a pass through the deck. And this thing is technical, which is a little bit different from what they’re used to, they didn’t understand any of the slides. But they still wanted to feel like they were adding value, so they threw this slide in. And of course, I don’t understand this slide. But I hope you like it.”

Mark Russinovich, Microsoft technical fellow

Linkage here. Looks like a good idea, it’s always a pain doing this type of thing, especially remotely.

cool.. http://www.hpcsystems.com/blog/?p=24

Hyper V apparently only supports 16 cores , but here’s some info on hyper V running on an 4 x 4 CPU core system.

Won’t be long before the price point for these really drops; imagine how many VM’s you can cram on one of these at the recommendation of 3vCPu:1pCPU-Core.

(8 sockets x 4 core) x 3 vCPU = 96 single CPU VM’s per server Nice.

Yeah, one for the real geeks to appreciate (myself included!) I’ve been guilty of some not so nice installs in the past where time allocated supercedes art by a significant margin… but these are ace!

Shame we can’t really stack cabs with 42 x 1U servers anymore without someone coming to shout at me about power allocations.. ah those were the halcyon days of providers selling rack space by the U.. no power limits

I’ve looked at this topic a number of times as we often have requirements to send sensitive files around - lots of customers send them to me via email/FTP or on CD within encrypted WinZip files as this is what they find easiest as it’s pretty ubiquitous rather than having to agree a compatible encryption app/protocol and have it “blessed” by a security dept/PC build team - Dave Whitelegg has posted a useful article here outlining the practical limits of this approach and suggested password lengths.

Obviously if you have information that is worth an attacker spending several weeks brute-forcing then I would suggest maybe you shouldn’t be sending it electronically or even holding it at all; as I’m sure there would be quicker ways for an attacker to find this information once its in it’s unencrypted form at either end, social engineering/bribery etc.

And of course - if you do have to persist in the encrypted WinZip approach maybe rename the files held within for a bit of security by obscurity - “Board of directors - salary review.xls” is probably a lot more tantalizing to an attacker than “Photocopier Toner Audit.xls” or “AACD12323.DAT” or place a .zip file within another .zip file as you can see the table of contents with in the .zip regardless of its encryption state.

Our very own marvelous HMRC could do with reading this article being as it seems to be data breach disclosure month!

keep up the good work Dave!

Interesting article here

What if you could breach the hypervisor? best practice would dictate firewalling off the management traffic to the service console to a management network but what if you could exploit the VM Tools or other enlightenments/paravirtualizations to compromise the hypervisor - if you could you own every VM it’s running.

Does this compare to VLAN jumping on a Cisco switch? As far as I understand it show me a practical exploit to do this and the mitigation steps are quite well documented.

This is (and will) always a big issue with Multi-tennant systems but it’s the same issue that we currently face in most service providers, shared SANs, LAN, WAN, even physical buildings/suites etc. - virtualization is just a marketing tag, the same principals have been applied in the physical world for ages and mitigated against - I don’t think this is any different.

A session with the US Marine Corps at VMWorld 2007 mentioned that the US DoD had audited the code of ESX for this issue and found it to be satisfactory - but I’ve not seen this documented anywhere, if it’s safe for the US .mil isn’t it safe enough for you?

Compare risk vs cost saving, patch, mitigate, move on but keep your eyes open.

This is a bit underhanded; preventing debugging tools from tracing your applications especially when the underlying OS is derived from Open Source technology where one would expect to have such access.

Although you can obviously patch it yourself as you can have the source and recompile the associated binaries; bit of a waste of time?

Go and get it from here

I spent a lot of time at the start of 2007 building this type of system from scratch (see the build a better test lab posts). hopefully this will go a long way to making it easier to achieve.

(Apologies to fellow Brits for the spelling of “center/centre”, it bugs me too! but that’s the product name, spelling and all - plus it helps our worldwide friends who are coming in via Google)

Just incase you are interested here are the steps to do so.

I have a Windows 2003 Enterprise Edition “Gold” VM image that I’ve used for years (see this page for some more good ideas on that) and I’ve ported it all the way from VM Workstation 4.x, through 5.x, VMware Server 1.x, 2.x and now ESX 3.5.

I just clone it periodically and I keep updating and sysrep’ing the master image with the latest updates (SP2, current VM Tools, iSCSI initiator, BGInfo, etc.)

I used the VMWare P2V Convertor (which yes I slated earlier.. but it works in this instance) to convert from Workstation 6.x format for my new ESX server and manage it as a template via Virtual Centre.

1st off, Right click on the template and choose to deploy (hint: if you want to make a template right click on a VM you prepared earlier and clone/convert to template.)

Choose where you want to run the VM - this is a list of your VC data centres

Choose the ESX host where you want to run it - I only have 1 which is my desktop ESX server (http://vinf.net/2008/01/14/vmware-esx-v35-on-cheap-pc-hardware/)

I get this warning message, but this is because I’ve ported my VM across so many different versions of VMWare, and the template VM still has a virtual USB port - must get round to removing it!

Choose the datastore - this is my 500Gb SATA drive inside the PC

and you can pick a template to customise the VM, this essentially lets you choose (or not) to automatically run a SysPrep once the VM has booted - the “customization specification” is essentially a sysprep.inf file that you pre-created using the customization specification wizard (below).

The customization wizard does seem to add some bells and whistles as you can choose the VM machine name based on what you’ve called it in Virtual Center or spawn out to an external application/script which is a nice feature that I don’t believe you can do with standard Sysprep

Anyway, back to the VM deployment..

Choose from your set of templates, I have just one at this stage that incudes the product key, regional settings and create the server name based on the VM name, note you can also break out to the customization wizard to make one time adjustments to the specification you’ve chosen.

You are then shown a summary of the VM you are going to create and given options to power it on once the clone is finished, or edit the virtual hardware (add more CPUs, disks, RAM, etc.) - not sure why edit hardware is (experimental) would think it would just spring up the normal UI for doing this within VC.

Interesting to note the warning umm, this is deploying from a pre-built image - but I guess VC doesn’t know that for sure.

You’ll se a job submitted to Virtual Center’s queue

It took 9mins to deploy - and this was on my cheap ESX desktop PC so not the most high-performance disk subsystem - but more than acceptable, whenever I’ve had to do this in the past on a physical PC it usually takes at least this long to find the correct CD

Proof here

The VM is now booting and doing it’s sysprep/minisetup wizard without any hands-on required - it’s totally automated via the customization specification/template setup.

OS Starting, installing VM Tools in the background

VM Reboots automatically.. (but I wasn’t quick enough to get a screen cap of that..)

Built & Ready to go! (my customization template makes the administrator account auto logon on 1st boot)

Start to finish, a ready to use OS with all it’s service packs and any software I require in 11mins, and that’s on cheap hardware.. all the timestamp’s are in the screen shots if you need proof

Obviously the Asus is significantly cheaper and the the screen is annoyingly small - interesting review here

…I haven’t applied mine yet, but the Lone sysadmin has reported some problems with VMotion on their system since applying. details here might be co-incidence but always worth keeping this kind of thing on your radar.

Hilights the fact that automated tools do not a good patching process make.

…and it’s free! - get it here http://www.rtfm-ed.co.uk/?p=476

Thanks to Mike Laverick - an excellent doc I like the look of the new update manager and the dynamic power saving stuff… have to wonder how well suspend/wake on LAN will really work in a switched environment.. I’ve never had much success with it in the past.

No, not another post about floating data centres, A whole bunch of ESX patches just released here; thanks to Yellow Bricks for pointing that out.. will give the new 3.5 update manager a whirl and report back on what happened! fingers crossed.

See - one advantage of having your own test/cheap ESX home server is you can try these things out

Handy article here at VM/ETC

Microsoft OneNote is my favorite Microsoft product - I’ve used it for a couple of years now and it’s almost totally replaced my need to lug around hardback notepads.

I say almost as sometimes I still need to scribble down some diagrams, something that doesn’t work well with OneNote unless you have a Tablet PC, that said it’s easy to scan diagrams into OneNote with a scanner and keep a record and takes away all the concerns with loosing a paper notebook or not having it with you.

I work disconnected from our corporate network for most of the day but OneNote allows me to host a shared notebook on a SharePoint site which is accessible over SSL I can work on a local sync’d copy and I can make it sync with the server held copy.

It’s useful for sharing notes with my co-workers on a project - it’s essentially a Wiki like tool with good off-line capability.

I would like to see better support for drawing with a non-tablet PC and having it properly anchor scribbles over bits of text - but maybe that’s just me not working it correctly as when I move the text block my scribble over the top doesn’t move.

Anyways - the team have posted some useful Power Toys for OneNote which I will definitley be looking into..

Available from http://blogs.msdn.com/johnguin/archive/2008/01/17/a-summary-of-the-onenote-powertoys-from-the-test-team-for-2007.aspx

More OneNote goodness here http://blogs.msdn.com/chris_pratley/default.aspx 

and  http://blogs.msdn.com/descapa/archive/2008/01/04/blog-roundup-for-dec-2007.aspx

and http://stevepietrekweblog.wordpress.com/2008/01/17/links-1172008/

Microsoft really don’t push/promote this product enough.. it’s great, everyone I show it to ends up using it; as Chris point’s out here it spreads virally between you show it to!

You can even use it to manage cooking Christmas dinner (…maybe I’ll blog about how I did that one day…:))

This looks interesting, uses p2p storage syncing to allow live workloads to be moved between VM Hosts.

Would be a very interesting tech, you could build a large VM compute farm with cheap DAS storage without having to invest in “proper” shared storage such as a fibre channel SAN or iSCSI.

Considering even a DL360/380g5 can take about 800Gb of SCSI/SAS disk these days that’s a lot of storage for replicated/sync’d VMs.

Wonder if VMWare have something in the pipleline to compete, We often see the cost of the shared storage as a big blocker to building VM farms, servers can be obtained relatively cheaply and allow you to easily scale out horizontally, SAN’s not so much as you need to invest upfront in the fibre switching/shelves to allow you to scale out, it’s not so easy to do it incrementally - if you can just buy cheap incremental servers with DAS and just add them into a farm that’s quite appealing; to me anyway.

Interesting Blog to keep an eye on here http://shippingseven.blogspot.com/ allegedly from an engineer working on the new Windows OS, be good to see what comes out - MS haven’t said very much at all about it.

Mmmm, interesting.

They’re aiming for virtualizing much more than just the OS.. VMWare obviously realise the Hypervizor game is coming to an end where ESX will be a free/commodity release and all the add-ons/management will make the money.

This makes an interesting foray into application level virtualization - more anlysis here.

Another other interesting but unrelated point here is about how RSS feeds break news far quicker than Google can index it!

I’ve not tried this before but it seems to work, my config is as follows - note this is a home/test system so I don’t suppose this is supported for a production platform

• Single ESX 3.5 host
• 1 x Windows 2003 VM with Virtual Center 2.5, licence manager, update manager and convertor plug ins, SQL Express database for VC (not supported for production)
• VM is connected to the same IP network as the ESX service console.

Because I was short on hardware I built the 1st Virtual Centre server as a VM on my laptop with VM Workstation 6 as a linked clone from my standard VM server image.

Once everything is up and running I used VM Convertor from Virtual Centre to do a P2V migration of itself over the network (weird!) into ESX - technically a V2V

I then powered up the VC VM on the ESX box and changed it’s IP address to avoid conflicts.

Then shut down and killed the VC VM on my workstation.

It worked - I can connect to the virtual VC box and use it as before.

I also installed the VI client directly on my laptop so I could manage the virtual VC box over the network without RDP’ing into it.

I set the VC VM to auto-start with the ESX host and physically rebooted the ESX box to make sure - and it all rebooted fine and both the ESX and VC VM started up, I believe the ESX host caches licence data in case of a short-term VC server failure - this seems to get around the issue of not being able to power on a VM without a licence and seems to work ok for me.