Virtualization, Cloud, Infrastructure and all that stuff in-between
My ramblings on the stuff that holds it all together
Nothing happens when I try to run Starwind software Manager on Windows 2008 R2
Posted by on January 24, 2010
I am doing some work with a FusionIO solid state flash storage card at the moment (more on this in a future post) as part of this I need a windows based iSCSI target for my testing, and rather handily you can download an evaluation copy of the Starwind Enterprise Edition from here
I usually use OpenFiler for this sort of thing, but not being a particular Linux wizz (ok, and being a bit lazy and in a hurry) I wanted to try out the Fusion IO Duo card I have been loaned, and the Linux drivers are an .RPM or .deb package for which OpenFiler doesn’t have the required package management software – so I have installed it in a Windows 2008 machine and will use the Starwind software as an iSCSI target.
(In terms of disclosure, whilst i was writing this post up last week VMware vExperts were offered an NFR license for the product, this test was done with the freely downloadable eval version rather than the NFR license we have been offered – but I urge you to check it out, it’s pretty cool.
Anyway – when you first install the software on Windows 2008 , there is a Starwind icon on the desktop
When you double-click on it (or any of the start menu entries nothing happens, you don’t get a UI or anything. this confused me for a while until I discovered that it places a system tray icon on boot, which you use to configure the software.
by default on my Windows 2008 R2 machine this icon is hidden, and set to only show notifications – of which there were none yet.
A quick trip to the customize button on the Notification area menu options on the properties of the task bar shows the default setting which is hiding it
Setting this to show icon and notifications made it re-appear on the taskbar/notification area
You can now right-click and launch the management console 
The management console
It’s a bit strange that the desktop or start-menu icon doesn’t launch the manager ‘out of the box’ with Windows 2008 – but this is how to resolve it, the hint eventually came from the online help, which said to go via the system tray icon, so it just goes to show – maybe sometimes you should look at the help files!
Hopefully that will save you some time with your eval!
Using the VCE/vBlock concept to aid disaster relief in situations like the Haiti Earthquake
Posted by on January 16, 2010
Seeing the tragic events of the last couple of days in Haiti played out on the news spurred me into evolving some thinking that I had been working on, the sheer scale of infrastructure destruction left by the earthquake in Haiti is making it hard to get relief distributed via road, so airlifting and military assistance is the only realistic method of getting help around.
Whilst providing physical, medical, food and engineering relief is of paramount importance during a crisis, communications networks are vital to co-ordinate efforts between agencies, it is likely that whatever civil communications infrastructure, cell towers, landlines etc. are badly impacted by the earthquake so aid agencies rely on radio based systems, however as in the “business as usual” world the Internet can act as a well-understood common medium for exchanging digital information and services – if you can get access.
Crisis Camp is a very interesting and noble concept for gathering technically minded volunteers around the world to collaborate on producing useful tools for relief staff on the ground, missing people databases, geo-mapping mashups on Google Earth etc. using open source tools and donated people time makes this a free/low-cost soft-solution for relief agencies.
However, with the scale of infrastructure destruction in large disasters getting access to shared networks, bandwidth and cellular communications networks on the ground is likely to be difficult – in this post I propose a vendor neutral solution, whilst I reference the VCE/vBlock concept which is essentially an EMC/Cisco/VMware product line; the concept of a packaged, pre-built and quick to deploy infrastructure solution can apply equally to a single or multi-vendor “infrastructure care package” – standardisation and/or abstraction are the key to making it flexible (sound familiar to your day job?) by using virtual machines as the building blocks of useful services able to run on any donated/purchased/loaned hardware.
These care packages would typically be required for 2-3 months to aid disaster relief during the worst periods and whilst civil infrastructure is re-established. None of this stuff is free in the normal world, it’s a physical product, it’s tin, cables, margin and invoices but is flexible enough that it could be redeployed again and again as needs dictate, with my UN or DEC hat on this is a pool of shared equipment that can be sent around the world and deployed in 24hours to aid on the ground relief efforts, donated, loaned by vendors or sponsors.
What is it?
A bunch of low-power footprint commodity servers, storage and communications gear packed into a single, specialised shock-rack with a generator (gas/diesel/solar as available) and battery backup.
It makes heavy use of virtualization technologies to provide high-availability of data and services to work around individual equipment and/or rack failures due to damage or loss of power (generator out of fuel or localized aftershock etc.)
Because systems running to support relief operations typically will only be required for short term use, virtual appliances are an ideal platform, for example a pre-configured database cluster or web server farm, technologies like SpringSource can be used to deploy and bootstrap web applications around the infrastructure into virtual appliances.
Data storage and replication is achieved not using expensive hardware array based solutions but DAS storage within the blades (or shared disk stores) using virtual storage appliances like the HP Lefthand networks VSA or Celerra VSA or OpenFiler – allowing the use of cheap, commodity storage but achieving block-level replication between multiple storage locations via software – each blade uses storage within the same rack, if access to the storage fails it can be restarted on an alternative blade or an alternative rack (like the HA feature of vSphere)
These racks are deployed across a wide geographic area – creating a meshed wireless network using something like WiMax to handle inter-mesh and backhaul transit and local Femtocell/WiFi technology, providing 3 services
- private communications – for inter rack replication and data backhaul
- public data communications – wireless IP based internet access with a local proxy server/cache (backhaul via satellite or whatever is available – distributed across the mesh)
- local access to a public cellular system femtocell (GSM, or whatever the local standard is)
The availability/load balancing features of modern hypervisors like VMware’s HA/DRS and FT technology can re-start virtual machines to an alternative rack should one fail. Because the VSA technology replicates datastores between all racks at a block level using a p2p type protocol it’s always possible to restart a virtual appliance elsewhere within the infrastructure – but on a much wider scale and with a real-impact.
Ok, but what does it do?
Even if you were to establish a meshed communications network to assist with disaster relief activities on the ground, bandwidth and back-haul to the Internet or global public telecoms systems will be at a premium, chances are any high-bandwidth civil infrastructure will be damaged or degraded and satellite technology is expensive and can have limited bandwidth and high-latency.
The mesh system this solution could provide can give a layer of local caching and data storage, thinking particularly with the Google Maps type mashups people at Crisiscamp are discussing to help co-ordinate relief efforts that can require transferring a large amount of data – if you could get a local data cache of all the mapping information within the mesh transfer times would be drastically reduced.
this is really just a bunch of my thoughts on how you can take current hypervisor technology and build a p2p type private cloud infrastructure in a hurry, virtualization technology brings a powerful opportunity in that it can support a large number of services in a small power footprint; the more services that can be moved from dedicated hardware and run inside a virtual machine (for example a VoIP call manager, video conferencing system or GSM base station manager) mean less demand for scarce fuel and power resources on the ground; and virtualization brings portability – less dependence on a dedicated “black-box” that is hard to replace in the field, virtualization means you can use commodity x86 hardware, and have enough spares to keep things working or work around failures.
The technology to build this type of emergency service is available today with some tweaking. The key is having it in-place and ready to ship on a plane to wherever it is needed in the world, some more developed nations have this sort of service in-country for things like emergency cellular networks following hurricanes but it will need a lot of international co-operation to make this a reality on a global scale.
Whilst I’m not aware of any current projects by international relief agencies to build this sort of system I’d like to draw people’s attention to the possibilities.
The DEC are accepting donations for the Haiti earthquake relief fund at the following address.
or the international red-cross appeal here
It’s voting time..
Posted by on January 4, 2010
Eric Siebert runs vsphere-land.com which is a handy site listing popular blogs about vSphere; I’ve been on the list for a while but Eric is running a poll to determine the top 25 virtualization bloggers.
if you are feeling generous towards this site *cough* you may find this link useful 🙂
if it helps to remind you, or you haven’t seen them before – these are my most popular posts this year;
10 node virtual ESXi cluster on a trolley
How to deploy Windows 2008 server with a template in vCenter
Comparing disk I/O of virtual machines on SSD and SATA disks with IOmeter
Performance of cheap vSphere server
How to enable FT for a nested VM (a VM running on ESX inside ESX)
Applying Agile methodologies to Infrastructure – virtualization is your friend
Using virtualization to extend the hardware lifecycle
VMware ESX 5 – what would you like to see
My VMworld Europe 2009 posts (list courtesy of Duncan Epping)
It’s 2010 Your Usergoup and the blogsphere.. need you!
Posted by on January 4, 2010
Ok – it’s new year’s resolution time – how about this one..
User groups (and blogs, in a less face-face manner) are an excellent way to meet like minded tech people in an informal setting and are a useful way to get information about how other people are doing things and current real-world trends/best practice.
But, none of this is possible without people stepping up and contributing – you don’t have to be a genius to contribute (look at me! :)) nor do you have to be in-charge of the largest planet munching datacentre laden with the most advanced, cutting edge tech in the world, even the humblest IT shop have something in the way of experience that they can contribute – what problems have you seen, and how did you fix them, what do you think would be useful, something creative you’ve built?
Public speaking isn’t my idea of absolute fun, it’s hard and you are putting yourself “out there” in anticipation that people will be interested; or at the very least be polite enough not to throw things at you.
my advice..
Don’t feel you have to know everything about everything, it’s ok to say I don’t know, and throw it to the floor – the man who says he knows everything actually knows nothing! it could spark an interesting debate – you don’t get that kind of thing at formal conferences.
User groups are about users, not sponsors or vendors showing their wares – they have ample online and conference time for that (although sponsors are obviously an important part of it – as they pay for it!) so take advantage of the experience in the room
It’s also easy to start a blog, it’s easy to get your thoughts out there (however serial twitter RT’ers and blog-scrapers need not apply :)).
it’s also good personal and career development, even if it’s just about making you structure your thoughts properly – I wrote some thoughts on this a while back
If you have some ideas for user group presentation sessions, or indeed something different – just write up a proposal, it doesn’t need to be anything majorly formal – just an email with the salient points and submit it to the co-coordinators;
Title:
Format: presentation/panel/discussion
How long you would like: (keep it under 45mins)
Outline: agenda, key points and/or questions you would like to cover, what people would get from the session
Along similar lines – get a blog, get some thoughts, something you’ve fixed (with some pictures) even something you’d like to see in future versions – write it up, get it out there www.wordpress.com is all you need!
Go forth and contribute in 2010 .. 🙂
vT.A.R.D.I.S – 10 ESXi node cluster on a trolley as demonstrated at London VMUG
Posted by on December 17, 2009
I recently presented a session at the London VMware User Group meeting about home labs, this post is a follow-up with the slides I used and some more details on the configuration.
The kit I demo’d has affectionately been named the the vT.A.R.D.I.S which stands for Trolley Attached Random Datacentre of Inexpensive Servers 🙂 or Hernia-maker – don’t feel like you actually have to strap yours to a trolley though 🙂
This is part of a series of joint postings with my esteemed colleague Mr.Techhead, my sections of the series concentrate on the details of building a virtualized ESX cluster using the vSphere 4 for learning and test & development; Techhead’s posts will focus on the best low-cost hardware to use and specific configuration steps and I will cover some of the configurations and use cases.
You may be wondering why you would want to do this? well, if you are studying for your VCP or developing scripts or utilities for managing vSphere environments you rarely have a multi-node cluster at your disposal to test against because by it’s very nature it requires a lot of {usually expensive} hardware and you miss the more advanced configurations like HA/DRS/FT that this type of environment can use.
Also consider the larger production-type environment where you want to test some automatic deployment or management scripts and tools – this is an ideal approach which uses minimal hardware to conduct the 1st stages of test and development – if you’re an ITIL shop this is release management. Even the best equipped test labs won’t give you more than a couple of hosts to play with – this virtualized ESX approach means you can have many more ESX hosts to test against without busting the bank.
So we have put our heads together and have come up with what we think is the lowest possible cost way to build such an environment, and unsurprisingly it makes heavy use of virtualization – to allow you to study and work on without
- Being too noisy to leave switched on
- drawing too much {expensive} power
- costing the earth
The catch: Now, of course nothing is for free so to build this it will cost you some money, but it will be a lot less than your typical production environment and more into the hobbyist market – of course you get what you pay for, and I wouldn’t be going into this with the expectation that this will perform well enough for you to compete with EC2 🙂 but for your own general home use; and probably that of an SoHo/SME type organisation it’s ideal.
The photo below shows the demo kit we used for the London meeting cunningly strapped to a B&Q trolley for “portability” 🙂
<
To break it down into each major functional area and as a taster of the follow-up posts here are some of the things you need to consider..
Storage
Shared storage is a requirement for HA/DRS/FT and is usually the most expensive part in a production environment which would typically be Fibre Channel and SCSI disk SAN storage, you’ll never get this on our budget so we have taken the OpenSource and iSCSI and SATA approach, we have put this through its paces for the last 2 years in varying topologies and it performs very well and will more than service your own personal/study needs, it also has the advantage that it can probably be recycled from that pile of spare PC parts you have in the cupboard.
There are also a number of low-cost NAS devices which should be within your budget if you don’t; Techhead has a number of posts on the way around this.
Network
Building flat networks is easy – you just need a dumb switch, or even a hub and away you go; but by doing this you miss the subtle configuration problems you need to understand to do things properly in a production environment, so ideally you need something that will support VLANs and routing – you also need Gigabit ports for vSphere; although I have had vMotion working on a 100Mb switch in the past.
We have looked for a long time but there are no cheap (<£400) Gigabit switches even if you go 2nd hand.
There are numerous low-end switches that support VLANs, but can’t do the routing between VLANs so you either need an external hardware router like a Cisco 2600 or something else..
So, a compromise is needed – we opt for a low-cost Gigabit switch with VLAN support like the 8-port Linksys SLMxxx and compliment it with a virtual machine running the Vyatta community edition virtual appliance which can provide the L3 routing betweenn your VLANs (a sample of how easy to configure it is shown below)..
Server
Techhead is an avid HP-fan; and rightly so as they make great production kit but I had never really explored the lower-ends of their range such as the ML110 and ML115 range – these are single CPU socket servers with internal (non hot-swap) SATA storage, whilst they don;t have on-board redundant hardware they are quiet and more importantly – surprisingly cheap and fully ESX 3/4 compatible.
Techhead has some good deals on the ML115 G5 hardware at this link, here and here and best of all the ML115 G5 is compatible with the new Fault Tolerance feature of vSphere
if you wonder what is inside an ML115 server read this link
Hypervisor & Nested Hypervisor VM
VMware ESXi is my current weapon of choice for this environment and so will be the focus of this series of posts; unfortunately I’ve not found a way to run nested Hyper-V or Xen Virtual Machines, that would be the ultimate in flexible learning platform – unless anyone out there knows how to?
I make heavy use of the new Fault Tolerance feature of vSphere to protect the vCenter and Vyatta virtual machines in this environment.
It’s the ideal setup to test unattended deployments of ESX hosts as well as you can just delete them and start again.
Virtualized ESX Hosts – 10 ESXi hosts running on 2 physical machines
Detailed Posts Index
Rather than do one long post we have a series of break-out posts on the specific areas of this topic.
this is the list of topics to come; when articles are posted the links will be populated and become clickable.
Part 1 – Lab Hardware Overview (coming soon @Techhead)
Part 2 – Lab Hardware Configuration (coming soon @Techhead)
Part 3 – ESXi Installation & Configuration (coming soon @Techhead)
Part 4 – Shared Storage Installation & Configuration (coming soon @Techhead)
Part 5 – Networking Configuration (VLAN’ing & Jumbo Frames) (coming soon @Techhead)
Part 6 – VM’d ESXi (Coming soon @vinf.net)
Part 7 – VM’d vCenter; auto start-up of VMs (Coming soon @vinf.net)
Part 8 – VM’d FT and FT’ing vCenter VMs (Coming soon @vinf.net)
Part 9 – FT on the ML115 series – benchmarking with some Exchange VMs (Coming soon @vinf.net)
Part 10 – VM’d Lab Manager farm environment on a pair of ML’s (VM’d ESXi) (Coming soon @vinf.net)
Part 11 – VM’d View 4 farm environment on a pair on ML’s (VM’d ESXi) (Coming soon @vinf.net)
Part 12 – Home backup – VMware data recovery / fastSCP/Veeam backup or something else low-cost with USB drives/etc. (Coming soon – joint posting)
The slides from my original VMUG presentation are available online at this link
OWA 503 Service Unavailable following 973917 update
Posted by on December 17, 2009
I’ve seen a couple of instances of this in the last week where previously working Exchange 2003 servers suddenly stop serving Outlook Web Access (OWA) Requests overnight
Investigating the eventlog shows the following entry which corresponds with stopped application pools in IIS Manager;
Event Type: Error
Event Source: W3SVC
Event Category: None
Event ID: 1059
Date: 14/12/2009
Time: 02:01:37
User: N/A
Computer: EXCHANGESERVER
Description:
A failure was encountered while launching the process serving application pool ‘DefaultAppPool’. The application pool has been disabled.
For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp.
This seems to be related to a recent Microsoft Update – 973917 uninstalling the hotfix, followed by a reboot immediately resolves the problem and OWA starts working again.
I’ve also had reports from people that this update has affected some other bespoke IIS applications, again uninstalling the patch seems to resolve the problem.
This situation seems to arise from a mismatch of installed binaries – now I thought system file protection and the .msi based patch installers were supposed to avoid this situation from Windows 2003 and onwards – anyway for more information on the cause and how to resolve without uninstalling the 973917 patch see the following links
http://msdnrss.thecoderblogs.com/2009/12/16/aftermath-of-the-release-of-kb-973917-for-iis/
Cloud Camp London (21st Jan 2010) now open for registrations
Posted by on December 16, 2009
You can register for the next Cloudcamp London on the 21st Jan 2010 at this link
If you don’t know what cloudcamp is about – check out one of my previous posts if you are available I recommend it.
The GeekCabin
Posted by on December 15, 2009
I am currently building a new home office/lab/cave for my computer habit and have been maintaining a separate blog with all of the details – it’s online at http://geekcabin.wordpress.com so please check it out.
Don’t worry I’ll still be posting my normal content here and nothing is happening to this blog – but the other blog is for people interested in the construction etc. and keeps it away from my technical stuff
It’s turning up next week (just in time for xmas – hopefully) and I look forward to having some space to work on things again as children seem to take up so much room – especially after every xmas!
Amazon EC2 boot from EBS
Posted by on December 4, 2009
This is a much wanted feature, I haven’t checked yet – but if this is allowed for Windows instances I can see a whole heap of new use-cases in my work – check it out – this is akin to boot from SAN in a traditional infrastructure and allows for persistent OS images to be kept around when they are not running on EC2.
official announcement..
Amazon EC2 Boot from Amazon EBS
Amazon EC2 has also announced the ability to boot instances directly from Amazon EBS snapshots, providing significantly increased flexibility in how customers can manage their instances. You can still save an Amazon Machine Image (AMI) in an Amazon S3 bucket and boot it from the local instance store, but you can now also choose to save AMIs as Amazon EBS snapshots and boot directly from an Amazon EBS volume. When an instance is booted from an Amazon EBS snapshot, the root partition of the instance is created on an Amazon EBS volume. Instances booted from Amazon EBS volumes can be stopped and later restarted, preserving any of the state that is saved to your volume and allowing you to modify some properties of your instances while it is stopped. For example, you can change your instance size or update the kernel it is using, or attach your root partition to a different running instance, making it easier to do debugging when you are creating new boot images. When booting from an Amazon EBS volume, AMIs and root partitions are no longer limited to 10GB, but can be up to 1TB in size, enabling significantly more complex images. Additionally, you are not charged for stopped instance hours and you will only incur charges for your Amazon EBS volumes while your instance is stopped, allowing you to reduce your Amazon EC2 costs when you do not need your instances running. Customers can now use a newly launched API that makes it easy to bundle images without using the command line tools, and can also take advantage of the fact that the content of an Amazon EBS volume is available to the instance immediately on volume creation which can lead to much faster instance boot times. For more details on this new addition to Amazon EC2, please see the Boot from Amazon EBS Feature Guide.
Using VMware Fault Tolerance to Protect a Virtualized vCenter machine
Posted by on November 30, 2009
In my lab I have a virtualized vCenter installation, it works well and I’ve had no problems with this configuration in the last year.
I wanted to try to build a 2 node demo cluster for my VMUG session and needed vCenter to be protected by FT – so an individual host failure would not break vCenter during my demos.
My vCenter installation was thin-provisioned which isn’t compatible with FT so the quickest solution I found to this was to just clone it to a new VM with a fully provisioned (thick) disk.
Once completed I powered up the cloned vCenter installation whilst quickly switching off the old one to avoid any IP conflicts this worked fine and the ESX hosts didn’t really notice, I just had to re-connect my vSphere client.
I then enabled the FT features and after doing its thing I have a fully protected ESX/vCenter installation using FT.
it’s worth noting that you can only enable FT when using a vSphere client connected to vCenter – you can’t enable it if you connect directly to the ESX host itself (which is why cloning was the easiest approach for me)
Virtualization, Cloud, Infrastructure and all that stuff in-between 