Virtualization, Cloud, Infrastructure and all that stuff in-between
My ramblings on the stuff that holds it all together
Category Archives: vCloud
Career vMotion
Posted by on August 20, 2010
Apologies, my blog has been a bit quiet of late, some of this is due to a much overdue holiday but mainly because I have been nearing the end of my time with my current employer and have been very busy finishing up projects and documentation.
Ok, so here it is – I am going to work for VMware ! – yes, another blogger is going to a vendor! This wasn’t a simple decision for me but the chance was too good to pass on. I have been in my current role at ioko for 10 years (almost to the day) and it’s been great, I honestly have no serious complaints and have worked on some very interesting projects with great customers. We are parting company on excellent terms and I wish them well but it’s time for something different for me.
The next phase of my life will be working as a Senior Consultant within the VMware vCloud EMEA team on some very exciting technology. As my regular readers will know I’ve had a keen interest in all things cloud for some time and have been doing some practical work with the vCloud concept as a VMware customer for some time, now I’m making the wholesale transition to be closer to the technology by being part of VMware helping other people to implement it now, for real, in production.
It’s been interesting over the last year to watch the transition of so many people that I know within the virtualization community to work for VMware, EMC and other vendors, this is evidence of a real industry-wide demand for people that are motivated and able to learn technologies quickly but also those that have a good level of inter-personal and business skills.
In my experience those people are the ones who don’t just see it IT as a job, they are personally interested and thus motivated to learn through research, play and doing, many of those people keep blogs as a personal reference and a way of sharing their work with the community – I definitely fall into this category and it’s been great to see the explosion of people blogging about virtualization and associated technologies over the last 3 years.
There is a lot of consolidation going on within the industry and because things are moving so quickly the vendors are quickly becoming the place to be if you want to be working with the latest stuff or the most cutting-edge projects, significantly more so than it has been in the past.
Getting a position at VMware certainly isn’t an easy ride and as a side-result I actually feel better-prepared for my VCDX defence panel. I had a *lot* of interviews through the process and a mix of presentation, technical and hands-on scenario work, the fact that I have a blog didn’t get me the job, I had to work hard for it and I have a lot of real industry experience behind me which stood in my favour. There is no space for complacency if you want to work with the best 🙂
I won’t officially start at VMware until after VMworld SF, so I’ll still be in SF under my own steam and expense, I’ll be blogging as much as possible and will be at the usual events during my time there. You can even come and see the vTARDIS in action in the session I am sharing with Eric Siebert and Simon Seagrave V18328 "Building an Affordable vSphere Environment for a Lab or Small Business".
In closing, this certainly isn’t’ the end of vinf.net and I’d like to refer you to this post’s title if there is any doubt – I see this as vMotioning my career from one provider to another whilst maintaining a business as usual service – I hear VMware have a product that does that sort of thing 🙂
I’ll definitely keep blogging and I now have the advantage that I will be able to write about vCloud stuff once it’s GA’d, within the usual bounds of customer/project confidentiality – in exactly the same way I worked pre-VMware – this isn’t an official VMware blog/mouthpiece and it won’t become one – but it will definitely get more interesting! 🙂
Thanks for reading. and here’s to things to come 🙂
.jpg)
VMware Licensing for the vCloud..
Posted by on March 29, 2010
I have been involved with VMware’s vCloud programme since it was announced late 2008, I was part of getting involved as one of a handful of partners on the launch-day and I have been doing private cloud work with customers since early 2008.
Whilst I was somewhat disappointed that as a smaller partner we were left out as VMware pressed on with the major partners and availability of information and tools were limited, it did eventually evolve into a set of emerging technology and products that we got access to.
One part of this that has troubled me for some time is licensing of VMware products for cloud/IaaS offerings, as an accredited VMware hosting provider we have access to service provider licensing (SPLA) for VMware products.
The concept of SPLA is simple, it’s a monthly rental of a product – the core VMware products being ESX and vCenter; the hoster pays VMware a monthly fee based on the peak number of virtual machines running in that month regardless of how many ESX installations/sockets/cores there are – this is a simple cost model for a hoster as the general unit of charging for an end-customer is a virtual machine over a period of time (the cost of which is made up of a slice of the supporting hardware/software/service-level).
As a service provider you want to minimise capital expenditure wherever possible, particularly when it comes to a cloud/IaaS model as there is often no long-term contractual commitment for a customer, if they need 100 VM’s one month and 2 the following it’s your burden as a service provider to provide {via investment} that infrastructure and software; for smaller/niche/private cloud players this is an issue – unless you have a large-scale and diverse customer base this is a risky operation and often breaks the cloud business case for the end-customer as the provider has to build in recovery of investment.
Amazon do well in this space with EC2 because they have massive scale and diversity of customers across all industry sectors so most peaks and troughs of demand average out in the bigger picture (see: commoditization and utility – Nicholas Carr’s book “The big switch” covers this well, as does Cannonical’s Simon Wardley at various Cloudcamps)
In my experience this is a problem right now for service providers looking to build cloud services on vSphere/vCloud as very few of VMware’s ancillary products are available on their hosted/SPLA licensing scheme, last time I enquired it it is limited to ESX, View, vCenter and I believe SRM has recently been added.
Also, companies looking to build internal cloud services can only leverage this SPLA type of licensing when the service is hosted and resold from a service provider’s own data centre & infrastructure, rather than customer hosted equipment – this protects the revenue stream of service providers, but does it harm the longer term private cloud prospect, particularly with customers that have a regulatory or security need that prevents traditional outsourcing/hosting?
There are a number of products that would be ideal for cloud/IaaS providers to better manage and control their services, but these are currently only available on a perpetual basis with traditional per CPU socket licensing – this perpetual model is hard for smaller/niche service providers where the capacity planning cycle continually looks for ways to deliver the required service with less infrastructure or have requirements to re-provision servers between physical and virtual instances, or where kit is leased/rented to cover short-term projects and demands.
The products that I see being ideal for service providers, if they could be offered on a SPLA basis rather than perpetual are;
- AppSense – this is almost a no-brainer; its a great way to manage end-end SLA for application delivery in a cloud platform.
- Cisco NX1000V – most service providers have big, highly skilled network teams and complex multi-customer environments to manage; this is ideal Nexus territory
- Chargeback – although I understand this is being built into the vCloud billing services, this has been available now for a while, and is a workable product to build billing systems.
- Lab Manager – some of the tech seems to be getting merged into the vCloud services at some level, but self-service lab environments in a private-cloud scenario with some control over VM lease/retention is something people have been asking for.
- CapacityIQ – designed for balancing load and capacity, important to a multi-tenant environment
- ConfigControl (licensing not announced) – but you can definitely see the need for this in a large multi-tenant environment
Microsoft, to their credit have this absolutely nailed; their SPLA licensing model is very mature and has all the products from their vast catalogue, even if the Service Provider Usage Rights (SPUR) documents can get a bit complicated to read/digest, VMware have moved to a similar contractual model via resellers for the latest iteration of their hosted license agreement with a more flexible model to add products; but it’s not there yet.
Whilst from a commercial point of view SPLA licensing doesn’t have the “big deal” values in terms of upfront revenues to VMware’s sales force it’s a constant revenue stream, and generally everyone accepts that SPLA will be more expensive than perpetual licensing over a given period; but the increased cost reflects the flexibility/lack of commitment advantage.
Interestingly, it also makes it easier for service providers (and thus customers) to deploy VMware products in their own hosting platforms – easier to build a business case, less upfront sizing/capacity planning & forecasting, costs for growth are incremental and pure op-ex (it’s almost like a cloud within a cloud!).
So, by way of conclusion – whilst a lot of the products I’ve previously desired on SPLA (lifecycle manager, lab manager, Chargeback etc.) are being merged into the vCloud “product” for service providers VMware should consider offering everything on a SPLA basis to hosters, and maybe even consider such a licensing scheme for customers own internal usage to bolster the private cloud model.
Long Distance vMotion… heading to the vCloud
Posted by on June 30, 2009
VMware have an interesting proof of concept document posted online here, this is great progress for the platform and it can only be helped out by the close partnership with Cisco that has resulted in the NX1000V switch.
I’m no networking expert but to my understanding there are issues with extending Layer 2 networks across multiple physical locations that need to be resolved for this to be a safe configuration. to my limited understanding traditional technologies like spanning tree can present some challenges for inter-DC flat VLANs so they need to be designed carefully, maybe using MPLS as a more suitable inter-DC protocol.
The interesting part for me is that this will be the nirvana for VMware’s vCloud programme, where services can be migrated on/off-premise to/from 3rd party providers as required and without downtime. this is do-able now with some downtime via some careful planning and some tools but this proposition extends the vMotion zero downtime migration to vCloud.
As this technology and relevant VM/storage best-practice filters out of VMware and into service providers and customers this could become a supportable service offering for vCloud Service Providers.
To achieve this you still need storage access from both sites, to me the next logical step is to combine vMotion and FT technologies with some kind of host based replication or storage virtualization like the Datacore products. this will remove the dependency (and thus potential SPOF) on a single storage device for vMotion/FT.
Virtualizing/replicating the actual VM storage between different arrays and storage types (EMC—>HP, or even DAS—>EMC) and allowing (encapsulating) it over standard IP links rather than relying on complicated and proprietary array based replication and dedicated fibre connectivity is going to be a key success factor for vCloud, it’s interesting to see all the recent work on formalising FCoE along with other WAN-capable standards like iSCSI.
Some further reading on how I see “the cloud” evolving at a more practical level here
https://vinf.net/2009/01/08/what-is-the-cloud/
https://vinf.net/2008/06/23/virtualization-the-key-to-delivering-cloud-based-architecture-now/
Workload Portability: Ultimate Cloud Edition
Posted by on January 23, 2009
I like the PlateSpin range of products a lot, it really does let you take an OS instance + app stack (workload) and move it between different physical machines, hypervisors etc. in a low impact way – if you’ve not come across it before – read this post for more info I see this portability as one of the key infrastructure components if you are looking to build or manage your own internal cloud infrastructures.
This isn’t possible at present, but put your architect hat on and imagine if you could plug PlateSpin Migrate (previously known as PlateSpin PowerConvert) tool into Amazon’s EC2 cloud, or a VMWare vCloud based farm – then you could do whatever you like with your Windows and Linux servers.
By design AWS and vCloud are both supposed to be automatable with web services and APIs to control machine provisioning and control etc. EC2 seems to have all of this now (API docs and example) and vCloud is coming along. (more real details at VMworld I’m guessing).
Moving services between on and off-premise cloud infrastructures is a key concept of vCloud; but I’m guessing this will only be between vCloud based infrastructures, what if you wanted to take advantage of the capacity and scale/commodity pricing from big providers like EC2 (which is Xen based under the hood) to offload some of your internal services – to my mind, there are a couple of scenarios here that PlateSpin could fulfil;
- Disaster Recovery – using the cloud (EC2 or other) for DR capacity; pay per use – use PlateSpin Protect to sync your machine images off to Amazon S3 and have a “panic button” that converts the S3 hosted images to running AMI’s. Brent has a similar idea here around SQL, my proposition takes this to the next level and does it from the OS up; if you did have to move over to the EC2 hosted DR cloud, then you could use it to go back to physical hardware again once you’ve repaired/rebuild your internal infrastructure
- Data centre moves or serious maintenance – use a cloud like EC2 as “swing” capacity to run services whilst you pick up your DC hardware and move it somewhere else (rather than a kit refresh).
- Test & Development; the ability to sandbox new apps in EC2 could be attractive to some organisations where corporate policies hinder or prevent this type of innovation taking place in-house; What if you could do this externally then just bring the machine instances back in-house to put into internal production use (I’ve seen this happening at several customers) – of course IT security teams would probably not be to happy about it.
- Short-term Expansion Capacity; if you experience an occasional surge of demand or load for an internal service. For example; if you have an internal application that you know will get really hit for a promotion or project then you could clone instances of the relevant web/application servers off to EC2 and use some kind of very clever load balancing tech to selectively hand off load to EC2 hosted instances when internal servers start getting saturated – or vice-versa.
Maybe even if PlateSpin were to position their product as a web service itself with downloadable agents – a connector/conversion hub between clouds – now that’s an interesting proposition.
Hopefully this diagram explains some of this idea visually
Issues at present:
- PlateSpin doesn’t have an interface to EC2 (consider this my feature request :))
- There is no secure connectivity back to corp HQ – this is something that as far as I can see AWS has an issue with – out of the box there is no way to have say an IPSec VPN or dedicated private subnet managed and provided by EC2, complicated networking scenarios don’t seem to be possible – you could build your own using software based routers and firewalls on EC2 hosted server instances but this is host based – would be good if EC2 add this sort of service to the platform in future – that would definitely be a killer feature as far as I’m concerned – AWS team, consider this my feature request :))
- VM Persistence is something of an issue with EC2 and I don’t think the EC2 model currently deals with it; with EC2 you pay whilst an instance is running, if you terminate it; i.e switch it off, it’s gone – the data (and that includes OS/app configurations) that you build into the instance are lost. there is no way to archive/suspend/freeze an instance to S3 and “spin it up” as required – I’m guessing this would be feasible for Amazon to build into EC2/S3 – you pay per GB stored on S3 so there is a cost-model for it – again this would be a killer feature for me – there are ways obviously to make your instances “vanilla” and have them auto-install relevant code and data when they are created; examples here and here but that takes a lot of work and isn’t so simple for most corporate type apps.
- You can attach an EBS (Elastic Block Storage) volume to an instance, this is persisted (as long as you keep paying for it) and you can mount it to a single host as a block disk device – but the issue remains with the actual OS instance not being persisted. if its a Windows OS, this is a particular problem as the config is all held in the registry etc. which is part of the OS itself.
- This still doesn’t get you past the concerns/issues over data ownership and cloud security, there is no magic bullet in this respect, just risk management/mitigation.
Anyways. just an idea, feel free to comment and give me your feedback..
Cloud Wars: VMWare vs Microsoft vs Google vs Amazon Clouds
Posted by on October 1, 2008
A short time ago in a data centre, far far away…..
All the big players are setting out their cloud pitches, Microsoft are set to make some big announcements at their Professional Developer Conference at the end of October and VMWare made their VDC-OS announcements at VMWorld a couple of weeks ago, Google have had their App Engine in beta for a while and Amazon AWS is pretty well established.
With this post I hope to give a quick overview of each, I’ll freely admit I’m more knowledgeable on the VMWare/Microsoft offerings… and I stand to be corrected on any assumptions I’ve made on Google/AWS based on my web reading.
So, What’s the difference between them…?
VMWare vCloud – infrastructure led play
VMWare come from the infrastructure space, to-date they have dominated the x86 virtualization market, they have some key strategic partnerships with storage and network vendors to deliver integrated solutions.
The VMWare VDC-OS pitch is about providing a flexible underlying architecture through servers, network and storage virtualisation. why? because making everything ‘virtual’ makes for quick reconfiguration – reallocating resource from one service to another is a configuration/allocation change rather than requiring an engineer visit (see my other post on this for more info)
because VMWare’s pitch is infrastructure led it has a significant practical advantage in that it’s essentially technology agnostic (as long as it’s x86 based) you, or a service provider have the ability to build and maintain an automated birth–>death bare ‘virtual metal’ provisioning and lifecycle system for application servers/services as there is no longer a tight dependency for everything on physical hardware, cabling etc
There is no one size fits all product in this space so a bespoke solution based around a standard framework tool like Tivoli, SMS, etc. is typically required depending on organisational/service requirements.
No re-development is necessarily required to move your applications into a vCloud (hosted or internal) you just move your VMWare virtual machines to a different underlying VDC-OS infrastructure, or you use P2V, X2V tools like Platespin to migrate to a VDC-OS infrastructure.
In terms of limitations – apps can’t necessarily scale horizontally (yet) as they are constrained by their traditional server based roots. The ability to add a 2nd node doesn’t necessarily make your app scale – there are all kinds of issues around state, concurrency etc. that the application framework needs to manage.
VMWare are building frameworks to build scale-out provisioning tools – but this would only work for certain types of applications and is currently reactive unless you build some intelligence into the provisioning system.
Scott Lowe has a good round-up of VDC-OS information here & VMWare’s official page is online here
Google AppEngine– pure app framework play
An application framework for you to develop your apps within – it provides a vastly parallel application and storage framework – excellent for developing large applications (i.e Google’s bread & butter)
Disadvantage is it’s a complete redevelopment of you applications into Google compatible code, services & frameworks. You are tied into Google services – you can’t (as I understand it) take your developed applications elsewhere without significant re-development/porting.
The Google AppEngine blog is here
Microsoft Cloud Services Hosted Application stack & Infrastructure play
An interesting offering, they will technically have the ability to host .net applications from a shared hosting service, as well as integrating future versions of their traditional and well established office/productivity applications into their cloud platform; almost offering the subscription based/Software+Services model they’ve been mooting for a long time.
Given Microsoft’s market current dominance, they are very well positioned to make this successful as large shops will be able to modify existing internal .net services and applications to leverage portions of their cloud offering.
With the future developments of Hyper-V Microsoft will be well positioned to offer an infrastructure driven equivalent of VMWare’s VDC-OS proposition to service and support migration from existing dedicated Windows and Linux servers to an internal or externally hosted cloud type platform.
David Chou at Microsoft has a good post on Microsoft and clouds here
Amazon Web Services – established app framework with canned virtualization
the AWS platform provides a range of the same sort of functionality as Google AppEngine with SimpleDB, SQS and S3 but with the recently announced ability to run Windows within their EC2 cloud makes for an interesting offering with the existing ability to pick & choose from Linux based virtual machine instances.
I believe EC2 makes heavy use of Xen under the hood; which I assume is how they are going to be delivering the Windows based services, EC2 also allows you to choose from a number of standard Linux virtual machine offerings (Amazon Machine Image, AMI).
This is an interesting offering, allowing you to develop your applications into their framework and possibly port or build your Linux/Windows application services into their managed EC2 service.
Same caveat applies though, your apps and virtual machines could be tied to the AWS framework – so you loose your portability without significant re-engineering. on the flip-side they do seem to have the best defined commercial and support models and have been well established for a while with the S3 service.
Amazon’s AWS blog is available here
Conclusion
Microsoft & VMWare are best positioned to pick up businesses from the corporate’s who will likely have a large existing investment in code and infrastructure but are looking to take advantage of reduced cost and complexity by hosting portions of their app/infrastructure with a service-provider.
Microsoft & VMWare offerings easily lend themselves to this internal/external cloud architecture as you can build your own internal cloud using their off-the-shelf technology, something that isn’t possible with AWS or Google. This is likely to be the preferred model for most large businesses who need to retain ownership of data and certain systems for legal/compliance reasons.
leveraging virtualization and commercial X2V or X2X conversion tools will make transition between internal and external clouds simple and quick – which gives organisations a lot of flexibility to operate their systems in the most cost/load-effective manner as well as retain detailed control of the application/server infrastructure but freed up from the day-day hardware/capacity management roles.
AWS/Google are ideal for Web 2.0 ,start-ups and the SME sector where there is typically no existing or large code-base investment that would need to be leveraged. For a greenfield implementation these services offer low start-up cost and simple development tools to build applications that would be complicated & expensive to build if you had to worry about and develop supporting infrastructure without significant up-front capital backing.
AWS/Google are also great for people wanting to build applications that need to scale to lots of users, but without a deep understanding of the required underlying infrastructure, whilst this is appealing to corporate’s I think the cost of porting and data ownership/risk issues will be a blocker for a significant amount of time.
Google Apps are a good entry point for the SME/start-up sector and startups, and could well draw people into building AppEngine services as the business grows in size and complexity, so we may see a drift towards this over time. Microsoft have a competing model and could leverage their established brand to win over customers if they can make the entry point free/cheap and cross-platform compatible, lots of those SME/start-ups are using Mac’s or Netbooks for example.
VMWare vCloud
Posted by on September 15, 2008
The news is out, VMWare are building some very interesting technology frameworks to enable you to build your own cloud architectures, but also to be able to transition VMs from your environment to a service provider offering a hosted service and mix & match as required.
All very clever stuff, I’ve been working with VMWare on this for the last couple of weeks and it all links in nicely to an article I wrote a couple of months back on how VMWare can deliver this type of infrastructure now. nice to see it’s being “productized” and being explained as a concept to the world, I see Scott’s point and I also hope that people do realise it’s the underlying virtualization tech they are focusing on not some overarching end-end GoogleOS that does everything – although the clever bit is building management frameworks to allow another vendor to do this type of integration.
Read the vCloud page here and overview of the virtual data centre stuff here
Virtualization, Cloud, Infrastructure and all that stuff in-between 