Virtualization, Cloud, Infrastructure and all that stuff in-between
My ramblings on the stuff that holds it all together
Category Archives: EC2
Bring your own Windows license for EC2, but where are Microsoft?
Posted by on March 25, 2010
This is interesting, Amazon have announced a pilot scheme to allow customers with a Microsoft Enterprise agreement to port their Windows licenses to EC2, which they say reduces the average cost of a Windows EC2 instance by up to 41% (interesting to see where the infrastructure/software cost split is – maybe this is why open source tech is generally more popular in the cloud)
I’ve looked at this a fair bit – Microsoft have their SPLA service provider licensing, which is a monthly rental that EC2 must leverage for their licensing, and customers can obtain this sort of licensing from any accredited service provider as long as the license is used on and in provider owned equipment and datacentres – you can’t legally take those SPLA licenses and run them in-house, you need to re-license under a traditional Microsoft license agreement.
If you have a reasonably fixed Windows server footprint that you want to run on EC2 this is a good idea, or maybe lots of spare Windows licenses left over following an internal consolidation/virtualization programme.
If you need to scale up and down on a monthly basis (which is more akin to the typical EC2 use-case, although the minimum unit of purchase for SPLA is 1 month, EC2 works on the basis of hours) – SPLA is still probably more cost-effective as SPLA typically works out more expensive than a perpetual/EA type Windows license over 3 years, so you pay for that flexibility.
This is another offering in Amazon’s arsenal, combined with boot from EBS to tempt corporate customers into it’s IaaS platform. most enterprises don’t necessarily need all the “clever” EC2 features like automated scale-up and down but want somewhere easy and reliable to store their corporate systems.
You can see an interesting market opening up for IaaS cloud services between EC2 and vCloud resellers for enterprise Windows customers, but the big question is, where are Microsoft?
Do they have to have some kind of offering in the IaaS space in the pipeline (Azure is pure SaaS/PaaS at this stage); I’ve not seen Microsoft do any real GTM effort with partners around building this sort of cloud service like VMware have done with vCloud.
Microsoft have all the tools/software/frameworks in place in terms of Hyper-V/SCVMM but lack a productized offering and marketing back-up to partners (like BDD, but for servers & cloud) are they are starting to miss the boat?
Double-Take puts DR into the Cloud
Posted by on February 22, 2010
A colleague passed me this link today, Double-take have a new product offering allowing copies of app-servers to be replicated to and run on Amazon’s EC2 cloud service (register article here) – syncing disk writes in a delta fashion to an EC2 hosted AMI.
I suggested a similar architecture last year using Platespin, recent changes to EC2 to allow boot from elastic block storage (i.e persistent storage and private networking) make this a feasible solution, and as it’s pay per use you only pay for the EC2 instance(s) when they are running (i.e during a recovery situation).
You can read more about it here on the Double-Take site unfortunately their marketing department have coined another ‘aaS-ism’ in Recovery as a Service (RaaS) but we’ll forgive them as it’s a cool concept :).
There is a getting started guide here and it looks to operate on a many to one basis with one EC2 hosted instance of their software receiving delta changes from protected hosts over a VPN and writing them out to EBS volumes; if you need to recover a server an new EC2 instance is invoked and boots from the EBS volume containing replicas of your data, presumably inserting appropriate EC2 virtual h/w driver into the image at boot time (essentially P2V or V2V conversion).
My quick calculations; for a Windows 2008 server with a moderate amount of data (not factoring any client-side de-dupe) initial sync would transfer approx 15Gb into EC2 charges here – they vary by region so you can do your own figures EBS storage charges, and, of course; the initial sync might take a while depending on your internet connection.
If you are a *NIX admin you are probably thinking, huh, so what? copy data to S3 and just start-up a new AMI with the software and config you need and off you go; this solution seems targeted to Windows servers, where this sort of P2V, V2V recovery is very. very complicated due to the proprietary (i.e non-text file based) way Windows stores its application and system configurations in the registry.
In conclusion they would seem to have pipped Platespin:Protect to the post on this one – I had some good conversations with Platespin’s CTO about this solution last year but I have to say I’ve not seen significant new functionality out of the Platespin product range since Novell acquired it which is a shame, Double-Take Cloud looks like an interesting solution – check it out, and being “cloud” it’s easy to take it for a test drive – you would do well to consider whatever data protection laws your business is bound by, however (the curse of the cloud).
Private Connectivity to Amazon EC2 – your own Private Cloud, in the Cloud
Posted by on August 26, 2009
VPN connectivity and private networking within EC2 are now available, this is great news – I mused on the possibilities of this sort of thing previously in this post.
This is a key step to gaining corporate acceptance, and proves that there is definitely still a use case and demand for a private cloud,
This new offering provides better opportunities for integrating internal systems with large-scale commodity service from people like Amazon, extending your own address space into EC2 opens up interesting opportunities for selective offloading and “cloud-bursting” of services as well as DR.
Private or shared/dedicated cloud infrastructures take the principals of public cloud computing (on-demand, pay as you go, scalability) and apply them to private infrastructure (along these lines through the adoption of virtualization technology) some people see this as a bit of a cheat, or not “real” cloud computing… however, in the real world* they are very appealing where outsourcing to a commodity provider isn’t an option due to regulatory, compliance or security issues and it can provide extra assurance levels because you have the ability to “look the service provider in the eye” via a traditional business relationship, rather than an anonymous entity on the web.
I like the quote “virtualization is a technology, cloud computing is a business model” and to me that means that you can apply that “cloud” business model internally or externally (chargeback/leasing/outsourcing), it really doesn’t matter – it’s just how you do the sums, not the technology.
See this post from the AWS team for more details, and some analysis from the hoff here.
<flame>*I define real world as not in the land of whiteboards, workshops and architectural models, but in the non green-field land of doing business, making money and delivering service </flame>
Workload Portability: Ultimate Cloud Edition
Posted by on January 23, 2009
I like the PlateSpin range of products a lot, it really does let you take an OS instance + app stack (workload) and move it between different physical machines, hypervisors etc. in a low impact way – if you’ve not come across it before – read this post for more info I see this portability as one of the key infrastructure components if you are looking to build or manage your own internal cloud infrastructures.
This isn’t possible at present, but put your architect hat on and imagine if you could plug PlateSpin Migrate (previously known as PlateSpin PowerConvert) tool into Amazon’s EC2 cloud, or a VMWare vCloud based farm – then you could do whatever you like with your Windows and Linux servers.
By design AWS and vCloud are both supposed to be automatable with web services and APIs to control machine provisioning and control etc. EC2 seems to have all of this now (API docs and example) and vCloud is coming along. (more real details at VMworld I’m guessing).
Moving services between on and off-premise cloud infrastructures is a key concept of vCloud; but I’m guessing this will only be between vCloud based infrastructures, what if you wanted to take advantage of the capacity and scale/commodity pricing from big providers like EC2 (which is Xen based under the hood) to offload some of your internal services – to my mind, there are a couple of scenarios here that PlateSpin could fulfil;
- Disaster Recovery – using the cloud (EC2 or other) for DR capacity; pay per use – use PlateSpin Protect to sync your machine images off to Amazon S3 and have a “panic button” that converts the S3 hosted images to running AMI’s. Brent has a similar idea here around SQL, my proposition takes this to the next level and does it from the OS up; if you did have to move over to the EC2 hosted DR cloud, then you could use it to go back to physical hardware again once you’ve repaired/rebuild your internal infrastructure
- Data centre moves or serious maintenance – use a cloud like EC2 as “swing” capacity to run services whilst you pick up your DC hardware and move it somewhere else (rather than a kit refresh).
- Test & Development; the ability to sandbox new apps in EC2 could be attractive to some organisations where corporate policies hinder or prevent this type of innovation taking place in-house; What if you could do this externally then just bring the machine instances back in-house to put into internal production use (I’ve seen this happening at several customers) – of course IT security teams would probably not be to happy about it.
- Short-term Expansion Capacity; if you experience an occasional surge of demand or load for an internal service. For example; if you have an internal application that you know will get really hit for a promotion or project then you could clone instances of the relevant web/application servers off to EC2 and use some kind of very clever load balancing tech to selectively hand off load to EC2 hosted instances when internal servers start getting saturated – or vice-versa.
Maybe even if PlateSpin were to position their product as a web service itself with downloadable agents – a connector/conversion hub between clouds – now that’s an interesting proposition.
Hopefully this diagram explains some of this idea visually
Issues at present:
- PlateSpin doesn’t have an interface to EC2 (consider this my feature request :))
- There is no secure connectivity back to corp HQ – this is something that as far as I can see AWS has an issue with – out of the box there is no way to have say an IPSec VPN or dedicated private subnet managed and provided by EC2, complicated networking scenarios don’t seem to be possible – you could build your own using software based routers and firewalls on EC2 hosted server instances but this is host based – would be good if EC2 add this sort of service to the platform in future – that would definitely be a killer feature as far as I’m concerned – AWS team, consider this my feature request :))
- VM Persistence is something of an issue with EC2 and I don’t think the EC2 model currently deals with it; with EC2 you pay whilst an instance is running, if you terminate it; i.e switch it off, it’s gone – the data (and that includes OS/app configurations) that you build into the instance are lost. there is no way to archive/suspend/freeze an instance to S3 and “spin it up” as required – I’m guessing this would be feasible for Amazon to build into EC2/S3 – you pay per GB stored on S3 so there is a cost-model for it – again this would be a killer feature for me – there are ways obviously to make your instances “vanilla” and have them auto-install relevant code and data when they are created; examples here and here but that takes a lot of work and isn’t so simple for most corporate type apps.
- You can attach an EBS (Elastic Block Storage) volume to an instance, this is persisted (as long as you keep paying for it) and you can mount it to a single host as a block disk device – but the issue remains with the actual OS instance not being persisted. if its a Windows OS, this is a particular problem as the config is all held in the registry etc. which is part of the OS itself.
- This still doesn’t get you past the concerns/issues over data ownership and cloud security, there is no magic bullet in this respect, just risk management/mitigation.
Anyways. just an idea, feel free to comment and give me your feedback..
Amazon EC2 Web Console
Posted by on January 12, 2009
I’ve been doing a bit of playing about with Amazon’s EC2 cloud services recently, this is just a quick post with some screenshots of the new beta web console they have launched.
Up until now you had to control it via command line, or a firefox plug in – now Amazon have launched their own… seems very easy to use and understand – available online here
You can browse the list of pre-configured AMI’s (Amazon VM Images) and choose which ones you want to spin up.
I’m writing up some posts on using EC2 for quick ‘n’ dirty test and development environments, but I can see a lot of potential for this service to provide automated overspill capacity for applications using the automation API and some clever management tools.
Cloud Wars: VMWare vs Microsoft vs Google vs Amazon Clouds
Posted by on October 1, 2008
A short time ago in a data centre, far far away…..
All the big players are setting out their cloud pitches, Microsoft are set to make some big announcements at their Professional Developer Conference at the end of October and VMWare made their VDC-OS announcements at VMWorld a couple of weeks ago, Google have had their App Engine in beta for a while and Amazon AWS is pretty well established.
With this post I hope to give a quick overview of each, I’ll freely admit I’m more knowledgeable on the VMWare/Microsoft offerings… and I stand to be corrected on any assumptions I’ve made on Google/AWS based on my web reading.
So, What’s the difference between them…?
VMWare vCloud – infrastructure led play
VMWare come from the infrastructure space, to-date they have dominated the x86 virtualization market, they have some key strategic partnerships with storage and network vendors to deliver integrated solutions.
The VMWare VDC-OS pitch is about providing a flexible underlying architecture through servers, network and storage virtualisation. why? because making everything ‘virtual’ makes for quick reconfiguration – reallocating resource from one service to another is a configuration/allocation change rather than requiring an engineer visit (see my other post on this for more info)
because VMWare’s pitch is infrastructure led it has a significant practical advantage in that it’s essentially technology agnostic (as long as it’s x86 based) you, or a service provider have the ability to build and maintain an automated birth–>death bare ‘virtual metal’ provisioning and lifecycle system for application servers/services as there is no longer a tight dependency for everything on physical hardware, cabling etc
There is no one size fits all product in this space so a bespoke solution based around a standard framework tool like Tivoli, SMS, etc. is typically required depending on organisational/service requirements.
No re-development is necessarily required to move your applications into a vCloud (hosted or internal) you just move your VMWare virtual machines to a different underlying VDC-OS infrastructure, or you use P2V, X2V tools like Platespin to migrate to a VDC-OS infrastructure.
In terms of limitations – apps can’t necessarily scale horizontally (yet) as they are constrained by their traditional server based roots. The ability to add a 2nd node doesn’t necessarily make your app scale – there are all kinds of issues around state, concurrency etc. that the application framework needs to manage.
VMWare are building frameworks to build scale-out provisioning tools – but this would only work for certain types of applications and is currently reactive unless you build some intelligence into the provisioning system.
Scott Lowe has a good round-up of VDC-OS information here & VMWare’s official page is online here
Google AppEngine– pure app framework play
An application framework for you to develop your apps within – it provides a vastly parallel application and storage framework – excellent for developing large applications (i.e Google’s bread & butter)
Disadvantage is it’s a complete redevelopment of you applications into Google compatible code, services & frameworks. You are tied into Google services – you can’t (as I understand it) take your developed applications elsewhere without significant re-development/porting.
The Google AppEngine blog is here
Microsoft Cloud Services Hosted Application stack & Infrastructure play
An interesting offering, they will technically have the ability to host .net applications from a shared hosting service, as well as integrating future versions of their traditional and well established office/productivity applications into their cloud platform; almost offering the subscription based/Software+Services model they’ve been mooting for a long time.
Given Microsoft’s market current dominance, they are very well positioned to make this successful as large shops will be able to modify existing internal .net services and applications to leverage portions of their cloud offering.
With the future developments of Hyper-V Microsoft will be well positioned to offer an infrastructure driven equivalent of VMWare’s VDC-OS proposition to service and support migration from existing dedicated Windows and Linux servers to an internal or externally hosted cloud type platform.
David Chou at Microsoft has a good post on Microsoft and clouds here
Amazon Web Services – established app framework with canned virtualization
the AWS platform provides a range of the same sort of functionality as Google AppEngine with SimpleDB, SQS and S3 but with the recently announced ability to run Windows within their EC2 cloud makes for an interesting offering with the existing ability to pick & choose from Linux based virtual machine instances.
I believe EC2 makes heavy use of Xen under the hood; which I assume is how they are going to be delivering the Windows based services, EC2 also allows you to choose from a number of standard Linux virtual machine offerings (Amazon Machine Image, AMI).
This is an interesting offering, allowing you to develop your applications into their framework and possibly port or build your Linux/Windows application services into their managed EC2 service.
Same caveat applies though, your apps and virtual machines could be tied to the AWS framework – so you loose your portability without significant re-engineering. on the flip-side they do seem to have the best defined commercial and support models and have been well established for a while with the S3 service.
Amazon’s AWS blog is available here
Conclusion
Microsoft & VMWare are best positioned to pick up businesses from the corporate’s who will likely have a large existing investment in code and infrastructure but are looking to take advantage of reduced cost and complexity by hosting portions of their app/infrastructure with a service-provider.
Microsoft & VMWare offerings easily lend themselves to this internal/external cloud architecture as you can build your own internal cloud using their off-the-shelf technology, something that isn’t possible with AWS or Google. This is likely to be the preferred model for most large businesses who need to retain ownership of data and certain systems for legal/compliance reasons.
leveraging virtualization and commercial X2V or X2X conversion tools will make transition between internal and external clouds simple and quick – which gives organisations a lot of flexibility to operate their systems in the most cost/load-effective manner as well as retain detailed control of the application/server infrastructure but freed up from the day-day hardware/capacity management roles.
AWS/Google are ideal for Web 2.0 ,start-ups and the SME sector where there is typically no existing or large code-base investment that would need to be leveraged. For a greenfield implementation these services offer low start-up cost and simple development tools to build applications that would be complicated & expensive to build if you had to worry about and develop supporting infrastructure without significant up-front capital backing.
AWS/Google are also great for people wanting to build applications that need to scale to lots of users, but without a deep understanding of the required underlying infrastructure, whilst this is appealing to corporate’s I think the cost of porting and data ownership/risk issues will be a blocker for a significant amount of time.
Google Apps are a good entry point for the SME/start-up sector and startups, and could well draw people into building AppEngine services as the business grows in size and complexity, so we may see a drift towards this over time. Microsoft have a competing model and could leverage their established brand to win over customers if they can make the entry point free/cheap and cross-platform compatible, lots of those SME/start-ups are using Mac’s or Netbooks for example.
Performance Expectations on a Shared Virtual Platform
Posted by on March 12, 2008
Some interesting discussion linked-to within this article on Storagezilla. people using Amazon’s EC2 platform are complaining because they feel they are getting less performance than they should.
Always an interesting point to bear in mind and useful in expectation setting for developers. You may want a dedicated CPU/core – but do you really need all of that CPU all of the time? in most cases I would guess not; and if you do need that level of performance – shouldn’t you be considering a physical platform rather than a virtual one?
Virtualization, Cloud, Infrastructure and all that stuff in-between 