Virtualization, Cloud, Infrastructure and all that stuff in-between

Following on from some discussion on Scott Lowe’s blog around the lack of a clear definition of cloud computing, I offer this as my opinion… it’s just that; and I’d welcome comments. I’m an infrastructure chap by trade but my motto is there are no apps without infrastructure, and this cloud stuff is all about making things easier for developers to create useful stuff quickly and cheaply but it needs somewhere to run.

Sam Johnstone has also done some work on formalising this stack, but I offer the following up for discussion.

A cloud is just a pool of flexible/on-demand “resource” a way of abstracting the  underlying complexities of how things are executed or stored, provisioned etc.

This is analogous to the way the modern computer, OS and applications have developed over the last 20 years into multiple abstraction layers – meaning a developer no longer has to explicitly know how to control devices at a low level, for example; moving bits in and out of registers, reading/writing sectors to disks. They make API calls down a stack to BIOS, firmware, operating systems, drivers, code libraries (.DLL etc.) and more recently moving to web services at the top level interface.

Presently solution/infrastructure architects work with individual servers, roles and services on bits of dedicated hardware which are often bound to a specific location, ensuring that they architect the solution to deliver the required level of availability, serviceability etc.

I see “the cloud” as a way of providing the next level of abstraction – with an eventual goal of architects being able to design systems with no roles or services tied to specific servers/databases/hardware/datacentres/continents(!) where SOA-type applications and transactions are executed across one or more cloud platforms without having to have a detailed understanding of the underlying infrastructure.

In the interim the cloud can deliver a platform to quickly deploy and change infrastructure to deliver applications, right-sizing capacity based on actual usage rather than over-sizing done early into the development cycle of an application.

Fewer hard physical ties to the local infrastructure that supports application components, combined with adoption of virtualization of servers, networks etc. means that you can relocate services locally, nationally or globally through data and configuration migration rather than a traditional lift & shift of servers, switches, SAN’s racks etc. with the associated risk and downtime etc.

With standardisation or adoption of a common infrastructure architecture this could allow for a real market place to develop, where the customer can choose the most appropriate or cost-effective region or service provider to run their application(s) – either based on the cost of local power, comms, or response time, legal jurisdiction or SLA without being tied to a specific service provider or physical location.

For example; some of my previous posts on this sort of physical DC portability are here, if you combine this with a high level of virtualization and the cloud reference architecture you have a compelling solution for large infrastructures, Zimory also have an interesting proposition for brokering resources between multiple cloud providers

There are two fundamental things required to deliver this nirvana…

1) Flexible Infrastructure Architecture (do-able now with current tech )

This is where I see my cloud reference architecture sitting, you could have multiple instances of this architecture split between on/off/3rd party premise providers – this provides a layer of abstraction between the physical hardware/networking/site world and an “application” or server instance (as it’s encapsulated in a VM – which is just persisted as a file on some storage.

2) Distributed Runtime/Services Layer (work starting now, needs development and standardisation)

To enable the cloud to really distribute applications (and thus computing power) across the Internet means you have to build a distributed or entirely democratic/autonomous controller mechanism (almost an OS for the cloud) which acts as a compiler, API, interpreter, Job Controller etc. to execute and manage applications (code/apps/scripts) that developers produce

This distributed runtime /services layer runs on server instances hosted on/in a cloud infrastructure (see item 1) that are managed by a service provider, to my mind there is no other way to achieve this you can’t easily write an app and just have it run across multiple locations without some kind of underlying abstraction layer taking care of the complexities of state, storage, parallelism etc. this is where Microsoft’s Azure, Amazon AWS & Google have API’s for things like databases , payment gateways, messaging, storage across their distributed infrastructures.

However all of them are doing it in a proprietary way – Azure/AWS provide their own API rather than a standardised set of services that true cloud apps can be written to and then moved between various cloud infrastructures or providers.

It’s important to note that the two are not mutually exclusive, clouds based on this reference architecture can still run traditional static one server/one app type services, even desktop/VDI but it also maintains server instances that run the actual distributed runtime/services layer for applications that are written to take advantage of it – there is a common underlying platform.

This model and standardisation helps address the concerns of most businesses when they start to talk about clouds, data ownership and security by giving them the opportunity to selectively adopt cloud technologies and transition workloads between on/off-premise clouds as needs dictate.