Virtualization, Cloud, Infrastructure and all that stuff in-between
My ramblings on the stuff that holds it all together
Category Archives: Microsoft
Windows Azure under the hood
Posted by on October 30, 2008
There is a an excellent video interview with Manuvir Das from the Azure team on the MSDN Channel 9 site here.
The interview is quite long, but I’ve tried to summarise it for infrastructure people/architects like me as follows;
Azure is an overall “OS” for the cloud, akin to VMWare and their VDC initiative but with a much richer re-usable services and applications framework layer.
In terms of describing the overall architecture diagram (below), Azure is sort of the”kernel for the cloud”, “Xbox for the cloud?” buy it in increments and (ab)use it – don’t worry about building the individual infrastructure components – you get all the tools in the box and the underlying infrastructure is abstracted so you don’t have to worry about it.
The services layer Microsoft provide on top of Azure are as follows
Live Services Mesh (high level user/data sync – will run as app on Azure, doing some now) will be migrated to run on Azure over time
.net services (Zurich) high level services to enable rich scenarios like authentication, Federation, liveID, OpenID, Active Directory Federation Services etc.
SQL – premium Database services in the cloud offering data warehousing, and I would assume massive scalability options – but I’m not sure how this would be implemented.
Sharepoint/Dynamics I understand are coming soon but would offer the same sort of functionality in the cloud.
It’s based around modified Windows with Dave Cutler’s involvement (no specifics offered yet) virtualized server instances are the base building blocks with an allocated and guaranteed amount of resource – 1×1.9GHz CPU, 2gb ram, 160gb disk) which is dedicated to your machine and not contended, which would mean MS are doing no over-subscription under the hood? that seems unlikely, and maybe wasteful to me; DRS anyone?
Dell have provided the underlying physical hardware hosted in Microsoft’s data centres with a customised server model, as noted here – and you can see a video tour inside one of the hosting data centres here from BBC news
There is an overall Fabric Controller which is essentially a resource manager, it continually monitors hosts, VMs, storage via agents and deploys/allocates/moves .net code packages around hosts.
to deploy your service to the Azure cloud;
You build your application as a code package (.net, others coming later)
You build a service model, this describes the number, type of hosts, dependencies etc.
The Azure storage layer a distributed, flat table-based storage system with a a distributed lock manager and keeps 3 copies of data for availability – it’s not SQL based (interesting) uses a REST API and is more akin to a file system so sounds like it’s been written from the ground up.
Interestingly it seems that the storage layer is deployed as a service on Azure itself and is controlled by the fabric manager, parts of the current live mesh services are using it now in production.
Interestingly Manuvir describes your service as containing routers, load balancers as well as traditional services so it sounds like they may have either built a complex provisioning framework for physical devices, or have implemented virtualized versions of such devices (Cisco Nexus type devices implemented as VM’s maybe?)
Azure can maintain staging and production platforms within the cloud, you can swap between production/stage etc. with an API command that re-points DNS.
There is a concept of an upgrade domain; where VMs are taken out of service for updates/deployments etc. – your service description I assume describes what are key dependencies and it works out the least-impact sequence?
No automatic paralellism, you can’t just issue a job and have it execute in a distributed fashion using all the Azure resources without being designed/built as such, which I think Amazon offer (but I may be wrong, as that does sound like something v.complicated to do)
Azure strategy for scale out is the traditional MS one, make the most use of individual resource allocation for your VMs (see above), scale out multiple independent instances with a shared nothing architecture
Azure is a programmable API, it’s not an end-user product, it’s a platform for developers to build services on.
There is no absolute requirement for asp.net will provide PHP/RoR/Python facilities over time and .net and visual studio integration out of the box – but can use other developer tools too.
A “Developer fabric” is available – it can run on a desktop, it mocks up the whole Azure platform on your desktop and behaves the same way so developers can understand how it works and debug applications on their desktops before pushing out to the cloud – this is an important shiny for Microsoft, as it’s a simple and quick way to get developers hands-on with understanding how to use Azure.
The cool part is that you can export your service model and code packages directly to Azure from your developer tool, akin to a compile and public option for the cloud. it’s part of SDK which can be downloaded here.
You can debug service copies locally using the SDK and developer fabric, no debugging in the cloud {yet} but provides an API to get logs and are working on an end-end transaction tracing API
Microsoft have made references to making Azure on-premise as well as in Microsoft’s own data centres in the same way that VMWare have with the VDC-OS stuff… but I would think that’s going to need some more details on what the Azure OS is to understand how that would be feasible.
As I concluded in an earlier blog post here, Microsoft could be poised to clean up here if they execute quickly and well – they have the most comprehensive offering for the corporate space due to having a very rich applications/services layer that is directly aligned to the desktop & application technology choices of the bigger customers (.net), they just need to solve the trust in the cloud issue first; and the on-premise piece of the puzzle is key to this… Maybe a server version of Windows 7 or MiniWin or Singularity is the enabler for this?
Microsoft Moves into the Clouds
Posted by on October 28, 2008
As you’ve probably seen and I mentioned here earlier Microsoft are laying out their vision for Microsoft-centric cloud computing this week at their Professional Developers Conference.
If you’re short of time to understand this there is a good quick overview here, here and here, apologies for lack of posting recently which has been due to the awful cold I’ve had and a backlog of “real” work to deal with.
I’m attending Microsoft TechEd next week in Barcelona, so I’m hoping to get more real information about how this will work in the real world and I’ll be blogging as much of that content as possible.
Not sure I can live up to the level of posts Scott managed earlier in the year at TechEd US but I’ll try 🙂
Cloud is the new Mesh 🙂
Cloud Wars: VMWare vs Microsoft vs Google vs Amazon Clouds
Posted by on October 1, 2008
A short time ago in a data centre, far far away…..
All the big players are setting out their cloud pitches, Microsoft are set to make some big announcements at their Professional Developer Conference at the end of October and VMWare made their VDC-OS announcements at VMWorld a couple of weeks ago, Google have had their App Engine in beta for a while and Amazon AWS is pretty well established.
With this post I hope to give a quick overview of each, I’ll freely admit I’m more knowledgeable on the VMWare/Microsoft offerings… and I stand to be corrected on any assumptions I’ve made on Google/AWS based on my web reading.
So, What’s the difference between them…?
VMWare vCloud – infrastructure led play
VMWare come from the infrastructure space, to-date they have dominated the x86 virtualization market, they have some key strategic partnerships with storage and network vendors to deliver integrated solutions.
The VMWare VDC-OS pitch is about providing a flexible underlying architecture through servers, network and storage virtualisation. why? because making everything ‘virtual’ makes for quick reconfiguration – reallocating resource from one service to another is a configuration/allocation change rather than requiring an engineer visit (see my other post on this for more info)
because VMWare’s pitch is infrastructure led it has a significant practical advantage in that it’s essentially technology agnostic (as long as it’s x86 based) you, or a service provider have the ability to build and maintain an automated birth–>death bare ‘virtual metal’ provisioning and lifecycle system for application servers/services as there is no longer a tight dependency for everything on physical hardware, cabling etc
There is no one size fits all product in this space so a bespoke solution based around a standard framework tool like Tivoli, SMS, etc. is typically required depending on organisational/service requirements.
No re-development is necessarily required to move your applications into a vCloud (hosted or internal) you just move your VMWare virtual machines to a different underlying VDC-OS infrastructure, or you use P2V, X2V tools like Platespin to migrate to a VDC-OS infrastructure.
In terms of limitations – apps can’t necessarily scale horizontally (yet) as they are constrained by their traditional server based roots. The ability to add a 2nd node doesn’t necessarily make your app scale – there are all kinds of issues around state, concurrency etc. that the application framework needs to manage.
VMWare are building frameworks to build scale-out provisioning tools – but this would only work for certain types of applications and is currently reactive unless you build some intelligence into the provisioning system.
Scott Lowe has a good round-up of VDC-OS information here & VMWare’s official page is online here
Google AppEngine– pure app framework play
An application framework for you to develop your apps within – it provides a vastly parallel application and storage framework – excellent for developing large applications (i.e Google’s bread & butter)
Disadvantage is it’s a complete redevelopment of you applications into Google compatible code, services & frameworks. You are tied into Google services – you can’t (as I understand it) take your developed applications elsewhere without significant re-development/porting.
The Google AppEngine blog is here
Microsoft Cloud Services Hosted Application stack & Infrastructure play
An interesting offering, they will technically have the ability to host .net applications from a shared hosting service, as well as integrating future versions of their traditional and well established office/productivity applications into their cloud platform; almost offering the subscription based/Software+Services model they’ve been mooting for a long time.
Given Microsoft’s market current dominance, they are very well positioned to make this successful as large shops will be able to modify existing internal .net services and applications to leverage portions of their cloud offering.
With the future developments of Hyper-V Microsoft will be well positioned to offer an infrastructure driven equivalent of VMWare’s VDC-OS proposition to service and support migration from existing dedicated Windows and Linux servers to an internal or externally hosted cloud type platform.
David Chou at Microsoft has a good post on Microsoft and clouds here
Amazon Web Services – established app framework with canned virtualization
the AWS platform provides a range of the same sort of functionality as Google AppEngine with SimpleDB, SQS and S3 but with the recently announced ability to run Windows within their EC2 cloud makes for an interesting offering with the existing ability to pick & choose from Linux based virtual machine instances.
I believe EC2 makes heavy use of Xen under the hood; which I assume is how they are going to be delivering the Windows based services, EC2 also allows you to choose from a number of standard Linux virtual machine offerings (Amazon Machine Image, AMI).
This is an interesting offering, allowing you to develop your applications into their framework and possibly port or build your Linux/Windows application services into their managed EC2 service.
Same caveat applies though, your apps and virtual machines could be tied to the AWS framework – so you loose your portability without significant re-engineering. on the flip-side they do seem to have the best defined commercial and support models and have been well established for a while with the S3 service.
Amazon’s AWS blog is available here
Conclusion
Microsoft & VMWare are best positioned to pick up businesses from the corporate’s who will likely have a large existing investment in code and infrastructure but are looking to take advantage of reduced cost and complexity by hosting portions of their app/infrastructure with a service-provider.
Microsoft & VMWare offerings easily lend themselves to this internal/external cloud architecture as you can build your own internal cloud using their off-the-shelf technology, something that isn’t possible with AWS or Google. This is likely to be the preferred model for most large businesses who need to retain ownership of data and certain systems for legal/compliance reasons.
leveraging virtualization and commercial X2V or X2X conversion tools will make transition between internal and external clouds simple and quick – which gives organisations a lot of flexibility to operate their systems in the most cost/load-effective manner as well as retain detailed control of the application/server infrastructure but freed up from the day-day hardware/capacity management roles.
AWS/Google are ideal for Web 2.0 ,start-ups and the SME sector where there is typically no existing or large code-base investment that would need to be leveraged. For a greenfield implementation these services offer low start-up cost and simple development tools to build applications that would be complicated & expensive to build if you had to worry about and develop supporting infrastructure without significant up-front capital backing.
AWS/Google are also great for people wanting to build applications that need to scale to lots of users, but without a deep understanding of the required underlying infrastructure, whilst this is appealing to corporate’s I think the cost of porting and data ownership/risk issues will be a blocker for a significant amount of time.
Google Apps are a good entry point for the SME/start-up sector and startups, and could well draw people into building AppEngine services as the business grows in size and complexity, so we may see a drift towards this over time. Microsoft have a competing model and could leverage their established brand to win over customers if they can make the entry point free/cheap and cross-platform compatible, lots of those SME/start-ups are using Mac’s or Netbooks for example.
Microsoft now Officially support many of their products under ESX 3.5u2
Posted by on September 5, 2008
As noted here and here, VMWare have had ESX 3.5u2 certified under Microsoft’s SVVP programme, this is excellent news and will knock down one of the long standing barriers to greater adoption of virtualisation as I wrote about here – support.
Most notably for me this means blessed support of Exchange 2007sp1 running under ESX!
Excellent work to get this done so quickly – MS only announced the SVVP programme a short while ago.
Official list of MS products supported under VMWare is here.
Handy Reference Chart for Microsoft Server Application Licences
Posted by on July 2, 2008
Taken from a download on the Microsoft Partner Licencing Specialist site, the following diagram makes for a useful quick reference chart for what licencing options are applicable to the big MS Server apps – far easier than having to check the product sites and documentation individually if you are trying to spec something up.
Also lots more useful information on this site – it’s designed to train people to become Microsoft licencing specialists (MLSS/MLSE) it’s mainly sales staff orientated training, but some useful/easy to digest reference material for techies/consultants alike if you’ve ever struggled to understand Microsoft licencing.
Useful links..
Revision Presentations – .PDF files to download https://partner.microsoft.com/UK/40033119
Training Videos – downloadable http://www.microsoft.com/uk/partner/learningpaths/?id=licensing.mlss
Microsoft Blogger on Licencing
Posted by on May 24, 2008
This is a great blog, from one of the licencing managers at Microsoft – Emma has made a good start on providing easy to understand articles on MS licencing in bite sized chunks.
It’s not everyone’s favourite topic, but we all have to do it 🙂
Useful for virtualisation people as there’s a whole section on it with pictures some further hilights (for me) on SPLA licencing, SQL and System Centre
Welcome Emma, keep up the good work – even if you are forced to adopt the American spelling of the word for your main job role like me 😉
Thanks to James for the original link.
VMWare Workstation 6.5 Beta – Run Multiple Copies of Outlook/Exchange via Unity
Posted by on April 30, 2008
I use a single laptop for my day-day use, it has all the stuff i need, I run Vista and Office 2007, for our corporate mail we use Exchange like everyone else and I use Outlook Cached Mode to work online/offline..
My own personal email is also an Exchange mailbox – provided by fasthosts (why – well, because..ok?) the problem with this is that I can’t have a single copy of Outlook connected to more than one Exchange server at the same time or run multiple instances of Outlook (I’ve tried all the hacks and Thinstall etc.), and to be honest even if I could it would probably violate the security policies of all the involved organisations as it would be quite simple for an Outlook-aware worm to try to propagate itself across multiple organisations or harvest confidential details.
The problem is further compounded by the fact that I often work on long-term customer projects and have to have a mailbox on their Exchange system as well… which leads to multiple diary sync nightmare, maybe I’ll blog about that some other time).
So at present I have 4 Exchange mailboxes that I need to keep track of, auto-forwarding mail between them is a no-no, I used to be an Exchange admin and I’ve lost many bank holidays due to corporate->Hotmail NDR mail loops!
So, up until now I’ve had to run one full Outlook client and multiple OWA clients in a browser, which is ok as long as I’m connected to the Internet, but no good if I’m on a train unless I want to close and restart Outlook with multiple profiles, which is a pain especially when you are collaborating on a project between multiple organisations. To be honest as good as OWA 2003 is it’s no substitute for a full outlook client. (still waiting for Fasthosts to go to Exchange 2007, oh and enable EAS!).
So, anyway a solution – VMWare Unity, this is a feature like Parallels for the Mac which lets you “float” an application window out of a guest VM to the host desktop meaning you can use the applications without working within a single VM’d desktop window.
VMWare Fusion also has the same feature, but Workstation 6.5 is the 1st time its been available on the PC platform.
To use Unity you need to have upgraded the virtual machine to 6.5 “hardware” by right clicking on the VM in the sidebar pane (below) and install the latest VM Tools – it also only seems to support XP at present, or at least it didn’t work on the Server 2003 VM I had.
Boot the VM… and install the latest VM tools.
VM Workstation Screen – note VM is set to “Unity mode”
My Vista desktop (yes, I have the start bar at the right hand side – widescreen laptop!) with the popup menu for the VM, showing all the start menu for applications installed within in it.
the following screen shot is Calculator running from inside the XP VM but in a single window on the Vista desktop – note the red border and the 
icon, denoting that its presented via Unity.
It even shows up on the start bar with the correct icon; although this doesn’t seem to work until its been run a couple of times; I assume it needs to cache an icon or something.
it also seems to respect the window snapshots you get whilst Win-Tab between applications, even for pop-up windows
Technically I can use this to run n x Windows XP/Outlook 2003 VM’s presenting Outlook through to my Vista desktop and comply with all organisations security policies, as each VM and its respective copy of Outlook runs in isolation from each other with the relevant company-specific AV client (or at worst, the same level as if I were using a machine connected to a public network in that they all share a vm network) – I don’t enable shared folders between the VMs.
It’s still a beta feature at the moment, and there seem to be a few bugs particularly when resizing windows sometimes it doesn’t work properly and double clicking to expand to full screen overlays the start-bar on my vista machine.
And it does seem to get confused sometimes and not allow keyboard input, so you have to flick back to non-unity mode and then back to continue, and sometimes a reboot of the guest VM but it is an early build so I would guess this will be resolved.
As an added bonus VM Workstation seems to allow the Vista host OS to go into sleep mode even whilst VMs are running, this is something I’ve not had much luck with in the past – it would generally refuse to sleep when I closed the lid (but thats not a scientific comparison… it may have just been bad luck!)
So, the pay-off – 2 copies of Outlook (2003 and 2007) seemingly running on the same desktop, alt-tab works ok and you have access to all the functionality of both without having to switch between or run multiple OWA sessions and from a security perspective it’s not really any different from having 2 physical PCs in front of you (slight memory overhead, but my laptop has 4Gb RAM, so not a huge issue).
Opening attachments is obviously going to be a bit of an issue, as you’ll technically need an individually licenced instance of Office 2003 in each VM as they can’t (yet) exchange data between them… and that would compromise the security principal.
VMWare Server Performance – A Practical Example
Posted by on April 19, 2008
The following screen dump is from an HP DL380G5 server that runs all the core infrastructure under VMWare Server (the free one) for a friend’s company which I admin sometimes.
It is housed in some co-lo space and runs the average range of Windows servers used by a small but global business, Exchange SQL, Windows 2003 Terminal Services.
As a result of some planned (but not very well communicated!) power maintenance the whole building lost power earlier today, when it was restored I grabbed the following screenshot as the 15 or so Virtual Machines automatically booted.
interesting to note that all the VM’s had been configured to auto-start with the guest OS, meaning there wasn’t any manual intervention required, even though it was a totally dirty shutdown for both the host and guest OS’es (No UPS, as the building and suite is supposed to have redundant power feeds to each rack – in this instance the planned maintenance was on the building wiring so required taking down all power feeds for a 5 yearly inspection..)
There are no startup delay settings in the free version of VMWare Server so they all start at the same time, interesting to note the following points..
The blue line that makes a rapid drop is the pages/second counter, and the 2nd big drop (green) is the disk queue length. the hilighted (white) line is the overall %CPU time, note the sample frequency was 15 seconds on this perfmon.
After it had settled down, I took the following screenshot, it hardly breaks a sweat during its working day. there are usually 10-15 concurrent users on this system from around the world (access provisioned via an SSL VPN device) and a pretty heavily used Exchange mail system.
The box is an HP DL380 G5 with 2 x quad core CPUs (8 cores in total) and 16Gb of RAM, it has 8 x 146Gb 15k HDDs in a single RAID 5 set + hot-spare, it was purchased in early 2007 and cost c.£8,000 (UK Prices)
It runs Windows 2003 Enterprise Edition x64 edition with VMWare Server 1.0.2 (yes, its an old build.. but if it ain’t broke..) and they have purchased multiple w2k3 ent-edition licences to take advantage of the virtualisation use-rights to cover the installed virtual OS’es.
It’s been in-place for a year and hardly ever has to be touched, its rock-solidly available and the company have noticed several marked improvements since they P2V’d their old servers onto this platform, as follows;
- No hardware failures – moving from lots of low-end servers (Dell) and desktops to a single box (10:1 consolidation)
- The DL380 has good redundancy built in, but it’s also backed up with a h/w maintenence contract, and they also have a spare cold-standby server to resume service from backups if data is lost.
- Less noise, the old servers were dotted around their old offices in corners, racks etc – this is the main thing they liked!
- Simple access anywhere – using a Juniper SA2000 SSL VPN, its easy to get secure access from anywhere
- Less reliance on physical offices and cheap DSL-grade data communications, now the servers are hosted on the end of a reliable, data centre class network link with an SLA to back it up. if an individual office looses its ADSL connection, no real issue – people pick up their laptop(s) and work from home/starbucks etc.
- Good comms are cheaper in data centres than in your branch offices (usually)
Hopefully this goes to show the free version of VMWare’s server products can work almost as well if budget is a big concern, ESX would definitely give some better features and make backup easier, they are considering upgrading and combining with something like Veeam Backup to handle failover/backup.
New Microsoft Data Centre is Container Based
Posted by on April 2, 2008
Article here, it’s coming people!
Some interesting discussions on how you can measure the productivity of a container and come up with some common metrics to compare and contrast and handle charge-back.
How to Overlay Multiple Calendars in Outlook 2007
Posted by on March 25, 2008
This is a really useful feature; I didn’t realise you could do this (until I had a need to do so, thanks Google!); makes it much easier to look and manage multiple calendars (or people’s shared calendars).
Linky here
Shame the UI for this isn’t particularly obvious, or at least maybe I’m blind but the button is quite small 🙂
Virtualization, Cloud, Infrastructure and all that stuff in-between 
