Archive for the ‘Windows’ Category

h1

Windows 7 Remote Desktop Client – Nice Touch

May 19, 2010

 

You can download and install the new, updated RDC 7.0 client for free for any OS from Windows XP SP3 and later (it ships with Windows 7) known issues here and detailed feature comparison per OS here and downloads for various client OS’es are below

Update for Windows Vista, x86-based versions

DownloadDownload the Update for Windows Vista for x86-based systems package now.

Update for Windows Vista, x64-based versions

DownloadDownload the Update for Windows Vista for x64-based systems package now.

Update for Windows XP, x86-based versions

DownloadDownload the Update for Windows XP for x86-based systems package now.

A nice touch that i discovered by accident is that you can move the top title bar along the top of a full-screen RDP session by clicking and dragging, this is really handy if you work on multiple full screen RDP sessions inside another RDP session – for example a jump-off box to a protected subnet or when you have a full-screen application.

image

image

AkA Terminal Services client, Remote Desktop Protocol Client, RDS Client, Remote Desktop Client, RDC, RDP, RDS :)

Enjoy

h1

Redesigning Active Directory for 2010 and on..

October 3, 2009

 

Active Directory has been implemented as part of Windows since approx 1998 when the betas of the initial Windows 2000 version were circulating. At the time directory services was Microsoft’s answer to all NT4 scalability woes and the superior management that Novell offered in Netware 4.x, that was a radically different IT world {cue Waynes world flash back}

  • Most people worked in a set of fixed locations, mobile workers were by far the minority
  • Those fixed locations had full all-ports network access to corporate resources internal network and/or personal firewalls were unheard of.
  • People who needed remote access to the network came in by dial-up or VPN type access with token or user/password type authentication
  • Starbucks was NOT your office :)
  • your PC/laptop was owned by the company and you had less need to keep your personal on-line life running during work time or using work-resources (you shopped in real shops and people still used the phone to communicate)
  • Viruses were there but the most prevalent forms propagated by infected documents and emails.
  • Network connectivity was slow and/or expensive from remote locations

I’ve worked with Active Directory in a lot of depth during this time and it’s an excellent and flexible tool, however it’s now 2009 and whilst Active Directory has been enhanced over this time it isn’t radically different in terms of supporting the way we work today.

There is still a very tight integration* between a workstation (domain member) and the Domain/Forest – this relies on periodical machine account password changes.

  • All authentication and group policy type activities like interactive logon, policy downloads etc. still require a large number of ports and RPC services to function – this makes firewalls like swiss-cheese, and doesn’t work well in locations with latent or slow network connections (although there are tweaks; most of these involve turning off GPO processing on slow links).
  • To provide remote access to domain and corporate services a VPN layer is required to provision access, this is ok but a large part of the Windows interactive logon process still requires access to a domain controller at the CTRL-ALT-DEL logon screen – support for this is hacky at best when you are not on a full all-ports open network connection to the corporate domain – 3rd parties have custom GINA code that allows you to initiate a VPN connection before the logon is processed but it’s not a one-stop shop and users still *just don’t get it*.
  • Disconnected machines (like roaming sales people) rely heavily on cached credentials, these credentials are only refreshed when you make an interactive logon to the corporate network – which requires VPN, large number of port rules; machine hygiene routines etc.
  • User profiles/folder redirections don’t work particularly well in long-term disconnected scenarios and it’s difficult to maintain a consistent user profile environment for these users.

If you’ve ever had to re-build a user’s machine whilst disconnected from the network this can be a real issue.

*Machines can only be part of one domain at a time, they rely heavily on it for authentication and control.

Building standalone/workgroup machines is one answer but you have no way of managing any of the machines, tracking them, distributing configurations etc. – there is too much all or nothing and there is no middle ground in Active Directory at present – and this also makes multi-tier firewalled application platforms problematic – do you put in multiple domains to support tiers/DMZ’s or compromise security and use a single domain and wider firewall rules? if you put in workgroup machines manging security across all of them is problematic, some Microsoft products (Exchange, etc.) require an Active Directory domain and change is difficult.

In addition, high-speed Internet access is now very common and the move to “the cloud” is underfoot, with end-user devices being little more than very clever terminals.

Microsoft have made moves to support single sign on through web applications with the Active Directory Federation Services (AD-FS) in Windows 2008 but this is still geared at web applications rather than the core authentication and application services Microsoft’s desktop and server OS relies on for normal operations.

This is a list of the things I would like to see in future Active Directory and/or add-on endpoint security checkers to better support the upcoming generations of users who won’t always be on the corporate LAN, or purchase and use their own PC/laptop as well as the needs that virtualization and dynamic scaling infrastructure requires.

  • Move authentication services to HTTP/S interfaces and away from RPC and dynamic ports.
  • Make the group policy services available over the same HTTP/S interfaces
  • This has already been done for Outlook/Exchange via the RPC over HTTP/S interface – Active Directory could use a similar concept for allowing access from external/edge services.
  • Introduce a further class of machine to compliment the traditional “computer” account; an “external managed machine” (or similar) – where it isn’t necessarily a direct member of the domain but you allow a degree of trust – maybe leveraging the AD Federation Services, no local passwords held but hashed with the core AD service with an intermediate service (or core-OS component) to facilitate authentication between applications and the AD to maintain backwards compatibility for anything that runs locally and relies on traditional Windows authentication.
  • Allow all communication between these external managed devices and core infrastructure over HTTP/S – so as to be tolerant of latent connections and carried over common network services.
  • Allow those managed external machines to be locally administered/installed/maintained etc. (think of the Windows Mobile Phone or iPhone model that is used to allow access to Exchange email but give it a representative object in Active Directory that can be managed through policies or even disabled – even if that object is just a certificate for the device or some other representation it should be accessible through the AD tools and scripting interfaces.
  • Add support for configuration compliance scanning for external managed devices (end-point security) and centralised reporting – some of this is in next gen ISA tools.
  • Support for transient (often virtual..) machines that are dynamically added to a domain and removed – think of the VDI model where hundreds of machines could be created and destroyed automatically – leaving hundreds of “dead” machine accounts and reboots to support the domain join operations.
  • Support and manage a corporate PC “out on the Internet" as if it were in the office (..using web services/HTTP wrappers) much like we can with Outlook 2003+ and Exchange 2003+ using RPC over HTTP/s – no complicated and difficult to use local VPN client

What would you like to see?

As an addendum; Apologies for the lack of posting recently on vinf.net which has been due to the arrival of our second child, which as you might imagine has taken up a lot of my blogging time! hopefully will get a bit more time in the coming months to support my habit!

h1

Is your MS Application Supported under VMware, Hyper-V, Xen? – the DEFINITIVE Statement from Microsoft

August 14, 2009

 

A colleague has just made me aware of a new tool on the Microsoft website, it is a wizard that can tell you if specific Microsoft App/OS/Architecture combinations are supported under the SVVP (Server Virtualization Validation Programme) – I previously wrote about the SVVP here, which promised to resolve many of the pains we were experiencing.

The output from the SVVP programme has been compiled into a great web based wizard that saves all the previous leg work of reading several (sometimes conflicting) whitepapers.. here you get it straight from the horses mouth (so to speak).

You can access the Wizard via this Link

http://www.windowsservercatalog.com/svvp.aspx?svvppage=svvpwizard.htm

The wizard lists all Microsoft products

image

The list of hypervisor platforms supported is shown below, and you can choose the OS version (Windows 2000 and later) and the CPU architecture (x86, x64 etc.)

image image

And, finally the most important part – a definitive statement on support for this combination

image image

Excellent work Microsoft – come on other vendors (Oracle, Sun this means you…)

h1

ExPrep – Script to Automate Exchange 2007 Pre-Requisite Installation

May 13, 2009

 

If you have ever had to install Exchange 2007 on a Windows 2008 (and 2003) server you will know that there are a number of pre-requisites that need to be installed from the OS for each role; for example IIS web services and metabase compatibility components.

You have two choices, do this via the UI using the add/remove features and roles Wizard in Server Manager or using the ServerManagerCmd.EXE command line utility – either way it’s pretty tedious to do if you have several servers to install.

Based on this handy reference from Microsoft I have built a very basic batch file that automates the installation of the pre-req components for you.

It only works on Windows 2008 (sorry no 2003 equivalent) and you use it entirely at your own risk – there are much cleverer ways of scripting this but I’m a pretty old skool DOS person, this works for me and is easy for me to maintain – feel free to re-write in something more modern and post it back here this code is probably quite hacky.

The contents of the file are here (just cut & paste into a .bat file)

@echo off

REM ExPrep.bat by Simon Gallagher, ioko (http://vinf.net)
REM Usage Instructions and further information here
REM http://vinf.net/2009/05/13/exprep-script-to-automate-exchange-2007-pre-requisite-installation/

REM YOU USE THIS SCRIPT ENTIRELY AT YOUR OWN RISK
REM
REM Version 1.0

SET %EXPREP%=999
echo 1…….Mailbox Role (non-clustered)
echo 2…….Mailbox Role (Clustered)
echo 3…….Client Access Server (CAS)
echo 4…….Hub Transport (HT)
choice /C 1234
SET EXPREP=%ERRORLEVEL%
echo you chose %EXPREP%
pause

echo Preparing for base pre-req install

ServerManagerCmd -i Web-Metabase
ServerManagerCmd -i Web-Lgcy-Mgmt-Console
ServerManagerCmd -i Web-Server
ServerManagerCmd -i Web-ISAPI-Ext
ServerManagerCmd -i Web-Metabase
ServerManagerCmd -i Web-Lgcy-Mgmt-Console
ServerManagerCmd -i Web-Basic-Auth
ServerManagerCmd -i Web-Digest-Auth
ServerManagerCmd -i Web-Windows-Auth
ServerManagerCmd -i Web-Dyn-Compression
ServerManagerCmd -i PowerShell

echo you chose %EXPREP%

if %EXPREP%==1 goto MBX
if %EXPREP%==2 goto MBX-CLUSTER
if %EXPREP%==3 goto CAS
if %EXPREP%==4 goto HT

goto end

:MBX
echo preparing for Mailbox Role (non-clustered)
pause
ServerManagerCmd -i Web-Server
ServerManagerCmd -i Web-ISAPI-Ext
ServerManagerCmd -i Web-Metabase
ServerManagerCmd -i Web-Lgcy-Mgmt-Console
ServerManagerCmd -i Web-Basic-Auth
ServerManagerCmd -i Web-Windows-Auth

goto end

:MBX-CLUSTER
echo preparing for Mailbox Role (clustered)
pause
ServerManagerCmd -i Web-Server
ServerManagerCmd -i Web-ISAPI-Ext
ServerManagerCmd -i Web-Metabase
ServerManagerCmd -i Web-Lgcy-Mgmt-Console
ServerManagerCmd -i Web-Basic-Auth
ServerManagerCmd -i Web-Windows-Auth

ServerManagerCmd -i Failover-Clustering

goto end

:CAS
echo preparing for CAS role
pause
ServerManagerCmd -i Web-Server
ServerManagerCmd -i Web-ISAPI-Ext
ServerManagerCmd -i Web-Metabase
ServerManagerCmd -i Web-Lgcy-Mgmt-Console
ServerManagerCmd -i Web-Basic-Auth
ServerManagerCmd -i Web-Digest-Auth
ServerManagerCmd -i Web-Windows-Auth
ServerManagerCmd -i Web-Dyn-Compression

ServerManagerCmd -i RPC-over-HTTP-proxy

goto end

:HT
echo preparing for HT role
pause
Echo nothing extra needed, PowerShell done already
goto end

:END
echo Done

Instructions:

1) Copy the script (ExPrep.bat) to your would-be Exchange server (remember Windows 2008 x64 is the only supported OS for Exchange 2007).

2) Run ExPrep.bat

3) Choose the appropriate role from the menu (note: there is no clever input validation – make sure you choose the correct one, there are pause statements before it actually does anything so you can CTRL-C to break out.

clip_image002

4) Sit back and wait for it to complete.

5) then run the Exchange 2007 installer from your DVD or network share as normal.

If you need to install multiple roles on a single server you can run the script multiple times, all changes are cumulative and if a component is already installed ServerManagerCmd.EXE (which the script calls) will just skip it.

If you wanted to take it further there is some excellent information about the setup process, failures and doing full unattended installations of Exchange 2007 here and here

Remember you use this entirely at your own risk, and you assume full responsibility for checking its suitability for your environment; the batch file is easy to read and customize for your own use, although I ask that if  you do make changes link back here via a comment or trackback so that other people can benefit.

h1

Windows 7 and the Intel 855GM Video Driver "Solution"

January 12, 2009

 

I’ve been playing about with Windows 7 in a VM for a while now in a VM, but now the beta is out I wanted to install it on a physical machine, I’m not ready yet to upgrade my main laptop to Windows 7 (although I have a cunning plan to p2v my Vista install and convert to a VHD so I can dual-boot that way which is a neat trick)

image

I have a Dell Inspiron 510m laptop that I use for testing things (I used it for my Patespin series) that I wanted to install Windows 7 on, it still gives pretty good performance and has 2Gb RAM – the installation itself went smoothly and quickly – less than 45mins from format to finished 1st boot, but it doesn’t detect the wireless or video card.

In my experience this isn’t that unusual for a Dell, although video did surprise me as Vista had a default driver for the Intel 855GM on-board video that worked well, there is no built-in driver in Windows 7 it would seem.

So, a bit of a problem – I’m stuck with 640×480 VGA mode which isn’t much use.

I tried several ways to hack the Vista version of the driver into my installation, all without success – it always defaulted back to the default VGA drivers, some discussion here if you are interested

In the end I came across a post suggesting that I use an application called DriverMax – this is capable of exporting and importing installed drivers, I’d not tried it before but decided to give it a go, I know Vista had a working 855GM driver so the plan was to export it from there, and import it into a Windows 7 installation as I was unsure of how to extract it from the Vista installation media.

This necessitated a format and reinstall of the Dell 510m with Vista, which was painless enough as I had an auto-install DVD that I’d previously built

Once Vista was installed there was a working video driver running – I used DriverMax to export the working driver from the running OS – no source or driver CD required via a couple of clicks in the UI to a .zip file on a USB drive.

I then formatted and reinstalled Windows 7 again and on the laptop and installed DriverMax again.

then I simply imported the driver from the .zip file

image image

Note – it knows the driver I saved was a default Windows driver

image

Summary screen – important to note it can install unsigned drivers if required

image

After a reboot the Windows 7 installation is running with a working (full-res) video driver.

I did find one slight problem with DriverMax that I had to work-around, with the default VGA video driver the buttons on the dialog boxes were inaccessible and I couldn’t resize or hot-key around it to progress, so in the end I had to do the process via remote desktop to the Win7 machine from another machine on my network over a wired LAN connection!

It’s not an ideal solution as you have to have a working Vista installation to extract the driver from and is probably totally unsupported, this is essentially Windows 7 running a Vista video driver – but it’s a beta anyway, hopefully MS or Intel will ship an 855GM driver again when Windows 7 goes RTM.

My initial impressions are that Windows 7 seems a lot more responsive than Vista, although to be fair it’s a vanilla installation thus-far. I have high-hopes for the beta, by my reckoning the change in the code-base isn’t as fundamental as it was between XP and Vista so it’s more focused on incremental features and performance improvements. I ran beta copies of Vista on my main work machine from Beta 1 through to RTM without too many problems, maybe I’ll be confident enough to do that again this time around – the VHD booting feature is certainly compelling for what I do.

h1

Windows OS Code Patching

October 22, 2008

 

Interesting article here from the ntdebug blog on how hotfixes get integrated into the windows code-base and update mechanism.

There have been some excellent posts recently on this blog offering detailed insight into the internals of Windows, if you’re interested in this kind of thing (like me) and general innards of Microsoft I’d also recommend Raymond Chen’s blog.

Many people underestimate the complexity of getting Windows out the door and keeping it serviced, I have to wonder just how well Apple* would cope given a similar scale of operation, and not having the luxury of a single “blessed” hardware platform rather than having to service literally trillions of combinations of 3rd party hardware/software/firmware/drivers etc.

I’ve seen lots of “Windows is rubbish and my Mac is ace” discussions at work and socially recently, whilst Windows definitely has its flaws, a more detailed analysis of the persons problem usually reveals that its a 3rd party app/device/driver that has caused a problem, for example;

  • Outdated DivX codec giving poor performance when browsing directories with thumbnails, or crashing – fix – updated codec
  • Vendor supplied wireless driver/utilities causing issues with sleep or disabling network card – using default Windows driver was as performant and fixed all issues

Microsoft get a lot of bad press around this but it’s actually because they have a pretty open framework and set of ISV/IHV/partner schemes to allow 3rd parties to tightly integrate their products (and thus profit from the Windows cash-cow) they have their HCL/SCL process, but it’s not an absolute requirement for being allowed to install product X from ABC inc.

*Not wishing to start a Mac/PC war – I use + like both, before you flame me, although I have used OSX under VMWare, as well as on Apple hardware #naughty!

h1

Cloud Wars: VMWare vs Microsoft vs Google vs Amazon Clouds

October 1, 2008

 

A short time ago in a data centre, far far away…..

All the big players are setting out their cloud pitches, Microsoft are set to make some big announcements at their Professional Developer Conference at the end of October and VMWare made their VDC-OS announcements at VMWorld a couple of weeks ago, Google have had their App Engine in beta for a while and Amazon AWS is pretty well established.

With this post I hope to give a quick overview of each, I’ll freely admit I’m more knowledgeable on the VMWare/Microsoft offerings… and I stand to be corrected on any assumptions I’ve made on Google/AWS based on my web reading.

So, What’s the difference between them…?

VMWare vCloud - infrastructure led play

VMWare come from the infrastructure space, to-date they have dominated the x86 virtualization market, they have some key strategic partnerships with storage and network vendors to deliver integrated solutions.

The VMWare VDC-OS pitch is about providing a flexible underlying architecture through servers, network and storage virtualisation. why? because making everything ‘virtual’ makes for quick reconfiguration – reallocating resource from one service to another is a configuration/allocation change rather than requiring an engineer visit (see my other post on this for more info)

because VMWare’s pitch is infrastructure led it has a significant practical advantage in that it’s essentially technology agnostic (as long as it’s x86 based) you, or a service provider have the ability to build and maintain an automated birth–>death bare ‘virtual metal’ provisioning and lifecycle system for application servers/services as there is no longer a tight dependency for everything on physical hardware, cabling etc

There is no one size fits all product in this space so a bespoke solution based around a standard framework tool like Tivoli, SMS, etc. is typically required depending on organisational/service requirements.

No re-development is necessarily required to move your applications into a vCloud (hosted or internal) you just move your VMWare virtual machines to a different underlying VDC-OS infrastructure, or you use P2V, X2V tools like Platespin to migrate to a VDC-OS infrastructure.

In terms of limitations – apps can’t necessarily scale horizontally (yet) as they are constrained by their traditional server based roots. The ability to add a 2nd node doesn’t necessarily make your app scale – there are all kinds of issues around state, concurrency etc. that the application framework needs to manage.

VMWare are building frameworks to build scale-out provisioning tools – but this would only work for certain types of applications and is currently reactive unless you build some intelligence into the provisioning system.

Scott Lowe has a good round-up of VDC-OS information here & VMWare’s official page is online here

Google AppEngine- pure app framework play

An application framework for you to develop your apps within – it provides a vastly parallel application and storage framework – excellent for developing large applications (i.e Google’s bread & butter)

Disadvantage is it’s a complete redevelopment of you applications into Google compatible code, services & frameworks. You are tied into Google services – you can’t (as I understand it) take your developed applications elsewhere without significant re-development/porting.

The Google AppEngine blog is here

Microsoft Cloud Services Hosted Application stack & Infrastructure play

An interesting offering, they will technically have the ability to host .net applications from a shared hosting service, as well as integrating future versions of their traditional and well established office/productivity applications into their cloud platform; almost offering the subscription based/Software+Services model they’ve been mooting for a long time.

Given Microsoft’s market current dominance, they are very well positioned to make this successful as large shops will be able to modify existing internal .net services and applications to leverage portions of their cloud offering.

With the future developments of Hyper-V Microsoft will be well positioned to offer an infrastructure driven equivalent of VMWare’s VDC-OS proposition to service and support migration from existing dedicated Windows and Linux servers to an internal or externally hosted cloud type platform.

David Chou at Microsoft has a good post on Microsoft and clouds here

Amazon Web Services – established app framework with canned virtualization

the AWS platform provides a range of the same sort of functionality as Google AppEngine with SimpleDB,  SQS and S3 but with the recently announced ability to run Windows within their EC2 cloud makes for an interesting offering with the existing ability to pick & choose from Linux based virtual machine instances.

I believe EC2 makes heavy use of Xen under the hood; which I assume is how they are going to be delivering the Windows based services, EC2 also allows you to choose from a number of standard Linux virtual machine offerings (Amazon Machine Image, AMI).

This is an interesting offering, allowing you to develop your applications into their framework and possibly port or build your Linux/Windows application services into their managed EC2 service.

Same caveat applies though, your apps and virtual machines could be tied to the AWS framework – so you loose your portability without significant re-engineering. on the flip-side they do seem to have the best defined commercial and support models and have been well established for a while with the S3 service.

Amazon’s AWS blog is available here

Conclusion

Microsoft & VMWare are best positioned to pick up businesses from the corporate’s who will likely have a large existing investment in code and infrastructure but are looking to take advantage of reduced cost and complexity by hosting portions of their app/infrastructure with a service-provider.

Microsoft & VMWare offerings easily lend themselves to this internal/external cloud architecture as you can build your own internal cloud using their off-the-shelf technology, something that isn’t possible with AWS or Google. This is likely to be the preferred model for most large businesses who need to retain ownership of data and certain systems for legal/compliance reasons.

leveraging virtualization and commercial X2V or X2X conversion tools will make transition between internal and external clouds simple and quick – which gives organisations a lot of flexibility to operate their systems in the most cost/load-effective manner as well as retain detailed control of the application/server infrastructure but freed up from the day-day hardware/capacity management roles.

AWS/Google are ideal for Web 2.0 ,start-ups and the SME sector where there is typically no existing or large code-base investment that would need to be leveraged. For a greenfield implementation these services offer low start-up cost and simple development tools to build applications that would be complicated & expensive to build if you had to worry about and develop supporting infrastructure without significant up-front capital backing.

AWS/Google are also great for people wanting to build applications that need to scale to lots of users, but without a deep understanding of the required underlying infrastructure, whilst this is appealing to corporate’s  I think the cost of porting and data ownership/risk issues will be a blocker for a significant amount of time.

Google Apps are a good entry point for the SME/start-up sector and startups, and could well draw people into building AppEngine services as the business grows in size and complexity, so we may see a drift towards this over time. Microsoft have a competing model and could leverage their established brand to win over customers if they can make the entry point free/cheap and cross-platform compatible, lots of those SME/start-ups are using Mac’s or Netbooks for example.

h1

Mapping a drive to a VSS Snapshot & General DFS-R woes

September 14, 2008

 

Microsoft’s volume snapshot service is pretty handy right? quick hardware independent snaps of a file system – all free and out of the box, well it’s now officially saved my bacon…. whilst it’s a bit klunky (more on this in a bit) it was damned useful.

I had a pain of a problem to deal with this weekend, helping out a friend doing some server re-organising (plan was to migrate these guys from VMWare Server 1.x to ESXi – but didn’t get that far due to some other Windows issues that took all of our time as we checked everything was ok before the move)

Firstly, if you use DFS-R (as comes with Win2003 R2) never, ever, ever, ever use the “distributed file system” applet to administer DFS, we needed to add a new replica of a large DFS-R set to another server and because (in our defence) the server was a fresh R2 install, we forgot to install the newer DFS-R components via control panel, but original DFS was still installed by default and we were in a hurry (read: not paying attention) we used the “Distributed File System” applet to add a new target, and followed the wizard which actually re-created the DFS volume (note to self – pay more attention when clicking!) from scratch.

It proceeded to delete all the contents of all the DFS shares and moved them to a folder called NtFrs_PreExisting___See_EventLog and started afresh, that wouldn’t be so bad except for some inexplicable reason it then purged the contents of that folder from all replicas so we had no quick cut & paste file copy solution.

This was not going to be a fun weekend.

Don’t use this one image

 imageimage 

use this one!image

 image image

 

So, basically it was our (my) fault – but it was compounded by some weird corruption in one of the directories that looked like it had been there a while that meant recovery wasn’t going to be straightforward.

The data Backup was about 24hrs older than the last VSS snapshot on the central file server (hub & spoke replication topology) so as we now had a flat, deleted DFS volume with no data (thanks!) we decided to try and revert to the most recent VSS snapshot for the relevant directories.

But no dice, it just threw an error – can’t copy, I can view the files and see the contents and can drag and drop one or two a time, but any more and it would throw an error.

Not good, I can only assume that this was because of some logical corruption within the file system as there was one whole directory tree I couldn’t access (more on how I recovered this later).. there were over 60k files so I wasn’t going to do that by hand – so a command line was in order as at least XCopy can ignore errors etc. and just pull out the good data.

I found these excellent articles here and here and documentation here but some of them were more geared towards taking a snapshot and extracting data in-situ rather than from a persistent snapshot like you get with VSS.

so, none of them worked for me ; and even a lot of hacking with Vshadow and MOUNTVOL I couldn’t get the VSS Snap to mount at all and time was short

I did discover the following though, if you view a snapshot using the Previous Versions tab (remember this only works if you browse for files to restore via UNC path) it opens the snap in Explorer, but you can’t map a drive to it or run a command line copy against it…. or can’t you :)

When you open it in explorer this way it does create a sort of hidden temporary share – easiest way I found to expose the name of the share was to try and zip a file in the explorer session that is looking at the snapshot using WinZip, if you follow the wizard at some point it will expose a UNC path like \\SERVERNAME@GMT-DD-MM-YY-{GUID} if you can cut & paste that you can then map a network drive to it

NET USE * \\servername@gmt-dd-mm-yy-{guid}

And you can then run xcopy etc against that mapped drive to copy out all the good data – in reality we used SyncBackSE – which is great for complex file copies and we already had it installed.

All of these Windows servers were installed as VM’s in VMWare Server(s), so it actually made our lives a lot easier as we could quickly clone a known-broken server as-is (do no further harm) and then spin it up disconnected from the network to recover data using this method and also undeleted files using Get Data Back NTFS etc. and then use that data to re-seed the DFS-R volume – but much easier than if it were a physical box and at no real risk of making things worse.

So, in conclusion this was human error, rather than a 100% technical problem and should have been better planned and prevented by maintenance and a better recovery plan- but here it is, with the solution we found to get things back in all its gory details… and mainly as a footnote so I don’t make the DFS mistake again and in my defence this is a shoe-string charity operation rather than a blue-chip org with significant money and time to invest in such efforts.

This solution worked for us, but you need to have your own tried & tested solution – don’t rely on this as far as I can tell it’s unsupported, use at your own risk!

h1

Virtualization – the key to delivering "cloud based architecture" NOW.

June 23, 2008

 

There is a lot of talk about delivering cloud or elastic computing platforms, a lot of CxO’s are taking this all in and nodding enthusiastically, they can see the benefits.. so make it happen!….yesterday.

Moving your services to the cloud, isn’t always about giving your apps and data to Google, Amazon or Microsoft.

You can build your own cloud, and be choosy about what you give to others. building your own cloud makes a lot of sense, it’s not always cheap but its the kind of thing you can scale up (or down..) with a bit of up-front investment, in this article I’ll look at some of the practical; and more infrastructure focused ways in which you can do so.

image

Your “cloud platform” is essentially an internal shared services system where you can actually and practically implement a “platform” team that operates and capacity plans for the cloud platform; they manage it’s availability and maintenance day-day and expansion/contraction.

You then have a number of “service/application” teams that subscribe to services provided by your cloud platform team… they are essentially developers/support teams that manage individual applications or services (for example payroll or SAP, web sites etc.), business units and stakeholders etc.

Using the technology we discuss here you can delegate control to them over most aspects of the service they maintian – full access to app servers etc. and an interface (human or automated) to raise issues with the platform team or log change requests.

I’ve seen many attempts to implement this in the physical/old world and it just ends in tears as it builds a high level of expectation that the server/infrastructure team must be able to respond very quickly to the end-”customer” the customer/supplier relationship is very different… regardless of what OLA/SLA you put in place.

However the reality of traditional infrastructure is that the platform team can’t usually react as quick as the service/application teams need/want/expect because they need to have an engineer on-site, wait for an order and a delivery, a network provisioning order etc. etc (although banks do seems to have this down quite well, it’s still a delay.. and time is money, etc.)

Virtualization and some of the technology we discuss here enable the platform team to keep one step ahead of the service/application teams by allowing them to do proper capacity planning and maintain a pragmatic headroom of capacity and make their lives easier by consolidating the physical estate they manage. This extra headroom capacity can be quickly back-filled when it’s taken up by adopting a modular hardware architecture to keep ahead of the next requirement.

Traditional infrastructure = OS/App Installations

  • 1 server per ‘workload’
  • Silo’d servers for support
  • Individually underused on average = overall wastage
  • No easy way to move workload about
  • Change = slow, person in DC, unplug, uninstall, move reinstall etc.
  • HP/Dell/Sun Rack Mount Servers
  • Cat 6 Cables, Racks and structured cabling

The ideal is to have an OS/app stack that can have workloads moved from host A to host B; this is a nice idea but there are a whole heap of dependencies with the typlical applications of today (IIS/apache + scripts, RoR, SQL DB, custom .net applications). Most big/important line of business apps are monolithic and today make this hard. Ever tried to move a SQL installation from OLD-SERVER-A to SHINY-NEW-SERVER-B? exactly. *NIX better at this, but not that much better.. downtime required or complicated fail over.

This can all be done today, virtualization is the key to doing it – makes it easy to move a workload from a to b we don’t care about the OS/hardware integration – we standardise/abstract/virtualize it and that allows us to quickly move it – it’s just a file and a bunch of configuration information in a text file… no obscure array controller firmware to extract data from or outdated NIC/video drivers to worry about.

Combine this with server (blade) hardware, modern VLAN/L3 switches with trunked connections, and virtualised firewalls then you have a very compelling solution that is not only quick to change, but makes more efficient use of the hardware you’ve purchased… so each KW/hr you consume brings more return, not less as you expand.

Now, move this forward and change the hardware for something much more commodity/standardised

Requirement: Fast, Scalable shared storage, filexible allocation of disk space and ability to de-duplicate data, reduce overhead etc, thin provisioning.

Solution: SAN Storage, EMC Clariion, HP-EVA, Sun StorageTek, iSCSI for lower requirements, or storage over single Ethernet fabric – NetApp/Equalogic

Requirement: Requirement Common chassis and server modules for quick, easy rip and replace and efficient power/cooling.

Solution: HP/Sun/Dell Blades

Requirement: quick change of network configurations, cross connects, increase & decrease bandwidth

Solution: Cisco switching, trunked interconnects, 10Gb/bonded 1GbE, VLAN isolation, quick change enabled as beyond initial installation there are fewer requirements to send an engineer to plug something in or move it, Checkpoint VSX firewalls to allow delegated firewall configurations or to allow multiple autonomous business units (or customers) to operate from a shared, high bandwidth platform.

Requirement: Ability to load balance and consolidate individual server workloads

Solution: VMWare Infrastructure 3 + management toolset (SCOM, Virtual Centre, Custom you-specific integrations using API/SDK etc.)

Requirement: Delegated control of systems to allow autonomy to teams, but within a controlled/auditable framework

Solution: Normal OS/app security delegation, Active Directory, NIS etc. Virtual Center, Checkpoint VSX, custom change request workflow and automation systems which are plugged into platform API/SDK’s etc.

the following diagram is my reference architecture for how I see these cloud platforms hanging together

image 

As ever more services move into the “cloud” or the “mesh” then integrating them becomes simpler, you have less of a focus on the platform that runs it – and just build what you need to operate your business etc.

In future maybe you’ll be able to use the public cloud services like Amazon AWS to integrate with your own internal cloud, allowing you to retain the important internal company data but take advantage of external, utility computing as required, on demand etc.

I don’t think we’ll ever get to.. (or want) to be 100% in a public cloud, but this private/internal cloud allows an organisation to retain it’s own internal agility and data ownership.

I hope this post has demonstrated that whilst, architecturally “cloud” computing sounds a bit out-there, you can practically implement it now by adopting this approach for the underlying infrastructure for your current application landscape.

h1

Running ESX 3.5 and 3i Under VMWare Workstation 6.5 Beta Build 91182

May 18, 2008

 

Following on from my earlier post I upgraded my installation to the new build of 6.5. it un-installed the old build and re-installed the latest without a problem, took about 30mins and required a reboot of the host OS.

All my previously suspended XP/2003 VM’s resumed ok without a restart but needed an upgrade to the VMTools which did require a restart of the guest OS – all completed with no problems.

Now, onto installing ESX….

I used the settings from Eric’s post here to edit my .vmx file

ethernet0.virtualDev = “e1000″

monitor.virtual_exec = “hardware”
monitor_control.restrict_backdoor = “true”

Note – you need to select an x64 Linux version from the VM type drop down, if you have to go back and change it via the GUI after you’ve edited the .vmx file it overwrites the Ethernet card “e1000″ setting to “vlance” so you need to edit again otherwise the ESX installer won’t find a compatible NIC and won’t install.

it was initially very slow to boot; 5mins on my dual core laptop with only one error – which was expected..

imageimage

To improve the performance I changed my installation to run the non-debug version of the Workstation binaries (rename the vmware-vmx.exe to vmware-vmx-debug.exe)

note: this isn’t recommended unless you know what you are doing, VMWare will rely on the output from the debug version of the code if you need to report any issues)

It also seems to work for the installable version of ESX 3i… (although I’ve not quite figured out the point of that version yet :) ).

image

Install prompt

image

it did fail with an error the 1st time round..

image

this was because I had specified an IDE disk as per the ESX instructions, I changed it to a SCSI one and it worked ok.

image

Finished..

imageimage 

The ESX 3i install has a footprint of about 200Mb on disk, and ESX 3.5 uses 1.5Gb.

I’m going to keep the 3.5 install on my laptop and will try to use linked clones to maintain a couple of different versions/configs to save disk space.. I’m sure I could knock up a quick script to change the hostname/IP of each clone – if I do I’ll post it here.

Why would you want to do this? well because you can, of course :) and its handy for testing patch updates and scripts for ESX management etc.

I will  also try to get a ESX DRS cluster running under workstation with a couple of ESX hosts and shared storage over iSCSI using something like OpenFiler as shown here. won’t exactly be production performance, but useful for testing and demo’ing.