As for adding a tack on Operating System Cluster on top of that actually has caused more outages than it helps for most of these apps. It adds extra complexity to a running application for what? Return to service faster? VMware HA is what we use for that.

This is often resolved by finding the reason a given application can not handle a hard crash and working on getting that fixed so it can restart cleaner in a crash state.

The thing to think about is when 60 business facing critical applications go down at one time, that’s not just a single Severity Priority 1 ticket, its 60. That’s a huge hit on the reputation. Not a good thing. We can stomach 30 a little easier. Especially when you talk about the fact that I saved $500k by sticking all these critical apps into VMware. We can do the numbers and compare outage costs for these applications versus the capital and operational costs of the Infrastructure. If an app costs the company $100k/hour in lost sales and we only saved $20k per year…. Not a really good trade off if it goes down. We have to balance the impact.

Now if I ran VMware on all the blades in a single chassis it wouldn’t be nearly as big of a deal as long as I design to prevent too many eggs in a single basket. Regardless of how redundant something is, it will go down. Not a matter of if, just a matter of when. Hopefully you never see that day, yet when that day comes are you willing to take the fall for that design? So to do a serious VMware cluster you want to spread a given cluster across multiple chassis at a bare minimum. That’s not exactly cheap at the end of the day unless your buying 16 BL495s at the get go and don’t mind having the risk of 1/2 of an 8 host cluster going down at any given time. This would address the single point of failure discussion.

The “Eggs in a Basket” is not blurred. It is still very clear. Virtualization does not suddenly add more capabilities to the guest OSes. Just because it is virtual doesn’t mean you don’t have single points of failure. The virtual is just another layer in the discussion. VMs run in a single location at any given time. They are the point of impact for an IaaS environment concept.

My math and statistics have shown so far that scaling out does reduce our overall system impact at time of outage event. It does cost more though in the grand scheme if you don’t calculate in the outage impact to your business or clients. That can often even the cost discussion pretty quickly when a given VM is down for more than 15 mins cause it takes 30 mins to come or something and that runs the business $200k/hour in lost sales. Then well.. having 2 more hosts out there instead of 1 big host, sure can make that discussion moot quickly.

You put fimware management is an issue with BL495, can you elaborate?

I’ve read Duncan’s post as well, my though is that since virtualization came into the game, the notion of ‘all eggs in a single basket’ is blured, the boudaries are dynamic and expanding out (vs stretching in). Yesterday you would be flamed down if you say that you run 3 VMs on a single host. Today people would say that’s a waste of resources.

We could get a math geek in and compute the probability, my bet is that scaling out at the same time you scale up decreases significantly the risk associated with all in one basket, don’t you ?