After it’s all said and done, this trip will still be worth it all.  I look forward to seeing all the vExperts, Wizards, Stalkers and vEverything folks that I’ve met and look forward to meeting new ones.  See you there.

When you use Altiris DOS boot disk and PXE boot a
virtual machine running on ESX 4.1 with flexible
adapter, the virtual machine might fail to start when it
attempts to load MS‐DOS LAN Manager NetBind.

VMware is still supporting DOS long after Microsoft has sent it to the curb.  I know several Fortune 1000 companies that still have DOS applications that they need to run that are critical to the business processes.   What other top notch hypervisors support DOS still?

In our production environment one of our oldest clusters just broke 100,000 vMotions (or more accurately since they aren’t at ESXi 4.1U1 yet, VMotions).

This cluster has weathered the ups and downs of a functional vSphere cluster with hosts going into an out of Maintenance Mode, Hardware failures and general maintenance.  This cluster we had over allocated on CPU for a couple months per our standard policy, though we got no real complaints about performance there.  We have since updated that policy and I’m sure the number of vMotions has slowed down somewhat.

Just proof that vMotions/DRS & the entire vSphere solution is valid and solid.  Here’s to another 100k vMotions.

Congrats to all the excellent information sharers out there.   You have all earned it.

On my side I am super ecstatic that I had 20 people vote for me.   Thank you to all the readers of this blog.

One of the better opening keynotes in a while. It answers the question of “What is the cloud?” in a way that everyone can agree on. Thanks to @jtroyer & @tony_dunn for getting this available.

VMware vCloud Director (vCD)

So go read it and come back.    Pretty powerful stuff even at a 1.0.

If you are familiar with the concepts of organizations, VM Templates, network Fencing and self service that are presented from Lab Manager, you will quickly get about 75% of vCD.   The other 25% is coming from Chargeback and vOrchestrator capabilities.  The challenge with Lab Manager is being able to run true production out of it.   The management is a bit limiting and constrained by size.   vCD takes all those concepts and adds a few more and pushes up the scalability to Service Provider size where you need to deal with limits of 4095 VLANs and Petabytes of storage.

Does this mean that an SMB can’t use vCD.  I don’t believe so.   When I look at this I easily see Lab Manager as dead now.   Why spend any resources on a less functional, less useful, more limited product when you have something you just need to right size in licensing for someone that needs to use it for a “Lab Manager style Test/Dev” environment?   vCD can do everything we do in Lab Manager today along with being able to have production right next to it in the same management interface.

Lab Manager is Dead.   Long Live Lab Manager.

IE8 based Painting Issues

I then launched vSphere client only to be unable to see any of the objects in the left hand window in the client.

vSphere Painting Issues

This VM is running on ESX 3.5U4 on VM Hardware 4.   The quick fix is to upgrade to VM Hardware 7 which entails all the updates to vSphere 4 & VM tools updates.

The other fix is due to the following two bugs in Hardware version 4.

1. Completely uninstall the View Agent
2. Reboot
3. Reinstall the View Agent (Make sure that the Video Driver version is [...].0032)

If this doesn’t do it then you are probably having an issue with VRAM.   The fix is to adjust the pool inside of View for this machine and set the resolution and # of monitors so they come out to a number divisible by 64.  (Kudos to my Support Wizard for finding this one.)

The magic formula is

((#of monitors * Width of Resolution) * (# of monitors * Height of Resolution) * 4 )/1024 == Multiple of 64

Keep in mind if you have less monitors than you set the pool to, PCoIP handles this gracefully and it doesn’t cause issues.

Our conversation usually goes something like this:

The hot new badness just came out and we need to order more hardware.

Awesome.   So how much does this puppy have in it?  RAM?  CPUs?   Slots for HBAs & NICs?

Did you know the new motherboard comes with 4 NICs now so our standard config can go from 4U to 2U and gobs of RAM with 6 core CPUs now.

Awesome!   *pause*  You know with that much RAM I can put 100 Win7 VDI systems on there.   Umm.. What about when it goes down?

Oh.. Hrm.   That wouldn’t be so good.   ….

That being said we generally end up breaking it down to a couple of factors.

1. What is the current capacity configuration we run with today?
2. What is our current pain points in CPU, Memory, Network or Storage?
3. Is there any new architecture changes coming that will impact this design?   Is there a new switch fabric that needs to be plugged into?   Is there changes to storage that need to be addressed?
4. How much does this new hardware configuration cost?
5. How will this change affect DRS’s Chaos Theory?  The more hosts, the more DRS can do for you due to Chaos theory.
6. What is our Risk level for number of eggs in a single basket?

The point is most corporation’s environments aren’t starting from scratch.  In my case we have a known configuration today to use as a baseline and adjust the environment and design every hardware order to make it better.

In our most recent order we had this discussion all over again.  This time we had some architectural changes needed to prevent some false positive HA events from happening in a 2 time a year strange events.   So we are going to a 3 switch connectivity solution to enable network beacon for NIC teamed connections.  We started with the following information:

• Baseline:  HP DL585 G5, 4 sockets w/ quad cores, 128G of RAM, 3 Dual 1G NICs, 2 Emulex LPe11000 HBAs
• Cluster: 10 Host Clusters with ~30 per Host in Servers and ~65 Workstations per Host in View
• Pain Points:  CPU starvation, Licensing Issues with 10 Host sized clusters
• Risk Level:  Politically we are getting pretty touchy about more than 30 Servers going down in a single blow even if HA works on bringing them up in under 15 mins automatically.

We compared 3 different models of newer, faster, badder and more wicked hardware from HP since the DL585 G5s are not really on the manufacturing line anymore.   So we looked at the BL495cG6, DL585 G6, DL385 G6 and DL580 G6.

DL585 G6:

• Pros
  • Proven and comfortable AMD based stable platform with a good price/performance cost.
  • Gain more CPU resources with the additional 2 cores per socket.  6 core systems.
  • Can build 5 Host clusters to address licensing issues.   Issues with HA support for the density involved.
• Cons
  • Same Risk Level as before.

  Push

  • Same architectural solution today with maybe another NIC card to enable the NIC Beaconing

DL580 G5:

• Pros
  • Fastest individual cores out there.   Lots of good press about the Intel.
  • Should get better CPU resources with higher performing CPUs.
  • Can build 5 Host clusters to address licensing issues.   Issues with HA support for the density involved.
• Cons
  • Significant premium in cost for speed.   See easily a 25% premium for a 10% faster performance.
  • Same Risk Level as before.
• Push
  • Same architectural solution today with maybe another NIC card to enable the NIC Beaconing.

DL385 G6:

• Pros
  • Lowers the risk level without lowering performance
  • Best price/performance cost for 6 core systems
  • Has enough slots to move to the newer network layout to enable NIC Beaconing
  • Gain more CPU resources with the additional 2 cores per socket.  6 core systems.
  • Put 64G of RAM into them and build 5 host clusters for licensing problematic applications.
• Cons
  • More physical hosts to deal with (cabling, power, rack space, cooling, management)

BL495c G6:

• Pros
  • Blades reduce the amount of cabling
  • Gain more CPU resources with the additional 2 cores per socket.  6 core systems
• Cons
  • Firmware Management is an issue
  • Increases our Risk Level with more eggs in the same basket unless we get multiple chassis to spread the blades across
  • New solution from the ground up running ESX on blades
  • Not ready to support Flex10 and because of this we have limited NIC capabilities to fit our requirements

We decided to go with the DL385 G6s based on these criteria.  We will dedicate a specific 5 Host cluster for problem children applications with licensing issues.   The RAM size of the hosts will limit the number of VMs we can end up putting in a cluster which addresses the Risk Level of number of VMs per Host.  We are still way ahead of the game using VMware so having to have a couple more physicals for all these improvements is not an issue.

In your company or solution something else may be more appropriate.  The key in an ongoing improvement mentality is have things you can measure and then criteria on what to change along with why.   There is no one size fits all answer which is why VMware works so well for so many different folks.   We don’t have to change how we do things to gain a lot of flexibility in the Datacenter while not changing how we ultimately end up managing these systems.

Per the site manifest:

This is our place to share cool tools created by VMware engineers.  There is a wide range of tools here for you, including one for automating tasks, getting ESX performance graphs, a rich Internet application framework and much more. These tools are offered under Technical Preview or relevant Open Source License.

They are calling each app/tool/API a fling.   This is a pretty smart naming.   I have seen several start up companies employees start making some cool code that never gets to see the light of day.   They are just flings of interest to help a specific problem.   They don’t always become full fledged products.

Here’s the current list:

Apache Pivot

Like most modern development platforms, Pivot provides a comprehensive set of foundation classes that together comprise a “framework”. These classes form the building blocks upon which more complex and sophisticated applications can be built.

Dynamo RIO

DynamoRIO exports an interface for building dynamic tools for a wide variety of uses: program analysis and understanding, profiling, instrumentation, optimization, translation, etc. Unlike many dynamic tool systems, DynamoRIO is not limited to insertion of callouts/trampolines and allows arbitrary modifications to application instructions via a powerful IA-32/AMD64 instruction manipulation library. DynamoRIO provides efficient, transparent, and comprehensive manipulation of unmoOndified applications running on stock operating systems (Windows or Linux) and commodity IA-32 and AMD64.

esxplot

Esxplot is a GUI based tool that lets you explore the data collected by esxtop in batch mode. The program loads files of this data and presents it as a hierarchical tree where the values are selectable in the left panel of the tool, graphs of the selected metrics are plotted in the right panel.

Onyx

Onyx is a standalone application that serves as a proxy between the vSphere Client and the vCenter Server. It monitors the network communication between them and translates it into an executable PowerShell code. Later this code could be modified and saved into a reusable function or script.

SVGA Sonar

VGA Sonar is a demo application for SVGADevTap. SVGADevTap is a user-level library that communicates with the VMware SVGA guest driver to provide low-latency notifications of changes to the screen.

vApprun

The vApprun tool implements the same vApp/OVF feature set as the vSphere 4 release. Thus, Workstation/Fusion can be used as a development environment for advanced OVF packages, and it can be used to evaluate and test OVF packages on your desktops and laptops.

vCMA

VMware vCenter Mobile Access (vCMA) – vCMA allows you to monitor and manage VMware Infrastructure from your mobile phone with an interface that is optimized for such devices.

VGC

VMware Guest Console allows you to manage the Guest OSes from the VMware layer.

VI Java

vSphere Java API is a set of Java libraries that sits on top of existing vSphere SDK Web Services interfaces. It provides full managed object model and run-time type checking, resulting dramatic productivity boost. With the new Web Services engine in 2.0, it also performs much faster than engines like Apache AXIS up to 15 times.

Virtual USB Analyzer

The Virtual USB Analyzer is a free and open source tool for visualizing logs of USB packets, from hardware or software USB sniffer tools. As far as we know, it’s the world’s first tool to provide a graphical visualization along with raw hex dumps and high-level protocol analysis.

If you want to see what is possible with a companies products, these are some of the tools to go look at.  http://labs.vmware.com

Update 18 Feb 10 @ 8:28pm:

@vmwarecares Network outage here caused by small plane crash in Palo Alto

http://www.cnn.com/2010/US/02/18/texas.plane.crash/index.html?hpt=C1

VMware Support back and up and running by 9pm CST.