Tonight as I watched the election returns on CNN, I wasn't sure what was more amazing: Obama's historic victory, or CNN's technology.  CNN was able to display up-to-the minute results of each state's elections simply at the touch of their news anchor onto the screen of an election-reporting system.  The anchor could touch a state, then touch a metric, such as various demographics, and instantly cut the election results up by age, exit poll answers, or racial composition.  It blew me away.

But it also reminded me of trying to manage a complex set of application deployments into the Cloud - a virtual private data center.    When you take into account that a reasonably complex multi-tier application with significant load can consume tens of virtual servers, all of which need to function successfully in a coordinated ballet, you realize you need the kind of information and analytic capabilities that CNN has available, just to tune it and keep it working.  Because of this, we've invested in an amazing remote monitoring package, NimBus, which we provide as a service to our hosted customers as well as other customers.  NimBus allows measurement of pretty much any parameter inside a virtual server or the applications running inside of it, from simple (but important) aspects such as CPU or memory utilization, to more complex metrics like database queries per second, slow query count, or pages served per unit time from a web server.  In addition, NimBus can perform user-experience validation by running synthetic (fake) transactions against an application and reporting what the user experiences in terms of response time and page correctness.   All of this is summarized on a customizable dashboard, much like CNN's election status screen (click to enlarge):

So, armed with this information - and hopefully not overwhelmed with too much information - we (or our customers) can tune and adjust their applications for appropriate cost/performance tradeoffs or diagnose performance or efficiency issues.    It has produced great results for the customers who implemented remote monitoring, improving their application response time and uptime, as well as reducing costs.

However, the road hasn't been easy.  The Cloud, by its very nature, is constantly in flux, mutable.   This presents a contradiction in goals to an organization:  to optimize something, it needs to be stable so you can measure it and make changes; yet to get the best economies out of the cloud, you need your infrastructure to be elastic, scaling on demand.  Because servers can come and go, and IP addresses can change, setting up a monitoring system and keeping it running isn't easy.  How can you monitor Apache server #2 if it is only instantiated when the web site's load is too high for one Apache?   Luckily, most of our clients' deployments don't change radically over the short term, so the monitoring package can be set up and continue to run for quite a while before it needs reconfiguration.  However, for very elastic loads, you need to either observe the results of your cloud deployment instead of its internals (such as by snooping on its communications with customers) or have your automatic instance deployments also request on-demand monitoring.

Once you add monitoring to your cloud deployment, you can start to take advantage of the powerful capabilities of Total Quality Management, a management philosophy popularized by W. Edwards Deming.  A core principle of TQM is CPI or continuous process improvement, summarized with the following chart:

TQM says you want to set goals for your process (in this case your software deployment), then you want to run the process (deploy the software), measure the results against the goals, and adjust the settings based on the goals to control the process to produce the desired results (typically a satisfy SLA in the software deployment world.)  However, the real power comes when you report on the results of this process and then use it to take another look at your goals.  The result is continuous improvements in "quality" - in other words, in your ability to deliver the results of your process successfully.

This is how we use monitoring to get the most out of Cloud deployments.

But then I had this insight: why do us - humans - have to be in the loop at all with respect to acting on the monitoring?   Naturally, if the monitoring detects some sort of application or hardware failure, humans need to get involved.  But are humans really necessary for maintaining SLAs?   In today's cloud deployments, especially with systems like Amazon's EC2, the users' application is responsible for both measuring and taking action on application performance issues.   This complicates deployment and coding, as well as tying your application to a particular cloud provider.  However, I believe that the next generation of cloud deployment frameworks will be able to do this automatically, by integrating general-purpose monitoring applications with policy-based cloud management engines.   At ENKI, using our monitoring services, we are already able to automate some of this policy-based management without the need for the application to be aware of the details of this process.   However, a quick caution is in order: if the application isn't designed from the ground up to be elastic (for example, to have new web servers added dynamically) then all the automation in the world won't allow it to participate in automated SLA assurance.

Last week, I attended an event sponsored by SDForum on Cloud Computing called "Cloud Computing and Beyond: The Web Grows Up (finally)"   For me, being a Cloud Computing provider myself, I expected the highlight of the event to be James Staten's opening keynote address on trends in Cloud Computing (you can find the slides here .)  And, it didn't disappoint: James's usual gift for clearly summarizing trends and futures made for an interesting talk. 

However, the highlight of the event for me turned out to be the panel on users' views of Utility Computing.  Given the nature of the audience - mostly tech-savvy software entrepreneurs - I expected a great deal of details about how these companies were using Cloud Computing.   What found out was that most of them are customers of Amazon's EC2 service, and what I heard instead of technical details was their strategies for mitigating the risks of dealing with Amazon.   Each of the companies using EC2 had developed intense internal expertise on Amazon's service.  That's right, you read correctly: on Amazon's service.  Not on Cloud Computing in general (though there are those who argue that EC2 and Cloud Computing are the same thing) but rather on the intricate technical and organizational details of dealing with Amazon and its service as well as optimizing their use of it.

It wasn't a surprise to me that these companies, dependent on Amazon for their success, would get to know the service well.  What did surprise me was the lengths to which they went to mitigate the risks.   Generally, if you have doubts or fears about something, you want to learn all you can about it.  These companies showed incredible technical depth at understanding EC2's characteristics and peccadillos.  They asserted that it was critical to their success.   But this seems to me to be in direct conflict with the ideal of Cloud Computing, which is the democratization of information technology; in other words, lowering the barriers of entry.   I think this is an admission that Cloud Computing technology in any form is still not polished enough for completely casual use.  At ENKI, we realized this early on and decided to strive to be the "onramp to the Cloud" by providing all the services necessary to use our cloud without having to know anything about Cloud Computing.  In fact, we have customers - software startups, generally - who don't even have any technical people on salary.   We - with their outsourced development team - just take care of all that for them.  Naturally, there is a labor cost to that, but it's certainly less than hiring dedicated Cloud Computing Experts to manage your cloud provider.   

It has been said that you should keep your enemies close to you.  The idea is, that way you'll know what they're doing so they can't hurt you.   These companies have taken a similar approach to dealing with Amazon.  Each of them admitted to having high-level contact with Amazon's management and knowing the management hierarchy at Amazon well.  They felt this was critical to their success.  However, Amazon doesn't even answer the phone for their regular customers, so how can the average entrepreneur expect to use EC2 successfully?   Fortunately, there are third parties out there that simplify the interface to EC2, but yet these successful startups had instead chosen to take the extreme step of building a personal relationship with Amazon.   At ENKI this comes naturally since our value proposition is to build a partnership with our customer that automatically builds the personal connections.  

Each of the panelists underscored the need for trust between cloud vendors and customers and pointed out that a great deal more transparency was necessary if that trust was going to develop successfully. I think the trust is key, because as the panelists illustrated, the cloud vendor becomes part of their customers' value delivery system - a role that traditionally was kept in-house (at much greater cost, of course.)   Putting myself in the cloud vendors' shoes (!) I can understand the barriers to transparency, which include such necessities as hiding the internal technology used to deliver the cloud services in order to maintain its security for all users.  (I've had $45 a month customers demand to get a tour of our data centers!)  And then there's always the software engineer's number one Achilles' Heel, which is lack of modesty: it's hard to say "I don't know" when a customer asks you a question - after all, you want to look like you know what you're doing!  Yet, these and other barriers will need to be overcome if Cloud Computing is going to move forward smoothly.  The cost savings from shared infrastructure that it offers are drawing in lots of customers, but the trust between customers and vendors must be cultivated if the revolution is to move beyond the casual user and SMB space into the mission critical enterprise market.