I was having lunch the other day with the CTO of a successful web startup which is on its way to financial break-even, usually a good sign that the business model is working and the venture capitalists will support the company until it goes public.  He asked me about ENKI's Computing Utility offering and why it would be advantageous for a company like his to use it.

Ultimately, for a start-up company living off of investor cash, the name of the game is to be able to break even with the smallest investment possible, which allows the founders, investors, and employees to make the largest profit.  Even with a great business model and happy customers, this means controlling costs.  In the late nineties, wild parties, huge speculative marketing experiments, and vast, optimistic computing infrastructure build-outs were the rage - but most of those companies ran out of money, contributing to the dot-bomb crash.   Working in that world, I always wondered what happened to business fundamentals: careful investments and cost control.  Nowadays, these best practices are back in style even in the high-flying world of web startups, in large part because investors are more careful about how they spend their money.

This is where utility computing comes in.  Instead of investing in your own data center and staff to maintain it, you just buy as much computing as you need.   Large enterprises talk about depreciation expense, taxes, and capital cost as part of their break-even financial models, but for the startup it boils down to "how much money is in the bank and how long can I run with that?"  So buying computing as a service makes a lot of sense.  But how much sense does it make?

Recently I was at the web2.0 conference and attended a talk by one of the founders of Digg, who shared a little about how much computing resource they were applying to their business.  After getting a rough idea of what they were doing, I asked myself how much it was costing them, and if buying utility computing from ENKI would have helped them out.  A little time with Excel and some numbers I put together from published colocation costs got me a spreadsheet that said that a Digg-like company would save about a million dollars over a year and a half on data center costs.

The spreadsheet can be viewed here .

I had to make some assumptions, particularly that the start-up was trying to run very lean, working their IT staff hard, in my experience to the edge of burnout.  This seemed in line with Digg's staffing count and my experience with web start-ups.  I also had to make some assumptions about how the startup's application would map into ENKI's grid technology, which allows for much higher levels of server utilization and failover capability with less hardware.  So the numbers might be off by 20-30%, especially depending on the startup's IT philosophies.  But no matter how you cut it, paying as you go is what a start-up should do.  

Another thing I discussed with my CTO friend was if a start-up could outsource their computing needs with confidence.  In my experience working with start-ups, their computing infrastructure succeeded or failed based on the quality of their IT staff and management.   At ENKI, we have an experienced team, but we also make the interface with our customers easy by not just selling hosting, but rather consulting with our customers to implement their applications successfully on our computing resources.  I accounted for this cost as best I could as a per-server monthly labor charge.  It is surprisingly low, but that's the benefit of the grid technology we're using.

In the quest for ever higher computing efficiency, we see vendors touting more and more "green" hardware, which I talked about a bit in the last data center power consumption article.  This might lead one to believe that having the most effficient hardware is the best way to reduce data center power consumption.  However, it is not.  The true key to efficient computing is your software.  Software approaches to saving data center power fall into two categories: virtualization, and optimal applications.

Optimize your Applications!

Have you ever wondered how NASA can send a probe to Jupiter with multiple computers in it, running off just a few tens of watts?  Aside from special-purpose computers, those probes have software in them that's written for efficiency, usually in a very efficient fully-compiled language.  Engineers have pored over the code at length, removing every inefficiency, and using the best algorithms for the job.  If you're building a digital services company, like a web startup or Saas provider, it pays to think about efficiency up-front.  In my personal experience as a coder and software manager, I've seen efficiency improvements through careful coding of 1000% or more!  You can take that to the bank.

The keys to achieving high software efficiency are a deep knowledge of programming techniques and the languages and environments you're planning to implement your applications in, as well as good data on the performance of any server applications you plan on buying.  Often, efficiency can be assumed to be the same thing as speed: the faster the program executes on a given computer, the more it gets done for the amount of power and equipment you have invested in running it.  For software you write or have written for you, an execution profiler is an indispensable tool.  It will show your developers where their code is spending most of its time, and allow them to focus on speeding up the problem areas.  If an execution profiler isn't practical, you can still get similar information by putting timing checks into suspect code. Your choice of language can greatly affect the efficiency of your software, so it can pay to choose your implementation language carefully, or implement compute-intensive sections of code in a more efficient language. While today's rapid application development pace may let you get to market quickly, it can also cause your team to skip the optimizations that will save significant costs on IT infrastructure later.  Many startups find themselves "refactoring" (rewriting) their applications within a year or two just to be able to meet customer demand, not to mention containing IT costs.

For acquired software, it pays to do evaluations under the types of loads you expect to stimulate them with, or read software reviews very carefully.  If you're purchasing based on reviews, be sure the reviewer has done throughput testing on the software that is similar to the conditions you plan to run it under.    Databases in particular can eat up lots of computing resources, so which database you choose may heavily influence how much computing power you will need to run your application.  Proper setup parameters are critical to getting the most out of purchased software, including setting up the operating system correctly and normalizing the data in your database.  Often, application execution environments, such as the popular LAMP (Linux, Apache, MySQL, PHP) can be tuned to have a wide range of performance on the same hardware - and, you can often replace the database or another component with one that is radically more efficient.   Finally, when your application is running, don't forget to circle back and make sure that your hardware is tuned to support it: applications are either RAM-limited or CPU-limited, so building your servers to take this into account will allow you to reduce the number of servers, often dramatically.   Another important tuning technique for today's multimedia applications is to choose the storage solution that matches your software's data throughput requirements: different types of RAID or NAS appliances can greatly influence how much hardware you need to get good performance.

Server utilization is the key to saving on power and real estate

So, now that your application is running on its servers as efficiently as possible, what more can you do to increase efficiency?  Well, most CTOs will find themselves choosing to run their servers at 5-25% average load, to make sure that peak loads don't cause excessive response times for their users.  However, if you remember from our hardware discussion, much of the power drawn by a server is independent of load.  Our standard dual-core double-Opteron 1U servers draw about 200 watts under full load, but still use about 130W when they're idling.  (We put a minimum of disks and other accessories in our servers, so they use less power.)  So, you can see that our server running at 10% usage still uses 66% of its full power.  In other words, if you were able to keep the server busy at 100%, you'd get six times higher energy efficiency, and ten times more space efficiency!

Server virtualization allows you to run multiple operating system environments, each potentially with multiple applications, on the same server.   Virtualization splits up the compute and memory resources of the server, making each virtual machine think it has the server to itself.  Clearly, you will have to provision your server with adequate resources (including CPU and memory) to allow each virtual machine to adequately run its applications.  There are a couple big names in the virtualization space: VMWare, which is proprietary code, and Xen, which is open source.  Xen is also resold with support, utilities, and management software by Virtual Iron and XenSource.  These systems are not cheap, though the Xen based systems can be more cost effective.  They are also problematic to set up and operate.  Not the least problem of which is knowing which server contains which applications in case of hardware failure!  Also, your administrators will have to pay more attention to server utilization, and move applications on and off them in order to use them optimally.  For this, I recommend  a good monitoring application that can tell you when things are headed south, as well as some of the virtualization management suites that are becoming common.

Despite the drawbacks, the payoff for virtualization is huge.  For example, if you run 5 applications that have an average CPU usage of 10% on a virtualized server, you'd still have 50% headroom and enjoy a 5X hardware cost savings, including energy usage.  Many large companies are virtualizing their whole data centers, together with new high-density blade servers, and getting giant payoffs in space and energy savings.  But even if you just have a rack of 4 servers, virtualization can bring you significant benefits.

Here at ENKI, we use a virtualization scheme based on Xen to run our data center.  However, on top of the virtualization, we use AppLogic, a Grid Operating System from 3Tera that allows us to build Virtual Private Data Centers on top of a grid of identical servers.  AppLogic can dynamically scale the resources each application is allocated as well as bring in cold standby machines automatically in case of failures.  This allows us to provide our utility computing product.  As a result, we only deploy as much computing power as our customers need, allowing us to run green and pass the savings on to them.  It also allows us to run our servers at even higher levels of utilization, since we can easily provision additional hardware resources and move virtual servers to them to handle spikes in demand.