Since announcing automatic allocation and scaling of cloud resources in January, ENKI has been selling and providing customers with the ability to have their applications monitored by our monitoring system, and automatically have additional resources added and removed from their virtual private data centers to conform to varying loads and to save them money.  I wanted to share an update on this with you and show you some pictures!

Customers who can benefit from this must have an application in which additional instances of virtual servers can be added and utilized by the application software: for example, additional web, application, or other data processing servers that are handed work by a load balancer or job scheduler.  ENKI offers reserve capacity on an as-needed basis, or accepts reservations for reserve capacity so that it is guaranteed available when needed.

The mechanism we use to effect the auto-allocation is a controller node inside the customer's virtual private data center that can access monitoring information either for virtual servers or the load balancers that send requests on to them, and is authorized to request resource provisioning and disposal from AppLogic.  The policies for allocation and disposal can be set uniquely for each customer to optimize their load response profile to their business needs.  Because AppLogic has a persistent model for virtual instances, there is no need to allocate a new instance, but rather only turn it on, which speeds deployment to less than 30 seconds, versus other cloud vendors' potential 10 minute delays.  This makes the application more responsive, though it doesn't eliminate the need to size the "first" instance correctly for base loads.

The good stuff - a case study

We have one customer who serves ads for people and companies wishing to advertise on electronic greeting card sites.  The greeting card companies auction off blocks of ads, and then give the ad services company (ENKI's customer) the responsibility of serving the ads when their bid is accepted.  This creates a widely varying load on the ad services company, which is an ideal opportunity to take advantage of auto-allocation.    In this case, the customer has three virtual machines that serve ads, called A, B, and C.   A runs all the time, serving the base needs of our customer's business.  When bids are accepted, the controller watches the load on A go up, and turns on B when a predefined threshold is reached.  Similarly, if B becomes too busy, C is turned on.

Now for the pictures.  You can see the load (as total network traffic on the load balancer) in the first graph, with daily peaks corresponding to people's electronic greeting card usage habits.

Load:

Below are usage graphs for each of the three servers as they serve the load shown above.   The first graph shows the load on the application server which is always running. As this server, A, peaks, our Cloud Auto-Scaling system activates additional servers. 

Server A:

Server B:

 Server C:

What we see from these graphs is that there is some cost-saving optimization left to be applied: the primary server, A, is still idle a lot of the time.  Decreasing the resource allocation to A and perhaps increasing B or even making C twice as big as B could potentially save our ad-serving friends some money.   This really illustrates the power of in-depth monitoring when added to auto-scaling technology in the cloud: it produces analytic performance analyses that can allow businesses better control over cost and performance tradeoffs. As it currently stands, this customer is saving about 33% over simply allocating for peak usage.  With some tuning, they could save up to 50% or more.

Comments (12)

Subscribe to this comment's feed

Show/Hide comments

ENovikoff: ...

I attended a Cloud Camp a few weeks ago at Java1 (and wrote a blog at http://www.enkiconsulting.net/...d-101.html about). What I saw was that customers want their applications to run quickly and reliably at a reasonable cost without having to invest in deep technological understanding to do it. They want computing to run their business, and don't care whether it comes from Cloud, hosting, or managed services. Those distinctions are useful to providers, but selling customers on cloud is like trying to sell cars based on what engine technology is inside: it matters to the mileage, but they don't really care how it works under the covers. The current frenzy over Cloud is largely generated by developers and the cloud companies that cater to them, not by the customers themselves. They are only attracted to Cloud - as to the previous Next Big Thing in IT - because they think it'll solve their problems. So, ENKI focuses more on selling solutions than selling Cloud, because that's what we want our customers to think that we deliver on. Cloud is merely an enabling technology that allows us to deliver solutions cost effectively, a means to an end. That's not to say that the technology we have - both AppLogic and now PrimaCloud - is not best-in-class, but rather that at the end of the day it isn't what makes a happy customer. It's the relationship.

1

July 16, 2009

William Louth: ... http://williamlouth.wordpress.com

Sounds like your customers are looking for hosting or a managed service more so than a cloud platform.

2

July 15, 2009

ENovikoff: ...

It's pretty simply why our customers are in the Cloud: they want to get out of the IT business by finding a trusted partner that can do the job of an IT department for them without the associated cost of having their own. Cloud is merely a technology we use to enable that service. This has been my observation at numerous conferences and events: customers want computing and IT services, not cloud. In fact, they resent having to learn all about a new technology infrastructure and sourcing method that is confusing and rife with hucksters trying to take advantage of them. If you look elsewhere on this site, you'll see a presentation on the future of cloud, which will be a division into product lines are price and value focused. There will always be a place for price focused customers who feel the need to control their business at the server level by simply replacing hardware with cloudware, and system administrators with cloud administrators, though these people are realizing that Cloud is more expensive per CPU hour than colo and are becoming disillusioned with it. Where our vision of Cloud shines is for customers who are focused higher on the value chain, especially those who are choosing our PrimaCloud product. They're asking themselves, "how can I build a profitable technology company without legions of techies"? They know that's where the large cost and headache is, not in whether their application costs $0.45 versus $0.52 per CPU hour to deploy. To them, Cloud is a way of standardizing technology and services so that they can be delivered reliably by a remote provider such as ourselves while taking advantage of the economies of shared resources. Not that I'm against micro-optimization: if you read my blogs on power consumption, you'll see that I'm all about software efficiency, but ultimately my customers realize that that if they're focused on activity based costing inside their application, they are focused on the wrong activity. Missing the forest for the trees, as is commonly said. This is reflected in their consistent and constant demands for a fixed-price consumption-independent monthly package that includes resources as well as labor-based services - even if it is more expensive than itemizing every service as most Cloud vendors seem to be doing.

3

July 15, 2009

William Louth: ... http://williamlouth.wordpress.com

If that is indeed the case then it beats me why your customers are in the cloud in the first place.

You seem to imply that everything is relatively static (simple equation? no changes in behavior?) and that your customers are actually not concerned about metering (which by the way includes counting and aggregating the occurrence and cost) of services offered. Chargebacks? How does that actually happen without contextualizing the activity which is impossible via system monitoring.

Are you proposing that they build/buy adhoc solutions for every management domain (problem, slm, cost, performance, ...) when they could base it largely on the same model (activities resource usage/consumption) which works as an universal approach to optimization whether it be cost based or not.

Fine grain metering is needed as it is an important feed into billing, resource optimization, scaling (grid environments), performance management, cost management, even problem management and service level management. It seems to me that system monitoring should be largely based on this foundation.

Instrumentation need not be such a problem especially in Java/.NET runtimes which are more than likely going to dominate both private clouds and public clouds. Also most metering information will come via the request return payloads from interactions with other cloud services which seems to be whole point of the public cloud.

I think are differences are just in timing: present and future.

4

July 15, 2009

ENovikoff: ABC

I concur with you that ultimately companies want to see how their activities result in costing. However, I'm not sure your proposed methods apply to everyone:
1) Large numbers of our customer base (and I suspect Amazon's as well) use canned software which cannot be penetrated with ad-hoc probes. Nor do they have the skill or sophistication to place probes in the software they're using. It's up to the cloud vendor to deploy software such as NimSoft to collect what can be collected from the application and relate it to the charges.
2) For complex applications, such as NetSuite's systems which I was the director of development for, a great deal of context needs to be collected together with the probed costing information. At that time, we chose to do the probing ourselves, since we could store the probed values together with the end-customer data that ended up making sense of the costing (in other words, "why did this COGS report for customer Y use so much CPU time?")
3) My customers are invariably interested in an equation that looks like C = f(a,b,c) where C is cost, and a,b,c are factors relating to their sales of the software services they host in the cloud. Once they have this equation, they don't want to meter their apps at the detailed level required to keep the equation current (which can cause problems later of course.) As a result, a monitoring platform that sticks to goesintas and goesoutas is all they want.

5

July 15, 2009

William Louth: ... http://williamlouth.wordpress.com

By the way I am not trying to sell a product just getting the industry to move beyond the current enterprise system monitoring solutions which do play a role but much less so in the context of the cloud services especially composite ones. There will be those that do not request itemized billing from their phone companies but for the most part companies moving to the cloud will start to see a much stronger & explicit link with the software execution (mirroring the business transaction) and the cost or resource consumption of such executions which might have varying behavioral patterns depending on the work context (user, payload,....).

6

July 14, 2009

William Louth: ... http://williamlouth.wordpress.com

Actually you do have to do activity based costing (at least metering) every day in the cloud because it is the whole basis of a cloud service business. Metering data feeds into billing as well as application performance management which cannot be done just by sampling coarse grain metrics as in the case of system monitoring tools.

Most business insight tools are all about looking for footprints in the sand (which is expensive and delayed) but if you can meter at the important execution points (service entry points) you get this data in real-time which can feed into other parts of the cloud especially those that grid based with fine grain task provisioning within a cluster.

This is all incredibly important when the cloud service or application is multi-tenant because you want to also charge back to the user/depart/organization/company which can only be done at the point of execution where such contextual information is available.

In the software world were the cost of injecting such probes is minimal compared to what needs to be done in service & manufacturing based we can greatly extend the reach and use of ABC.

A comprehensive metering model can benefit multi management domains as every business transaction/activity is mirrored in the execution of some tracked activity.

7

July 14, 2009

ENovikoff: I agree!

William, I agree that what you're trying to do is different from what we're discussing here. In some ways, what you suggest is more akin to debugging or execution profiling and overkill for SLA assurance and overall cost reduction, which is the primary focus of auto-scaling. What I mean by this is that I'm guessing you don't need to do activity-based costing every day 24x7 on your app - it just generates more data than you need. But it is very important: in fact, I'd say that it is the #1 challenge that my customers who develop their own software have. Many don't have any idea how their choices in programming will affect their costs, and I spend considerable time with them to help them out with this. So with that in mind, I hope to be able to recommend your software to my customers. And if they choose to use the analytics to control our cloud framework, we can accommodate that as well...

8

July 14, 2009

William Louth: ... http://williamlouth.wordpress.com

Hi Eric,

Reading over your reply I think we are approaching the problem from different perspectives. You from a platform hosting provider and I from the customer trying to understand what software activities increase my storage, cpu, bandwidth,... costs and what are the responses times correlated to such difference resource consumption behaviors.

For auto-scaling process level metrics are more than adequate but for a customer trying to reduce his costs or resource consumption (irrespective of scaling and billing) than activity analysis is much more appropriate and meaningful.

9

July 14, 2009

William Louth: ... http://williamlouth.wordpress.com

Hi Eric,

Our probes are activity based which means we track resource (metering) changes at the thread level and aggregate at the process, host and cluster. We can relate a change to a chain/series of execution flows and runs (call contexts: call stack, distributed trace stack, workflow sequence). These contexts can be mapped to one or more cost centers which themselves are composite hierarchical structures.

Enterprise system monitoring tools like Nimsoft cannot do this because they generally pull only process level metrics from exposed management interfaces which means you cannot perform a cause-effect (thread ~= worker) analysis other than statistical correlation which as we both know rarely works at this required granularity especially with inconsistency across different metric/measure providers even with a single management domain (i.e. jmx). You miss completely the whole cloud service call chain which is the point of activity based costing and the cornerstone for cost management in service based industries outside of the cloud today.

Kind regards,

William

10

July 14, 2009

ENovikoff: Reply

William,
Thank you for your comment. If you check the graphs, you will see they're from NimSoft, which is an enterprise monitoring system which is capable of the "business metering" you speak about, including probes that view what is going on inside a variety of commercial and open-source software packages. Our auto-scaling capability is capable of receiving control input from NimSoft as well as other packages (with some customization) in order to scale the application resources, so the control inputs our customers can use to manage resource assignment are essentially limitless.

11

July 14, 2009

William Louth: Possibilities but need much more information http://williamlouth.wordpress.com

Eric I think you have managed to identify a possible cost saving in this particular context (single purpose app) but to effect change (tune, reduce usage) a customer would require in-depth monitoring (metering) information at the software activity level so that could understand what activities drive the cost upwards at particular points in the daily workload of an application in the cloud.

System monitoring will only get you so far and is much more geared toward the hosting/platform vendor than the cloud service/application customer.

Why have separate monitoring, performance, cost and capacity models when it is all about the software executing in the context in terms of activity and resource metering which can be abstracted to represent business meters.

A Unified Approach to Performance Management and Cost Management for Cloud Computing
http://www.jinspired.com/products/jxinsight/meteringthecloud.html

ABC for Cloud Computing
http://williamlouth.wordpress.com/2009/01/27/abc-for-cloud-computing/

12

July 14, 2009

 Show/Hide comment form

Name

Email

Website

Title

Comment

Write the displayed characters

Add Comment