May 2010 Early Indications: Devising the cloud-aware organization

As various analysts and technology executives assess the pros and cons of cloud computing, two points of consensus appear to be emerging:

A) very large data centers benefit from extreme economies of scale

B) cloud success stories are generally found outside of the traditional IT shop.

Let us examine each of these in more detail, then probe some of the implications.

The advantages of scale

Whether run by a cloud provider or a well-managed enterprise IT group, very large data centers exhibit economies of scale not found in smaller server installations. First, the leverage of relatively expensive and skilled technologists is far higher when one person can manage between 1,000 and 2,000 highly automated servers, as at Microsoft, as opposed to one person being responsible for between five and 50 machines, which is common.

Second, the power consumption of a well-engineered data center can be more efficient than that of many traditional operations. Yahoo is building a new facility in upstate New York, for example, that utilizes atmospheric cooling to the point that only 1% of electricity consumption is for air conditioning and related cooling tasks. Having people with deep expertise in cooling, power consumption, recovery, and other niche skills on staff also helps make cloud providers more efficient than those running at smaller scales.

Finally, large data centers benefit from aggregation of demand. Assume facility A has 10,000 users of computing cycles spread over a variety of different cyclical patterns while facility B has fewer users, all with similar seasonality for retail, quarterly closes for an accounting function, or monthly invoices. Facility A should be able to run more efficiently because it has a more "liquid" market for its capabilities while facility B will likely have to build to its highest load (plus a safety margin) then run less efficiently the majority of the time. What James Hamilton of Amazon calls
"non-correlated peaks" can be difficult to generate within a single enterprise or function.

Who reaps the cloud's benefits?

For all of these benefits, external cloud successes have yet to accrue to traditional IT organizations. At Amazon Web Services, for example, of roughly 100 case studies, none are devoted to traditional enterprise processes such as order management, invoicing and payment processing, or HR.

There are many readily understandable reasons for this pattern; here is a sample. First, legal and regulatory constraints often require a physical audit of information handling practices to which virtual answers are unacceptable. Second, the laws of physics may make large volumes of database joins and other computing tasks difficult to
execute off-premise. In general, high-volume transaction processing is not currently recommended as a cloud candidate.

Third, licenses from traditional enterprise providers such as Microsoft, Oracle, and SAP are still evolving, making it difficult to run their software in hybrid environments (in which some processes run locally while others run in a cloud). In addition, only a few enterprise applications of either the package or custom variety are designed to run as well on cloud infrastructure as they do on a conventional server or cluster. Fourth, accounting practices in IT may make it difficult to know the true baseline costs and benefits to which an outside provider must compare: some CIOs never see their electric bills, for example.

For these reasons, among others, the conclusion is usually drawn that cloud computing is a suboptimal fit for traditional enterprise IT. However, let's invert that logic to see how organizations have historically adapted to new technology capability. When electric motors replaced overhead drive shafts driven by waterwheels adjoining textile mills, the looms and other machines were often left in the same positions for decades before mill owners realized the facility could be organized independently of power supply. More recently, word-processing computers from the likes of Wang initially automated typing pools (one third of all U.S. women working in 1971 were secretaries); it was not until 10 to 20 years later that large numbers of managers began to service their own document-production needs, and thereby alter the shape of organizations.

The cloud will change how resources are organized

Enterprise IT architectures embed a wide range of operating assumptions regarding the nature of work, the location of business processes, clockspeed, and other factors. When a major shift occurs in the information or other infrastructure, it takes years for organizations to adapt. If we take as our premise that most organizations are not yet prepared to exploit cloud computing (rather than talk about clouds not being ready for "the enterprise"), what are some potential ramifications?

-Organizations are already being founded with very little capital investment. For a services- or knowledge-intensive business that does not make anything physical, free tools and low-cost computing cycles can mostly be expensed, changing the fund-raising and indeed organizational strategies significantly.

-The perennial question of "who owns the data?" enters a new phase. While today USB drives and desktop databases continue to make it possible to hoard data, in the future organizations built on cloud-friendly logic from their origins will deliver new wrinkles to information-handling practices. The issue will by no means disappear:
Google's Gmail cloud storage is no doubt already home to a sizable quantity of enterprise data.

-Smartphones, tablets, and other devices built without mass storage can thrive in a cloud-centric environment, particularly if the organization is designed to be fluid and mobile. Coburn Ventures in New York, for example, is an investment firm comprised of a small team of mobile knowledge workers who for the first five years had no
corporate office whatsoever: the organization operated from wi-fi hotspots, with only occasional all-hands meetings.

-New systems of trust and precautions will need to take shape as the core IT processing capacity migrates to a vendor. It's rarely consequential to contract for a video transcoding or a weather simulation and have it be interrupted. More problematically, near-real-time processes such as customer service will likely need to
be redesigned to operate successfully in a cloud, or cluster of clouds. Service-level agreements will need to reflect the true cost and impact of interruptions or other lapses. Third-party adjudicators may emerge to assess the responsibility of the cloud customer who introduced a hiccup into the environment relative to the vendor whose
failover failed.

In short, as cloud computing reallocates the division of labor within the computing fabric, it will also change how managers and, especially, entrepreneurs organize resources into firms, partnerships, and other formal structures. Once these forms emerge, the nature of everything else will be subject to reinvention: work, risk, reward, collaboration, and indeed value itself.