Markets in compute power, much talked about by me and others are now it seems finally here (from The Economist:)
Fundamentally, SpotCloud works like other spot markets. Firms with excess computing capacityâ€”operators of data centres, cloud providers, hosting firmsâ€”put it up for sale. Others, who have a short-term need for some number-crunching, can bid for it. Enomaly takes a cut of between 10% and 30% depending on the size of the deal. But there is an important difference: SpotCloud is what Enomaly calls an â€œopaque marketâ€, meaning that the firms offering capacity do not have to reveal their identity. Thus selling computing services for cheap on SpotCloud does not cannibalise regular offerings.
So my question is when does Amazon.com split its retail operations from its AWS platform business. I’d love to see these priced separately. Actually, truth be told, I suggest Amazon.com is actually three businesses:
Earlier this year I suggested that AWS in particular could well be the totemic representative technology that inaugurates the sixth techno-economic paradigm:
Just as Intelâ€™s 4004 microprocessor was the catalyst for a wave of creative destruction in the 70s and 80s, will AWS prove the same for the 00s and 10s? Probably. Weâ€™re seeing it already. And itâ€™s going to disrupt the hell out of the mastodons of industry across most sectors of the economy. Why? Because their cultures and leaders are entirely ill-equipped to face such a fundamental paradigm shift. They know how to play by the old rules. The strategic competitive advantages they built up over decades risk suddenly â€“ poof! â€“ to become obsolete.
The central concept in this new option is that of the Spot Price, which we determine based on current supply and demand and will fluctuate periodically. If the maximum price a customer has bid exceeds the current Spot Price then their instances will be run, priced at the current Spot Price. If the Spot Price rises above the customer’s bid, their instances will be terminated and restarted (if the customer wants it restarted at all) when the Spot Price falls below the customer’s bid. This gives customers exact control over the maximum cost they are incurring for their workloads, and often will provide them with substantial savings. It is important to note that customers will pay only the existing Spot Price; the maximum price just specifies how much a customer is willing to pay for capacity as the Spot Price changes.
Spot Instances are ideal for Amazon EC2 customers who have workloads that are flexible as to when its tasks are run. These can be incidental tasks, such as the analysis of a particular dataset, or tasks where the amount of work to be done is almost never finished, such as media conversion from a Hollywood’s studio’s movie vault, or web crawling for a search indexing company. For most of these tasks their completion is not time critical and as such they are ideal targets for additional cost savings.
Before I go any further, let’s just say it’s pretty exciting to see vision become reality even if in this case I’m only a distant spectator. Markets in anything. Digital markets. Themes that go back to the founding mission statement of the Park Paradigm:
(December 2005) The technology of the digital age is driving an unprecedented explosion in the ability to create markets in anything. Trade anything. Not just physical goods. Not just financial instruments. But ideas. Events. Outcomes.
The emergence of these kinds of markets will â€“ over time â€“ impact how we view and interact with the world in all aspects of our personal and professional lives. They will fundamentally alter the current world economic and social paradigm.
Chris Swan calls them virtual resource markets and correctly points out that, at least for now, the market is “closed” – ie users cannot trade their capacity amongst themselves, however I suspect that it is just a matter of time before such a market is organized. But what would be even more useful (and exciting) than a closed market on Amazon EC2 resources, would be an open marketplace for on-demand spot computing resources. ie A marketplace which is agnostic as to where the compute resource comes from, so long as it is a robust and more or less uniform resource.* However for this to be useful for the end consumers of this computing commodity, the ability to switch automatically and seamlessly from one cloud computing source to another based on price and/or availability would be crucial. Indeed this would be the key value driver for anyone hoping to operate a compute resource exchange. Sure the price discovery and transaction mechanisms would be necessary but these are relatively trivial to build and hard (in isolation) to monetize. The real value creator for any exchange (just ask the CME) lies in clearing and settlements. (For the non-financial amongst my readers this is the back-end of the trade, fulfillment essentially.)**
Note, however, that this feature is not market-based pricing. Amazon determines the spot price and can raise that price enough to gain back capacity at will, at no real cost to itself. There is no competition. There is no commoditization. There is just consumption of what is not being used.
The truth is, real commoditization of infrastructure services–or any other cloud service, for that matter–isn’t in the best interest of Amazon or any other service provider.
Regardless, commoditization can’t happen without open standards that allow easy portability and interoperability of data and code, as well as security, control, service-level assurance and compliance systems. Those standards are coming, but it is impossible to predict when they will arrive. I only hope Amazon embraces them when they do.
I’m not sure I agree with his view however that commoditization isn’t in the best interest of Amazon. The underlying asset is ultimately relatively undifferentiated (a compute cycle is a compute cycle is a compute cycle) which is in fact the definition of a commodity. If you are a provider of a commodity – unless you can maintain a monopoly or a cartel – it is in your interest to create as big and vibrant a marketplace as possible. Supply creating demand. And particularly if you fancy yourself the most efficient, large scale producer of said commodity (as I’m sure Amazon does), all the more reason to want a big, liquid market of consumers. It is the exchange and clearer that want to create lock-in, not the producers. To be fair, for the moment AWS is both and indeed this is the point James is making I think, but I would be surprised if they had the intention (hubris?) to think this is anything but a transitionary arrangement.
Of course, as a traded market in this critical 21st century resource develops over the next decade and beyond, the business opportunities abound. Better yet, many of them are well known and can quickly be adapted (from other asset markets) to apply to the compute resource market. It’s not a business yet, but it only took a few hours before the first ticker tapes(here also) began to appear for EC2 pricing:
An entire ecosystem will surely emerge – exchanges, prime brokers, risk management derivatives, algorithmic trading… I’m sure there will also be some interesting second-order opportunities. Linking spot computing prices with spot electricity prices. Selling green compute cycles (ie powered by renewable energy sources only.) Allowing anyone to sell compute cycles into the grid (think SETI@home meets micro-generation). The mind races.
Welcome to the sixth paradigm.
* like a bond futures contract, one could imagine allowing any compute resource fitting a certain minimum specification into the “basket” of deliverable resources; typically in this scenario there would be a “cheapest-to-deliver” resource in the basket which presumably would get allocated first.
** I can’t help but wondering if the amazing technology developed by our portfolio company CohesiveFT couldn’t be adapted or re-purposed to form the core fulfillment engine of a compute resource exchange. The fact that they are Chicago based and their CEO/Founder is ex-O’Connor makes me wonder even more!
I’m going to keep this short, mainly because I’m not an expert by any stretch of the imagination. So discount this as a layman’s viewpoint as needed.
The most common, almost Pavlovian, stock response I hear from (both IT and senior management) financial services firms with respect to why they don’t see cloud computing as relevant to their high-level business strategy (ie ok around the edges but really just an IT cost/benefit thing..) is:
Of course you know, our business is different, it needs to be secure. The hardware needs to be sitting under my desk.
Ok, fine I made the last bit up, but you know they’re thinking as much. So without digressing into a debate as to just how secure most financial services IT is, the question I always respond with is this: does your organization know how to run a more secure data centre than Amazon or Google (or any other present or future specialist cloud infrastructure supplier)??? Really think about it. Do you make your own hardware? Perhaps you can make banking microchips better than Intel… (From Appirio’s CIO Guide to On-demand: )
On-premise does not equal secure: the biggest driver towards private clouds has been fear, uncertainty, and doubt about security. For many, it just feels more secure to have your data in a data center that you control. But is it? Unless your company spends more money and energy thinking about security than Amazon, Google, and Salesforce, the answer is probably â€œno.â€
VPN-CubedÂ® is the first commercial solution that enables customer control in a cloud, across multiple clouds, and between private infrastructure and the clouds.
VPN-Cubed provides an overlay network that allows YOU control of addressing, topology, protocols, and encrypted communications for YOUR devices deployed to virtual infrastructure or cloud computing centers. When using public clouds your corporate assets are going into 3rd party controlled infrastructure. Enterprise checks and balances require you to exhibit control over your computing infrastructure. VPN-Cubed gives you flexibility with control in 3rd party environments.
The other myth to dispell is that no one is suggesting migrating any or all infrastructure to a cloud environment overnight, or even as soon as possible. The decision whether or not to move existing infrastructure to a cloud (private or public) and when is indeed probably more of a ‘routine’ (but big) question for IT (although management should be interested in the answer.) The point I’m trying to make, the point that is relevant for the executive committee is:
How does the nature of my business – what products and services I provide to my customers and how – potentially change because of this new technological substrate?
This is the question that should animate the weekend executive strategy retreat. In order to answer that question, you need to have some understanding of the technology but not how it works so much as what it can do. I don’t need to know how the microchip works in a digital camera to think about how I can use that camera. The question management should be brainstorming is:
If we were to start with a blank page, with the technology that exists today (and will likely exist in the next 5-10 years) how would you best build a company to serve our customers, present and future? What does FaaS (Finance as a Service) look like?
This isn’t going to happen overnight so the suggestion is not to ‘throw the cards up in the air’ and panic. And incumbents have many advantages on their side (customer inertia being the most valuable). But it will happen. And quickly in the geological timescale of large organizations so they need to start moving, start mapping out this future. And – here is a shameless plug for Nauiokas Park – one facet of that should be placing a lot of small bets on emerging, disruptive start-ups that have the luxury of moving more quickly, experimenting more radically, with faster evolutionary cycles. (Like a genomics company experimenting using fruit-flies and mice first to isolate winning adaptations.) While at the same time preparing their supertankers for a significant change in direction.
Maybe we should offer to moderate these strategic retreats. Do you think we would get any takers? If you work in a financial services company, ask your CEO and let us know.
If you are looking for a good (albeit long) explanation of what VPN-Cubed does and why it really is a “game-changer” read this post from Mark Masterson who sums it up as follows:
So, let’s sum up. Enterprise cloud computing is a type of cloud computing that is suited to the specific requirements of existing companies, and allows them to leverage resources in the Cloud to provide economical ways of adding capacity to their existing environments. First, their existing data centre (or some portion of it) is virtualised. Once this is accomplished, capacity from external cloud providers can be added (and dropped) dynamically, using technologies like VPN-Cubed, allowing enterprises to use the cloud to elastically (and transparently) scale out to the cloud. And because all network traffic is securely encrypted, enterprises can effectively make use of public, cloud infrastructure as if it was part of their internal datacentre — entirely behind the (virtual) firewall. Moreover, the same technology can be leveraged to allow the use of multiple, disparate cloud providers, effectively solving the ‘eggs in one basket’ problem. Different cloud providers can be leveraged to allow for failover redundancy, load balancing, even the leveraging of different providers on a dynamic basis, using metrics such as SLA compliance, or changes in cost. And an enterprise might want to do this not because it will reduce costs, or allow a switch from capital to operating expenditures (although both of those things might be true or not, depending on the context), but because it will increase their overall agility.
Just as Intelâ€™s 4004 microprocessor was the catalyst for a wave of creative destruction in the 70s and 80s, will AWS prove the same for the 00s and 10s? Probably. Weâ€™re seeing it already. And itâ€™s going to disrupt the hell out of the mastodons of industry across most sectors of the economy. Why? Because their cultures and leaders are entirely ill-equipped to face such a fundamental paradigm shift. They know how to play by the old rules. The strategic competitive advantages they built up over decades risk suddenly – poof! – to become obsolete.
And yet all too often, I’m met not with disbelief but with apathy, indifference. You can see the thoughts forming in their heads: “I’m a CEO, a business man, a producer! Why is Sean boring me with this technology stuff? Why doesn’t he just talk to the CIO?” Worse, too often when I talk to senior technology managers in big corporations, they also are disdainful, thinking: “Yeah, yeah, that’s all fine for your start-ups and Web2.whatever companies, but this is a real business. Serious. Not some website for teenagers to swap gossip.” Ok I’m exaggerating but a lot less than you think. Sometimes I figure I must not be saying it right. So I’m always on the lookout for good articulations of the potential and importance of cloud computing and its incredible relevance to anyone who is pretending to run a business. Especially a big one.
In short, Enterprise IT must extend out to Consumer IT, otherwise those companies simply won’t be able to compete. As we’ll explore, Web 2.0 has changed the landscape with social networks, and companies can ill afford to ignore the shift…
…Cloud computing isn’t just about on-demand IT; it’s about on-demand business innovation…
…Cloud computing isn’t just for small- and medium-sized companies and garage startups. Cloud computing makes it possible to create new business platforms that will allow companies to change their business models and collaborate in powerful new ways that weren’t practical before. What’s important for companies to consider is that cloud computing isn’t about technology, it’s about technology-enabled business models.
So if you know a CEO, or any senior managers (in any business) pass them this article. It will only take 10 minutes to read. And maybe it just might make them reconsider. And maybe they’ll invite me to lunch! 😉
Carlota Perez is one of my heroes. Her fantastic articulation (in Technological Revolutions and Financial Capital: The Dynamics of Bubbles and Golden Ages) of how technological revolutions mark turning points in long economic cycles, building on the work of Schumpeter and Hayek, is in my opinion an incredible lens through which to understand long term economic growth and its effect on financial markets. Her approach is a key foundational pillar for our investment thesis, and is why we feel confident that it is possible to generate excess returns by catching the long term secular economic waves that ultimately govern capital markets. (Think of it as the polar opposite of day trading.)
In her thesis, each successive long wave of the economic cycle is initially catalyzed by a technological revolution, usually only visible in hindsight:
My suspicion is that we are living through a “phase change” now (be careful, “now” in this context means a period of a few years, not “today” or “this quarter”…) – and so I’ve been wondering what will come to be seen as the foundational technological revolution of the sixth paradigm. The previous five, as defined by Perez, are below:
There are many possibilities, but I’m starting to think that the transition to cloud computing (enhanced by ubiquitous wireless connectivity) just might be it. And for the sake of taking a punt on what might be a good symbolic starting point for this revolution, how about the launch of Amazon‘s S3 and EC2 in 2006?
Amazon today said it would bring web-scale computing power for use in workloads such as web indexing and data mining to just about anyone. The bookseller now offers MapReduce (a programming model created by Google to help deal with incredibly large data sets) using Hadoop on Amazonâ€™s Elastic Compute Cloud and Simple Storage Service. This allows AWS customers to access the power of a Google- or Yahoo-style server and programming infrastructure to model business decisions and analyze huges sets of customer or corporate data without having to invest in thousands of servers (as well as dozens of programmers). Dana Gardner over at ZDNet says one could think of it as having access to a personal supercomputer.
Just as Intel’s 4004 microprocessor was the catalyst for a wave of creative destruction in the 70s and 80s, will AWS prove the same for the 00s and 10s? Probably. We’re seeing it already. And it’s going to disrupt the hell out of the mastodons of industry across most sectors of the economy. Why? Because their cultures and leaders are entirely ill-equipped to face such a fundamental paradigm shift. They know how to play by the old rules. The strategic competitive advantages they built up over decades risk suddenly – poof! – to become obsolete. (from Dan Gardner:)
Think of it as having your own tuned supercomputer that you can plug gigantic data sets into and ask questions that will determine the course of your businesses for the next decade. Oh, and you can pay for the pleasure on a credit card.
This high-end BI value has pretty much been the sole purview of large, skilled and deep-pocketed enterprises. But there are plenty of people, researchers, government agencies, academics, small to medium enterprises, venture capitalists and the like that would hugely benefit from sussing out important trends and findings from the growing reams of raw data generated by modern businesses and societies. Talk about metadata on steroids!
“This high-end BI value has pretty much been the sole purview of large, skilled and deep-pocketed enterprises.” Not anymore… Think about that for a moment.
Size used to be an advantage in almost any industry…now? Not so much. New rules, new winners.
Thought experiment: Let’s take, oh say…banking. Which would you rather run (if say your life depended on success, which I know these days is a bit far-fetched but humor me…)?
A greenfield start-from-scratch-bank (assuming you had access to sufficient capital to get started, say $100 million or so)? Or,
[insert favorite megabank here] (assuming you had access to sufficient capital to not be immediately insolvent, say $100 billion or so)?
Well unless you are a sociopath as per Hugh and see the key metric of success being how many people report to you and whether or not global political leaders will take your call, I think the answer is pretty bloody obvious.
So what does all this mean? Well, for us it means investing in companies that are positioned to ride this wave (not build a levee against it, hoping it won’t break.) Some – like cohesiveFT are right in the heart of the technology facilitating this new paradigm. Others, like our most recent investments Zoopla and FX Capital Group, are building new business models adapted to the new technological landscape that will allow them to disrupt and extend existing markets. But it also means remembering that you can be right (about the future) but still not come out on top:
A pretty interesting space I’d say. I’d love to be able to attend the Greening of the Internet conference in a couple weeks but unfortunately it would be a bit of a luxury now and logistically impossible as well so I’ll just have to follow remotely. (If anyone spots a good live blog or two on the conference please let me know in the comments…)
Over the past couple years, I’ve become increasingly interested in the potential to combine technology, energy and markets to solve multiple problems concurrently. One of the absolute best people to follow if you are interested in this space is Bill St Arnaud; I highly recommend subscribing to his RSS feed or email list.
One back-of-the-envelope idea I’ve been touting for the past few years is that Iceland should reinvent itself as (one of) the data-centers to the world. Given the complete implosion of the Icelandic economy in 2008, if my idea makes any sense, it should be pursued even more aggressively. In a nutshell…
As the world inexorably moves to computing and storage as a utility, and as energy prices and climate change (price of GHG emissions) loom ever larger in the economics of cloud computing and data-centers, and as information transmission (bandwidth) costs continue to plummet, the relative importance of (carbon-neutral) energy operating costs will continue to increase strongly. (The other main factors being: construction costs, access to bandwidth, legal jurisdiction, security, availability of skilled workforce.) Sooo…. why Iceland? Cold climate and abundant green energy (geothermal now, offshore wave and wind later.) Heck the forward stream of CER’s (depending on what happens in Copenhagen) might even fund the upfront capital cost! Also one would guess that Iceland (as opposed to say Greenland or the Canadian Arctic) would have both reasonable construction costs and access to the skilled workforce needed to both build and then run these operations. I’m not an expert on data-centers but I don’t think they generate massive amount of jobs, but if this vision were to become true, Iceland could build a whole ecosystem around Green ICT and given it’s small population, I’m sure this would have a material impact on both employment and GDP over both the short and the long term. The biggest risk I can see would be natural disaster risk (ie volcanoes blowing up your data-center) and risk of damage to sub-oceanic fibre going in and out of Iceland. Both of which I think should be reasonably easy to mitigate and insure against. Plus, on the face of it, its geographic isolation and NATO membership probably make it a good choice in terms of security.
So…if I were managing a substantial Private Equity Infrastructure fund, I would (do a proper analysis of course but assuming my back of the envelope thinking holds):
Start talking to the Icelanding government, IMF and the World Bank/EIB about co-funding and incentive schemes (especially the required incremental undersea bandwidth
Encourage Iceland to join the EU asap because having data-centers in a EU jurisdiction will be a very important commercial imperative.
Start talking to Amazon (AWS) and Google about partnering / risk sharing as potential ‘anchor tenants’ (they could become to Iceland what Dell was to Ireland 20 years ago…)
Start talking to smart banks, hedge funds, etc. about innovative carbon-based financing and energy hedging structures.
Obviously this is just scratching the surface of the work that would need to be done but equally the myriad of collateral opportunities that would emerge as a part of such an ambitious project.
I’ve decribed this idea with varying degrees of detail to probably 2 or 3 dozen people over the past 2 years and no one has found a serious or obvious flaw in the logic. Then again none of them – myself included – had “done the math” nor was necessarily highly fluent in the mechanics/costs of building data-centers generally. Equally, none of them turned around and offered to raise a few million of seed finance either to do a feasability study, so you could say talk is cheap. So in the spirit of casting the net wider, I thought I’d write about it here and solicit experts to punch as many holes in the idea as it merits.
In any event, I never understood why Iceland put all its chips on banking and retail when it is so clearly at a comparative disadvantage in both domains…if nothing else my scheme at least has the semblance of logic behind it! 😉
With so many column inches these days dedicated to analyzing the storm clouds engulfing the world’s economy, its nice to have a good excuse to have a more positive cloud story to tell. One of our portfolio companies – cohesiveFT – just announced their new VPN Cubed ‘packaged service’ for enhanced cloud security (from the Elastic Server Blog:)
VPN-Cubed is an encrypted virtual private network (VPN), enabling customer-controlled security inside a single cloud, across multiple clouds, and between clouds and private infrastructure.
One of the key elements we see driving the emergence of the disruptive business models in markets and financial services is of course the ever decreasing cost of computing, storage and communication. This inexorable and exponential trend is absolutely critical when considering many opportunities in financial services and markets as more often than not they involve very high volumes of data processing, computational intensity or both, usually in real time. Indeed one of the most significant historical barriers to entry in finance was the cost and complexity of the required ICT infrastructure. In our minds, clearly the trend towards computing as a utility will underpin a new paradigm in a number of different industries and will fundamentally change ‘what is possible’. We think companies like cohesiveFT will play an important part in facilitating this transition, especially with young innovative companies that aren’t trapped in any particular legacy infrastructure. That said, we also believe that ultimately this trend will affect everyone. Indeed as the cohesiveFT team points out, the current poor economic conditions could actually end up catalyzing such a move sooner rather than later:
We see the current economic downturn as actually increasing Cloud Computing interest. Rather than writing that $2M check for your data center hardware refresh, wouldn’t you rather manage your infrastructure as a service? Moving part of your infrastructure to the cloud turns capital expenditures (or fixed costs) into operational expenditures (or variable costs). According to IDC, a global provider of market intelligence, cloud computing is poised to capture 25% of IT growth spend by 2012. Yet, in a recent IDC survey, 74% of IT executives/CIOs cited security as the top challenge preventing their adoption of the cloud services model.
So for any of you out there – whether you are working in a big bank, a small hedge fund or a ‘sixth paradigm’ start-up, I encourage you to engage your management and your IT leaders in a discussion about your cloud computing strategy and perhaps to have a look at VPN Cubed and cohesive’s other products. I think you might like what you find.
Disclosure: My company is an investor in cohesiveFT and we are represented on the Board by Amy Nauiokas. Further, Craig Heimark, cohesiveFT’s CEO is an advisor to Nauiokas Park.