4,391 Posts served

10,718 Conversations started

Hadoop and Cloudera: Open Source for the Cloud

By Dawn M. Foster (91 posts) on October 12, 2009 at 8:10 am

With the recent Hadoop World event hosted by Cloudera on October 2, 2009, Cloudera and Hadoop have been getting quite a bit of attention from the media, and the visibility for open source software in the cloud has increased along with them. I didn't attend the Hadoop World event, but I heard that it was well attended with solid content. Stephen O'Grady from RedMonk did a great summary of the event along with his analysis of the key trends if you want a little more information. The Hadoop World event is just a single point in time; however, the more interesting story in my opinion comes from the Hadoop / Cloudera combination.

Hadoop is an Apache project focused on open source software for reliable, scalable, distributed computing. One of the largest contributors to Hadoop is Yahoo, and Hadoop is part of the solution powering Microsoft's new Bing search engine. According to the Who Uses Hadoop page, other companies using Hadoop include Amazon, Adobe, AOL, Facebook, Google, Hulu, IBM, The New York Times, several universities, and many more.

Cloudera was founded to provide enterprise class support for organizations using the open source Hadoop product, and they recently announced the availability of a new product, Cloudera Desktop, a graphical interface for Hadoop that can be used with internal clusters or clusters running in a public cloud. Cloudera has also put together a strong team of people with deep expertise in open source, data management, search and cloud computing. One of the most recent additions to the Cloudera team is Doug Cutting, a search engine specialist from Yahoo and one of the founders of the Hadoop project. This is a big loss for Yahoo and a huge gain for Cloudera.

Om Malik recently weighed in on Cloudera to compare them to Red Hat. He sees many parallels between what Cloudera is doing now for Hadoop and what Red Hat was doing for Linux in the early days from similarities in the executive team, venture capital funding, technologies and more. Here's how he summarized his comparison:

The big change came this past August, when Doug Cutting left Yahoo and joined Cloudera. Cutting’s involvement is like the icing on the cake, giving the company the ability to corner all the Hadoop talent out there. It also helps that Cloudera has started to make inroads into newer markets, including biotech and retail. “Hadoop is going to find potential markets in any industry where there are large data sets that need complex analysis,” CEO Olson told me.

I remember talking to Red Hat executives back in the day and listening to their pitch about Linux everywhere, how they were going to go beyond the web community and help drive Linux into other corporate environments and eventually, build a services business around it.

Cloudera is following that same path. It’s developed its own version of Hadoop, one that’s optimized for the needs of large corporations, especially those that prefer a little hand-holding from their suppliers. By giving them this version of Hadoop, Cloudera hopes to make revenue from services. And the timing — the company unveiled Cloudera Desktop at Hadoop World (we are media partners) in New York, an event it organized — is perfect.

Game, set, match for Cloudera. (Quoted from GigaOM)

With cloud computing taking off, the open source Hadoop project is well positioned to become a key part of the many clustered solutions with large, complex data requirements, and Cloudera will be right there for those organizations that need help implementing Hadoop.

Categories: Open Source
Tags: apache, cloud, Cloud Computing, cloudera, hadoop, hadoop world

For more complete information about compiler optimizations, see our Optimization Notice.

Comments (2)

November 30, 2009 11:08 AM PST Anthony

Hadoop is GPL licensed. I don't see how all of these organizations are utilizing Hadoop commercially. Is there a separate commercial license?

November 30, 2009 9:14 PM PST Patrick

Actually, Hadoop is an Apache project, so it falls under the ASL (not GPL). Also, GPL does not prevent a commercial business model (see Linux/Red Hat).

Trackbacks (1)

• Blogging Elsewhere at Fast Wonder: Online Community Consulting
  October 17, 2009 9:53 AM PDT

Leave a comment