When the going gets tough, the tough get going.
Big Data has the potential to enrich our lives by enabling scientific discoveries, business models and consumer experiences. According to IDC, the market for Big Data will reach $41.5 billion by 2018, growing at a 26 percent CAGR, six times faster than the overall IT market. As Big Data deployments move beyond early adopters to the mainstream adoption within enterprises, the complexity of implementing Apache Hadoop* and related technologies has become an even more significant problem.
Because today’s infrastructure for most Apache Hadoop* and Apache Spark* deployments was not purpose-built to handle Big Data workloads or the needs of data scientists. As a result, converting enterprises’ existing infrastructure to handle tomorrow’s needs can often be expensive and time-consuming to implement. To address that challenge, modernizing and virtualizing the infrastructure for these Big Data workloads will become a key mechanism for organizations to future-proof their Big Data investments.
Given Intel’s vested interest in Apache Hadoop and Apache Spark and our engagement with these communities and ecosystems (see Intel’s previous Cloudera announcement) to drive the foundation of Big Data and Analytics in enterprise, we are exploring solutions to help the transition from early adoption to broader adoption and enterprise deployment become a reality.
Today, Intel announced that we are collaborating with companies such as BlueData to simplify and virtualize Big Data infrastructure. The collaboration will bring together BlueData’s infrastructure software for Big Data with the leading data center architecture based on Intel® Xeon® processor technology. The goal is to accelerate the adoption of Big Data solutions by making it easier, faster, and more cost-effective to deploy the applications and infrastructure required for Big Data analytics.
Intel technology like the Intel® Xeon® processor E5 family is built to overcome key virtualization challenges such as network latency, infrastructure security and power inefficiencies. It also provides a high-performance, secure, robust foundation for Big Data Analytics.
- Works with all of the major Apache Hadoop distributions as well as Apache Spark.
- Enables self-service Hadoop or Spark clusters, helps maintain security and control in a multi-tenant environment, and improves hardware utilization.
- Integrates with the leading analytical applications, so data scientists can use the tools they prefer.
- Runs with any shared storage environment, eliminating the need to move data and reducing data duplication.
- Delivers the simplicity of Hadoop-as-a-Service or Spark-as-a-Service in an on-premises deployment model, with the enterprise-grade security and governance that IT teams require.
Together, Intel and BlueData are focused on bringing Big Data into the mainstream and unlocking the value for our enterprise customers for faster time to insights with Big Data. BlueData will develop and optimize its Big Data infrastructure software for Intel architecture. The total solution enables easy deployment of Hadoop and Spark with greater simplicity and agility in a virtualized environment while delivering the security and performance that enterprise IT organizations require for their Big Data deployments.
My Big Data Technology team at Intel will be working closely with BlueData team on development, integration and testing of the optimizations in order to improve the function and performance of BlueData EPIC software platform on Intel architecture and other Intel platform technologies. We expect BlueData optimizations to enable Hadoop and Spark virtualization with close to bare metal performance, leveraging containers and other infrastructure technology advancements.
BlueData innovative solution enables self-service Big Data clusters, helps maintain security and control in multi-tenant environment and improve hardware utilization.
I am very excited for the deep engagement and collaboration with BlueData to solve challenging problems and bring more innovative solutions to our customers and the industry at large to accelerate Big Data deployment in the enterprise.
I am very excited for this deep engagement and collaboration with BlueData, solving challenging problems and bringing more innovative solutions to our customers and the industry at large to accelerate Big Data deployment in the enterprise. For more about the collaboration, you can also refer to this blog post from BlueData’s CEO, Kumar Sreekanti.
Intel is committed to make Big Data analytics faster, easier and more insightful. Our Intel processors are the foundation of most high-performance data centers. We are also developing critical software components and solution with Big Data and Analytics ecosystem players to help get enterprises get the most from Intel® processor-based platforms.
Big data analytics is transformational to all sectors and will drive new compute demand growth. We need hardware plus software optimizations and innovative solutions to scale-up and scale-out data analytics workloads.
Working together with innovative software companies such as BlueData, we can impact adoption barriers by making Big Data analytics cost-effective and easy to deploy.
*Other names and brands may be claimed as the property of others.
No computer system can be absolutely secure. Intel technologies’ features and benefits depend on system configuration and may require enabled hardware, software or service activation. Learn more at intel.com, or from the OEM or retailer.
About Michael Greene
Michael Greene is vice president of the Software and Services Group and general manager of System Technologies and Optimization at Intel Corporation. Greene is responsible for delivering software and solutions that enable Intel and its customers to accelerate their software development activities, speed up the adoption of new Intel-based platforms and capabilities, and optimize enterprise and client applications. His team is charged with providing efficient platform software solutions to original equipment manufacturers (OEMs), BIOS developers, software partners and IT end users. Since joining Intel in 1990 as an assembly test engineer, Greene has served in a variety of technical marketing, manufacturing, architecture and design roles. Most recently, he was director of platform technology and analysis, a role in which his responsibilities included evaluating future roadmaps for mobile, business, enterprise, graphics and software products. Outside of Intel, Greene serves on the executive committee of the National GEM Consortium, a partnership that promotes the participation of underrepresented groups in post-graduate science and engineering education and the technical workforce. Greene holds a patent in power-aware instruction scheduling and a bachelor's degree in electrical engineering from the Massachusetts Institute of Technology.