Hortonworks show YARN future Hadoop with HDP 2.0 preview
The Palo Alto company look in rude health following Hadoop Summit, thanks to a welcome update for HDP 2.0
Last week’s Hadoop Summit has been labelled a
resounding success by
analysts, signalling the big data
framework’s maturity in the enterprise, as well as being a thriving
hub of innovation with many contenders.
The company that appears to have come out on top
from the event is Hortonworks. Aside from revealing a
new hefty $50m round of funding, the Palo Alto company has offered
a community preview for Hadoop Data
Platform 2.0, containing the highly anticipated
The framework, which has been four years in the
making, is considered the crux of the next open source release as
it broadens Hadoop’s processing options and architecture. In
simpler terms, the introduction of YARN allows MapReduce to focus
on processing alone, with YARN itself acting as the general purpose
resource management tool. Through this approach, Hadoop developers
can now create batch or nonbatch jobs while running interactive SQL
queries in parallel.
While the batch approach of MapReduce ultimately
made Hadoop’s name, the inability to handle several applications
(and glean insights from them at close to real time) has proven to
be a stumbling block for some.
Speaking to IDG News Service’s Joab Jackson,
Hortonworks founder Arun Murthy hoped that the introduction of YARN
would “fuel a whole new wave of innovation” in Hadoop. This appears
to hold substance, with Continuuity and Yahoo unveiling YARN
Storm-YARN in the past month respectively.
The likes of Microsoft, Red Hat and Splunk are among 15 companies
to partner up with Hortonworks, to help preach the benefits of
Also included in the HDP 2.0 preview is
an MapReduce generaliser framework, and Stinger, Hortonworks’
internal initiative to boost the speed of SQL queries against
The community preview of HDP 2.0 is available
for VMware and Oracle
Virtual Box environments, with a full preview
version expected in the next few days.
Image courtesy of Hortonworks, epSos.de