Fabric softener

Continuuity release Weave to help Hadoop YARN devs

With the next major iteration of Hadoop (2.0) tentatively set for a summer release, the last few months have seen a number of vendors unveil their next-generation data processing strategies.

Most of the boasting up until this point had focused on the querying side, with companies opting to renovate Apache Hive or by introducing new projects entirely. Nobody had really talked about the new ‘beyond batch’ workload manager YARN, outside of announcing it as the centrepiece of Hadoop 2.0.

Continuuity, a Hadoop application server startup we profiled last December, has offered up the first tool to reinforce YARN, with the release of Weave. The project provides Java developers with a simplified API for building distributed YARN applications, allowing users to manage resources, jobs and nodes built on YARN. Alongside this, Weave also contains a “Generic Application Master” supporting simple apps, as well as log and metric aggregation and archive management.

According to Continuuity CTO Jonathan Gray, Weave is just one part of the company’s “mission to make building Big Data applications easier”.

“With the release of Weave, we're supporting an increasingly diverse set of real-time application patterns to spur innovation in the Hadoop ecosystem,” the co-founder added in a press release.

The project has already received a blessing from Hortonworks co-founder Arun Murthy, who said he was “thrilled” that Continuuity are making the entry level to YARN easier.

"We developed Apache Hadoop YARN to support a wide set of use cases in a generic manner, and it is very important to make it easy for developers to exploit the features and capabilities of YARN,” Murthy explained in a statement. “We are excited to see the new types of applications that can be built on YARN using Weave."

Weave is now available on Github under an Apache 2.0 license. In the runup to YARN’s arrival with Hadoop 2.0, we might see more members of the supporting cast appearing shortly, designed to make the learning curve far easier to scale.

Image courtesy of fauxto_digit

Chris Mayer

What do you think?

JAX Magazine - 2014 - 05 Exclucively for iPad users JAX Magazine on Android

Comments

Latest opinions