Saturday, January 10, 2015

Spring XD

Happpy New Year 2015 to all the followers.

Spring XD (eXtreme Data) is Pivotal’s Big Data play. It joins Spring Boot and Grails as part of the execution portion of the Spring IO platform. In simple term, itz a runtime environment for data proceessing.

As simple use case, you may define data streams reading from sources and writing to sinks, with processors in between to transform the data. The source could be Twitter, the sink could be HDFS, and the processor could convert the data to a certain format.  Behind the scenes it uses Spring Integration.

Spring XD contains two key components: Admin and Container servers. Using Domain Specific Language, you post the description of the required processing tasks to the Admin server over HTTP. The Admin server then maps the processing tasks into processing modules: a module is a unit of execution and is implemented as a Spring Application Context. The product has two modes of operation - single and multi-node.

Spring XD team have identified ingestion, real-time analytics, workflow orchestration, and export as being four major use cases common to creating Big Data solutions.  It is represented in the high level design as attached.

Spring XD reference is available at

No comments:

Post a Comment