Facebook Presto

Potentially raising the bar on SQL scalability, Facebook has released open source SQL query engine called Presto that was built to work with petabyte-sized data warehouses.

Currently, more than 1,000 Facebook employees use Presto daily to run 30,000 interactive queries, involving over a petabyte of processing. The company has scaled the software to run on a 1000-node cluster

Unlike Hive, Presto does not use MapReduce, which involves writing results back to disk. Instead, Presto compiles parts of the query on the fly and does all of its processing in memory. As a result, Facebook claims Presto is 10 times better in terms of CPU efficiency and latency than the Hive and MapReduce combo.
.
Now, Facebook wants other data-driven organizations to use, and it hopes, refine Presto. The company has posted the software’s source code at https://github.com/Facebook/presto. The software is already being tested by a number of other large Internet services, namely AirBnB and Dropbox.

Big Data Expo

Big Data Expo is the single most effective event for you to learn how to use you own enterprise data – processed in the cloud – most effectively to drive value for your business.

A recent Gartner report predicts that the volume of enterprise data overall will increase by a phenomenal 650% over the next five years.

These two unstoppable enterprise IT trends, Cloud Computing and Big Data, will converge in Silicon Valley at Big Data Expo 2013 Silicon Valley – being held November 4 - 7, 2013, at the Santa Clara Convention Center in Santa Clara, CA.

It offers in Silicon Valley a vast selection of technical and strategic breakout sessions, General Sessions, Industry Keynotes, our signature discussion "Power Panels" and a bustling Expo floor complete with two busy Demo Theaters so that as a delegate you can kick the tires of solutions and offerings, and discuss one-on-one with all the leading Cloud and Big Data players what they are offering and how to make use of it in your particular situation.

Just as Big Data and Cloud solutions will be side by side on the Expo floor, so they are in the conference program. We're including below a sampler of the breakouts you can look forward to in the Big Data part of the overall technical program...and if you look at the welter of company logos at the very bottom you'll get a foretaste of the hundreds of companies in attendance on the Expo floor.

The surest way to help yourself and your company obtain maximum value from your enterprise data and to understand why Big Data is going mainstream is by attending Big Data Expo 2013 Silicon Valley

BigData Elite

The fund, called Big Data Elite, is backed by celebrity investors like Ron Conway, one of Silicon Valley’s most prominent angel investors, and Andreessen Horowitz, the $2.5 billion venture capitalist firm, as well as Social+Capital Partnership and Anand Rajaraman, whose social media start-up was bought by Wal-Mart Stores in 2011.

In an announcement on Thursday, Big Data Elite described itself as a venture lab and early stage fund that will offer a six-month program beginning January 2014. The fund will choose 10 start-ups or individuals from a list of 20. Those chosen will work a Big Data Elite’s offices in San Francisco and will have access to advisers who work at Facebook, Zynga, Netflix, LinkedIn, Riot Games and a handful of other companies that rely heavily on data analysis.

Companies like Netflix, Google and Amazon rely heavily on the data its customers provide through their Internet searches, and over recent years they have poured large sums of money into data teams of computer scientists who sift through search and transaction data to find patterns.

“Big Data is behind every sector,” Mr. Venios said. “By 2016, the budget spend for Big Data will be larger in corporate business development than information technology,” Mr. Venios added, citing data from McKinsey & Company.

The fund will invest $150,000 to $1 million in each participant, in exchange for equity — typically 6 percent initially — and convertible notes.

QueryIO

QueryIO is a Hadoop-based SQL and Big Data Analytics solution, used to store, structure, analyze and visualize vast amounts of structured and unstructured Big Data. It is especially well suited to enable users to process unstructured Big Data, give it a structure and support querying and analysis of this Big Data using standard SQL syntax.

QueryIO enables you to leverage the vast and mature infrastructure built around SQL and relational databases and utilize it for your Big Data Analytics needs.

QueryIO builds on Apache Hadoop's scalability and reliability and enhances basic Hadoop by adding data integration services, cluster management and monitoring services as well as big data querying and analysis services. It makes it easy to query and analyze big data across hundreds of commodity Compute+Store cluster nodes and petabytes of data in an easy and logical manner.

Highly funded startup

Riding the Big Data tidal wave, MongoDB has raised $150 million in financing to accelerate its global expansion. The round was led by an unnamed financial services company and attracted industry heavy hitters such as Salesforce and Intel.

Now the most valuable startup in New York, MongoDB offers a highly scalable document-oriented database for analytical and fast-changing operational workloads. The platform has been downloaded over 5 million times, and deployed by SAP, MTV Networks, Foursquare and other big name companies.

As part of its aggressive growth strategy, MongoDB will expand its partner ecosystem and target new markets. The company also plans to double its headcount to 600 over the next 12 months.

Originally known as 10gen, the SQL vendor rebranded in August to “get back into alignment” with its flagship product. Shortly afterwards, rival MemSQL updated its distributed in-memory database to support JSON, a popular syntax for processing semi-structured data.  It also improves index scan performance and expands support for SQL.

Hadoop on Windows

Hortonworks, a leading contributor to and provider of enterprise Apache Hadoop, today announced the general availability of Hortonworks Data Platform 1.3 (HDP) for Windows, a 100-percent open source data platform powered by Apache Hadoop.  HDP 1.3 for Windows is the only Apache Hadoop-based distribution certified to run on Windows Server 2008 R2 and Windows Server 2012, enabling Microsoft customers to build and deploy Hadoop-based analytic applications. This release is further demonstration of the deep engineering collaboration between Microsoft and Hortonworks.

New functionality in HDP 1.3 for Windows includes HBase 0.94.6.1, Flume 1.3.1, ZooKeeper 3.4.5 and Mahout 0.7.0. These new capabilities enable customers to exploit net new types of data to build new business applications as part of their modern data architecture.

Hortonworks Data Platform 1.3 for Windows is now available for download at: http://hortonworks.com/download/

Intel Big Data

Intel, increasingly customizing server chips for customers, is now tuning chips for workloads in big data.

Software is becoming an important building block in chip design, and customization will help applications gather, manage and analyze data a lot quicker, said Ron Kasabian, general manager of big data solutions at Intel.

The plan includes developing accelerators or cores for big-data type workloads. For example, Intel is working with Chinese company Bocom to implement the Smart City project, which tries to solve counterfeit license plate problems in China by recognizing plates, car makes and models. The project involves sending images through server gateways, and Intel is looking to fill software gaps by enhancing the silicon.

A lot of research is also taking place at Intel labs on stream processing and graph analytics as the company designs chips and tweaks software.