Is this the perfect storm for those of us who live every day in the data world?

Two of the biggest buzzwords and changes to IT in the way that we manage data assets are occurring at just about the same time: data processing is moving from on-premises to the cloud … and the size and techniques that we use to manage and analyze that data is turning to Big Data distributed approaches.

Mobile is also a big focus for IT executives and probably fits in well as a 3 leg of the data platform and also part of this industry inflexion point. Microsoft is moving in this direction with BI tools in Excel and SharePoint in the Cloud with Office 365, Google has their Cloud-based productivity tools as well. But traditional business intelligence tools like Tableau, QlikView and Business Objects are still primarily on-premises products. Moving those from laptops to mobile devices like tablets and phones is where Big Data Analytics meets mobile. More on that in a later posting …

The ability to utilize cloud providers massive infrastructures to shard your data, process it in parallel and then analyze it is very compelling to control costs, complexity and maintenance of your own clusters.

The proof that Big Data in the Cloud can be the primary use case for Big Data Analytics becomes apparent when you look at what 3 of the biggest software companies, who also happen to be 3 of the largest consumers of Big Data Analytics, are taking to market:

1. Microsoft HDInsight is Hadoop on Windows Azure
2. Google’s BigQuery, which provides REST access into query across huge data sets
3. Amazon’s Hadoop in the Cloud is Elastic MapReduce

Amazon is far & away the leader in this market today. They had the advantage of being early to embrace these approaches and used Big Data & NoSQL techniques internally for many years before taking their platforms to the public as a service with Amazon Web Services (AWS).

Google has also been a Big Data leader and user for a long time, but has a long way to go before they become a platform of choice for Big Data Analytics.

Microsoft is interesting in that they are investing heavily in Azure and their partnership with Hortonworks on the Hadoop for Windows platform. Microsoft’s REST-based object store (ASV) is similar to Amazon’s S3 and is something to consider when you look at future Big Data projects. Just keep in mind that HDInsight is still in preview (beta) at this time.