Agile Analytics, Continued …

18 Sep

It’s been a while since I’ve posted here about the role of Agile Scrum in BI development teams and how you can be successful implementing Agile Scrum in your BI solution development teams. For a review, check out my post from almost exactly 1 year ago here.

That’s back when I was the BI Technology Director for Razorfish where we took a Big Data Analytics platform to market for online advertising analytics. Since then, Razorfish & Atlas were sold to Facebook & Omnicom and I moved back to the software vendor side of the business, with Penatho.

In both cases, we built BI and Analytics solutions using Agile. Pentaho is a bit simpler to implement because we are building Java applications with teams of developers who are already familiar with working in Agile teams and environments.

At Razorfish, we were really employing teams of BI developers: ETL developers, data modelers, cube developers, report designers and data scientists. Not only are many of those roles not comfortable naturally in an Agile environment, many of these roles have worked for years in their career previously working solo, not even in teams at all.

You can read through my earlier post and Ken Collier’s excellent (and very small, quick read) Agile Analytics which I also referenced in that link. But today I wanted to highlight 2 important points when considering Agile Scrum for BI solutions that I have run into in these environments, which also came up many times during my time in Redmond with Microsoft:

  1. BI engineers are indeed developers. You do not have to write code in Java, .NET or C++ to be a “developer”. An engineer writing ETL and models is writing code, testing, working from requirements and troubleshooting. That means that Agile can work and it also means that “BI Developer” is a role that is very important, deserves the same level of respect and can be very difficult and tedious at times. Give some love back to your BI Developer!!
  2. Sprint Reviews CAN work in BI / Analytics solutions and are VITAL to your success. In the Java development world, this may seem like a no-brainer. But I run into a lot of resistance when building Analytics solutions using Scrum. Many folks believe that building a data model, ETL, analytics, reports and demoing all of that in a 3 / 4 week iteration is not possible. It is possible, I have worked in this environment for years and will take you out of that “go off and build-it” approach to business intelligence that has killed many BI projects over the years. Sprint Reviews are your friend!

 

Advertisements

One Response to “Agile Analytics, Continued …”

  1. Stephen February 20, 2014 at 9:55 am #

    I’ve been following these posts and am hungry for more tips and details on exactly how one can harness the more tangible benefits of Test Driven Development (TDD) in a Data Warehousing / BI project.

    My vision is to build a foundation of TDD tests that will become the regression testing for the BI solution we are providing to an Investment Reporting group, but there isn’t alot of precedent to follow, so any hints you have would be greatly appreciated!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

cbailiss

Microsoft SQL/BI and other bits and pieces

TIME

Current & Breaking News | National & World Updates

Tech Ramblings

My Thoughts on Software

SQL Authority with Pinal Dave

SQL Server Performance Tuning Expert

Insight Extractor - Blog

Paras Doshi's Blog on Analytics, Data Science & Business Intelligence.

The SQL Herald

Databases et al...

Chris Webb's BI Blog

Microsoft Analysis Services, MDX, DAX, Power Pivot, Power Query and Power BI

Bill on BI

Info about Business Analytics and Pentaho

Big Data Analytics

Occasional observations from a vet of many database, Big Data and BI battles

Blog Home for MSSQLDUDE

The life of a data geek

%d bloggers like this: