It’s been a while since I’ve posted here about the role of Agile Scrum in BI development teams and how you can be successful implementing Agile Scrum in your BI solution development teams. For a review, check out my post from almost exactly 1 year ago here.

That’s back when I was the BI Technology Director for Razorfish where we took a Big Data Analytics platform to market for online advertising analytics. Since then, Razorfish & Atlas were sold to Facebook & Omnicom and I moved back to the software vendor side of the business, with Penatho.

In both cases, we built BI and Analytics solutions using Agile. Pentaho is a bit simpler to implement because we are building Java applications with teams of developers who are already familiar with working in Agile teams and environments.

At Razorfish, we were really employing teams of BI developers: ETL developers, data modelers, cube developers, report designers and data scientists. Not only are many of those roles not comfortable naturally in an Agile environment, many of these roles have worked for years in their career previously working solo, not even in teams at all.

You can read through my earlier post and Ken Collier’s excellent (and very small, quick read) Agile Analytics which I also referenced in that link. But today I wanted to highlight 2 important points when considering Agile Scrum for BI solutions that I have run into in these environments, which also came up many times during my time in Redmond with Microsoft:

1. BI engineers are indeed developers. You do not have to write code in Java, .NET or C++ to be a “developer”. An engineer writing ETL and models is writing code, testing, working from requirements and troubleshooting. That means that Agile can work and it also means that “BI Developer” is a role that is very important, deserves the same level of respect and can be very difficult and tedious at times. Give some love back to your BI Developer!!
2. Sprint Reviews CAN work in BI / Analytics solutions and are VITAL to your success. In the Java development world, this may seem like a no-brainer. But I run into a lot of resistance when building Analytics solutions using Scrum. Many folks believe that building a data model, ETL, analytics, reports and demoing all of that in a 3 / 4 week iteration is not possible. It is possible, I have worked in this environment for years and will take you out of that “go off and build-it” approach to business intelligence that has killed many BI projects over the years. Sprint Reviews are your friend!