What Agile Business Intelligence Really Means: Page 3
The Overall Business Intelligence Process
In traditional BI, reports are pretty invariant and analyses follow predictable patterns, so that "views" can accumulate over time, derived from a stable core. The business agility approach demands that we view BI as constantly changing, in response to the needs of an agile organization, or in order to support proactive NPD or business-process change. Thus, change in traditional BI is an evolution of previous insights — deeper, on larger data sets — while change in agile BI is embedded in the development process and tools — they are designed to change BI processes and solutions constantly.
An agile overall BI process, then, involves three new characteristics:
- It incorporates frequent input from the end user and from the environment;
- It "spirals in on" upgraded solutions, endlessly;
- It emphasizes integration with agile development and innovation.
Let's get more concrete, with an example. Wearers begin knocking your latest running shoe on Facebook, sending photos of dislodged heels. Your BI bot on the Internet, or one of your community, notes this within a day and passes back the product info, the nature of the complaint, and the photos to an event-stream processor at your data center. This notes the odd combination of data, relates it to a similar combination at your help desk, and updates the help-desk interface to support and watch for further complaints. It then passes the data directly to a product developer, as well as alerting media communications and marketing, and consolidates and aggregates the data to identify further insights from the complaint. While media communications deals with immediate fallout, the product developer figures out that for a certain type of runner, the resulting walking gait places unusual stress on this particular type of heel. Rather than attempting to fix the heel only for this particular case, the developer queries past data on shoes for this type of runner and finds, counter-intuitively, that moving the heel forward instead of extending it back or sideways works better. Quick prototyping reveals that the fix not only avoids dislodged heels for this type of runner, but also encourages most wearers to place more weight on the balls of their feet, giving them a feeling of greater energy. Marketing sells the first prototype to runners as "energy and fashion," while the developer continues prototyping to create a new product for the broader market. Manufacturing re-tools easily to handle the new form factor, and analytics is modified in a "spiral" fashion to search for additional heel patterns that work, both inside and outside the organization. The new insight eventually leads company strategists to ask if foot and leg wear can be modified to encourage improvements at each stage of the human lifecycle. The company redefines its market as "humanity in motion," incorporating footwear, legwear, exercise equipment, and casts that improve human motion — and make the wearer look energetic. The targets of BI and analytics are modified accordingly, semi-automatically. And then the next insight (which used to be known, quaintly, as a "complaint") arrives.
The Shortcomings of Present "Agile BI"
So why do I say Dyche's approach falls short? My take is that her idea of agile BI incorporates four elements:
- Slice "business capabilities" thinly, deploy them quickly ("every 60-90 days").
- Agile BI is a "delivery process", a particular type of formal process for creating new BI capabilities.
- Organizations should accept that their BI will be upgraded in smaller increments.
- Make sure agile BI software development processes take the time to understand the value to the business.
Here's how I think this falls short:
- Delivering "business capabilities" in thin slices is clearly making a long-term plan, then following it with relatively little deviation. Yes, there is room for feedback from on high as to "business value," but not much consideration of information targets or information sources other than traditional ones, nor enough ability to vary functionality based on customer feedback or the evolution of user needs.
- Agile BI is far more than a delivery process; it is a way of thinking about every BI process, as well as BI's effects on the business as a whole. By restricting agile BI to delivery processes, Dyche's approach fails to improve the agility and effectiveness of the BI information-handling process (discussed above) and to identify new applications of agile BI that would deliver greater "bang for the buck" — like embedded analytics in NPD or "self-adjusting" business processes.
- If all the organization has to do is to accept that BI will be upgraded in smaller increments, it will perpetuate a mindset in which the business accepts whatever BI dishes out. Instead, the organization should be changing its mindset to "thinking agile", and the aim of the consulting organization should be to empower the organization with the agile approach so that it can operate on its own, coming back to the consultant not for the next project but for best practices.
- The idea that agile BI development is too focused on speed seems to me to be a dead giveaway, not only that some organizations are trying to be agile "on the cheap", but also that Dyche believes that development should listen to experts who know BI. That, in turn, can lead to development typically getting user feedback at third hand and later, rather than establishing direct connections to end users and using those connections frequently. That, in turn, will inevitably lead to long-run slower response to unexpected change, lack of proactivity, and "fighting the last war."
Over all, then, Dyche's approach allows somewhat more agility in initial BI delivery, at the price of potential long-term cementing of un-agile processes. I would predict that the initial results would be more rapid delivery of higher-quality BI — after which everyone would declare the war won and move on to the next buzzword. Few will notice that agile analytics has not been a silver bullet, since it doesn't deliver permanent competitive advantage. But real agile BI should.
One of my frustrations in assessing the horde of products that are now flooding the computing market with marketing claims of "agility" is that I feel compelled to tell both vendors and users that, in fact, their solutions are far from achieving full business agility. I am sure that the reaction is often, "What arrogance this guy has to tell me that on the basis of very little knowledge of my product/organization!"
In fact, I have a very simple "smell test" about the product or organization. If the presenter is clearly thinking in agile terms — then business agility is a real possibility. But if the presenter starts focusing on speed, or "flexibility" from modularity and interoperability, or better preparation for the future from better predictions, or better management of anticipated risks, a red flag goes up. These folks are not thinking about how to design their processes to constantly adapt to the unexpected, nor how to improve innovation by better exposure to the outside environment. They are proceeding down the same old organizational path that, system dynamics has suggested, may lead to eventual overload and breakdown of a constantly patched business process. If the organization is not thinking agile, the solution is unlikely to be agile.
The problem with agile BI today, therefore, is fundamentally a failure of imagination. Clearly, Dyche is an example of the best that agile BI has to offer; and she pays diligent attention to folks like Stephen Swoyer, who have paid their dues in the agile development community. And yet, I find it easy to generate many more ideas than are obvious in her blog on how to use BI to improve business agility; and few if any other BI folks that I hear about really seem to be generating such ideas, either. They are, instead, using a narrow idea of "agility" to bless their products and approaches.
My recommendation for buyers looking for "agile BI," then, is to brainstorm, not about what faster or deeper BI will buy you, but rather about how you can make every part of your organization agile, and then how BI can help. I have noted a few of the tools out there that should play a role — data visualization, event processing, master data management with global metadata repositories. The rest is demanding that the tool or vendor or business process fit your agility vision, not accepting that you must accommodate the shortcomings of existing tools. You already know that you can buy useful BI. If you don't try, you'll never know whether you can buy agile BI for the agile organization.