Is true Business Intelligence still out of reach?

14th June 2009

Most organisations are better at collecting data than turning it into useful information, and although business intelligence solutions can do a lot to alleviate the problem, the gap between hype and reality has never loomed larger. Lesley Meall, FSN contributing editor reports.

“Perez Hilton tweets and then blogs that Lindsay Lohan was just spotted leaving Santa Monica Pier while drinking a Red Bull. Paparazzi swarm the promenade as curious onlookers follow Twitter from their iPhones. Tipped off by their new real-time BI solution, a cutting edge advertising executive receives a mobile alert that there is a spike in Twitter activity about a client. The agency shelves the usual morning Raisin Bran ads for an energy drink campaign, all in real-time. Sales volume surges as the agency establishes web advertising leadership by delivering unique value to their client. The morning ends with executives on their Blackberries discussing the rapid progress made toward KPIs and sales goals while drinking a …”

It’s possible, of course. It’s not just a press release from SQLstream, Pentaho Corporation, and SQL Power Group. “In this tough economic climate, companies need real time business intelligence in order to improve efficiencies and quickly react to changing market conditions,” says Sam Selim, president of SQL Power. “The integration of SQL Stream’s real time technology, Pentaho’s OLAP technology and SQL Power’s Wabit Dashboard will deliver affordable, real time performance metrics to progressive organisations around the world,” he asserts, “and help clients react to new information and fresh business opportunities in real time, thus improving performance on key metrics and positively impacting their bottom line.”

Well yes. The world of 1960’s advertising was exactly like an episode of Mad Men, somebody somewhere is making money out of Twitter (it could be Alicia Navarro at Skimbit), and there is nothing wrong with dreaming. But right now, most experiences of business intelligence are significantly less connected and dynamic than the Red Bull scenario. “In the mid-market, where we focus, businesses are impressed by the fact that we can deliver information on Key Performance Indicators in a meaningful way, because it is so far beyond what they can usually afford or manage,’ says Craig Sullivan, from Netsuite, which uses interactive role-based real time dashboards to provide users with “the business information that they consider to be important.”

Note his careful avoidance of the marketing-misnomer that is ‘business intelligence’. “BI is one of those phrases that has been adopted by various software companies because they want it to mean whatever they do,” asserts Sullivan. “It’s all about attempts to gain market share,” he adds, “and once you get out there in the larger marketplace, you find that very few businesses actually use the term business intelligence.” While this may be true in the mid-market, the world’s largest organisations are a little more evolved. But even vendors that ply their wares at the top end of the corporate marketplace, where the concept of business intelligence has more traction, acknowledge that it doesn’t always deliver on its promises. 

“A lot of people still can’t get at the data that’s locked up in their systems,” admits Chris Field, business consulting manager and BI expert at Infor. “I know there has been tendency towards ‘wizzy wozzy’ systems with flashy dashboards,” he says, “but I don’t think they delivered the value, and people are now turning to more staid- looking products,” that can actually help them to get value from the data in their systems. “There is now a trend towards the integration of budgeting and planning into BI, and making this all part of corporate performance management,” reports Field. “People still want high level data,” he adds, “but in the past, BI had a backward focus, and now people want to know what the future looks like.”

Predicting the future has never been easy, and it isn’t getting any easier, because the world is becoming increasingly complex and interconnected. The amount of data that any organisation or individual must process and analyse, before making an ‘informed’ business decision, is mushrooming. For BI or CPM to provide anything even resembling a complete picture, the technology would need to factor in real time data from thousands of sources, and handle not just structured information, but the sort of unstructured (and incompatible) data that is generated by blogs, electronic sensors, emails, TV broadcasts, weather reports, web pages, and myriad other sources. Take it to its natural conclusion, and you eventually enter the realms of chaos theory.

But somewhere between here and there, IBM recently introduced System S, which uses “stream computing” to offer perpetual analytics - think business intelligence on steroids. It isn’t easy to continuously process the massive amounts of incoming data streams that could potentially affect critical decision-making, so traditional BI systems retrospectively analyse stored data, and then use it to respond to queries from business users. System S has been designed to help businesses become more “real-world aware”: by processing and analysing hundreds or thousands of simultaneous raw data streams in real-time, and dynamically adapting to changing user requirements, it can assemble and serve up answers “on the fly”.

As part of IBM’s InfoSphere product line, System S is at the cutting edge of analytics and business intelligence, so it has the potential to deliver a BI experience that is a world away from what most organisations are used to; but most organisations will not be using it. At the moment, the Swedish Institute of Space Physics and the Marine Institute of Ireland are cited as System S clients by IBM, and with the help of its Blue Gene supercomputer and TD Securities, it has produced a prototype that it describes as “the world’s fastest financial analysis system” – and it can achieve a 21-times performance improvement on the volume of data consumed by today's financial trading systems.

It’s exciting stuff isn’t it, and eventually, at some point in the dim and distant future, your business just might be able to benefit from it. But it wouldn’t do to pin too many of your hopes and dreams on the prospect. Because while IBM can come up with systems that make HAL 9000 seem more evolutionary than revolutionary, and marketing spin can give the Hollywood block-buster treatment to BI (and deliver it via a perfect Web 3.0 ecosystem), the reality is much more mundane. Most businesses are still struggling to make informed decisions because they have too much historical data and not enough business information – and it will be some time before BI can eradicate the gap between hype and reality.