A Couple of Interesting OBIEE Surveys

Mark Rittman -

I've been analyzing the results of a couple of OBIEE-related surveys over the weekend, looking to understand how we can encourage the further take-up of OBIEE by developers and organizations. The BI Survey 9 (which readers of the blog helped to provide answers for) recently published its results, and as well as providing data on the success (or not) of BI projects across all vendors, gave some specific feedback on the results of just under fifty OBIEE projects around the world. In addition, I've been going through the results of the new-member survey for the OBIEE Enterprise Methodology Group, to get some statistics on OBIEE deployment and to see what feature requests OBIEE developers would pass on to the OBIEE development team, on the basis that if they were implemented they might encourage further take-up of the product.

Starting with the BI Survey 9, this is a vendor-independent, in-depth survey of BI customers and implementers that aims to understand the success factors behind BI Projects, and also what can make them go off the rails. I get a complementary copy as I help to organize Oracle responses, and I'm able to quote a few statistics from the report. Here's some interesting ones:

  • Finance, Sales, IT and senior management are the main business functions using BI
  • Still, very few companies evaluate and buy open source BI products, but of those, Jedox and Pentaho have the best improving "win rate"
  • The main benefits reported on a BI project are faster and more accurate reporting, better business decisions, improved customer satisfaction and saving non-IT costs
  • The main reasons for project failure are poor data quality (18% of projects), slow query performance (17%), company politics (13%) and failure to agree requirements (13%)
  • US, UK and Germany are the biggest BI markets
  • Interestingly, functionality and end-user ease of use were the most prevalent reasons for tool selection (with query performance rated quite low), but query performance and poor data quality ended up being the main reasons for projects failing
  • Projects had a median rollout time of six months, and those that took longer tended to suffer significantly reduced business benefits. Also, more modest projects (typically under $25k in consulting expenditure) tended to have the best ROI

Looking at OBIEE specifically, it was a mixed set of results. OBIEE scored well in terms of scalability, ability to deploy to the web and integration with other products. In terms of business benefits delivered, its strength seems to be in reducing costs and headcount in business/functional departments, which would correspond with my findings in the field. Average rollout times were in line with the industry median, at around six months compared to nine and a half for Business Objects and seven and a half for Cognos (QlikView, in contrast, was around three months). Where it fared badly were in terms of value for money, product quality and query performance.

In particular, some OBIEE-specific observations from the survey were:

  • Main deterrents to wider rollout of OBIEE within existing customers were cost of software, administration complexity and cost of implementation
  • OBIEE had the lowest win-rate when in multi-vendor evaluations, and when it was chosen it was primarily because of vendor reputation, integration with other products and because it was the corporate standard
  • The main reasons for OBIEE project problems were query performance too slow, and security limitation in the product (which surprised me)
  • The main business benefits delivered were saving non-IT costs and headcounts, and better reporting

So in summary, OBIEE in at least it's 10g guise tends to get chosen because of the Oracle name, and because of integration with the rest of the Oracle stack. Functionality is rated in the survey as so-so with problems reported around query performance and complexity of administration. Its strengths include integration with the rest of the Oracle tech stack and other applications, scalability, timeliness of data and web delivery. Realistically, nothing that we didn't know before and most of the reported weaknesses are of course focus areas for the 11g release.

So, the BI Survey 9 is a useful survey of customers, prospects and implementers working with OBIEE and other BI tools. The OBIEE Enterprise Methodology Group is an interesting complement to this survey though, as it asks questions of what we might call "OBIEE experts", or at least people who work closely enough with OBIEE, or are enthusiastic enough about it, to join a technical forum for the product.

In the OBIEE EMG survey, we asked a number of questions about how OBIEE was deployed in their organization, what data sources and platforms were used, and also interestingly, what features would they request of Oracle in a potential future release. This last question is interesting if we're looking at what might restrict usage of OBIEE by developers, assuming that these are features that are missing that they would otherwise expect to see in the product.

First of all, some highlights of the deployment questions:

  • 92% of members are running OBIEE 10g; 14% were running the 11g beta. 35% are running the BI Apps. 22% are running Essbase, 16% Oracle OLAP. For ETL tools, 36% are using Informatica, 30% OWB, 26% ODI
  • 93% of members have Oracle RDBMS as their main database; 5% MS SQL Server, 2% other (DB/2, Teradata etc)
  • 50% use Windows as their primary production OS platform; 30% use Linux; 9% Solaris; 5% AIX, 4% HP-UX
  • For those using the BI Apps, 30% have Siebel CRM as the main data source, 70% have Oracle EBS, 7% Fusion Apps, 3% SAP
  • Of EMG members, 36% are customers, 33% are Oracle partners, 10% are from Oracle, 15% are contractors
  • 34% also use Business Objects in their organization; 19% haveCognos, 44% have Discoverer, 60% use Excel, 7% use QlikView

For the "new feature" question, there were some common threads.

  • Software configuration management (SCM) with OBIEE 11g is seen by members as one of the tool's weak points. There's no standard, out-of-the-box version control, patching (in 10g) was hard to do (web catalog) or impossible in a supported way (RPDs), and multi-user development in particular is a real bugbear, with a lot of members requesting that the RPD move into a proper relational database thereby removing the complications of working with a single, monolithic file
  • Better integration with Essbase was a common thread, and I think this is something that's actually been addressed well in 11g
  • There was a lot of demand for additional visualizations (such as micro-charts/spark-line charts) plus more flexibility in pivot tables, dashboard prompts and graphs. This is partially addressed in 11g (dynamic pivot tables in the dashboard, hierarchical columns and so on) but the basic range of charts is still the same. I hear this might be addressed in later releases in the 11g cycle.
  • An API for the RPD (repository). 11g actually delivers a systems-management API, but I suspect an API for the repository won't come until we perhaps moved towards an integrated, Fusion Middleware-wide repository.

Interestingly, query performance never came up, though better integration with Essbase might be a proxy for this.

So, to summarize: what's holding back wider deployment of OBIEE? Well, in my opinion and based on the two surveys:

  • Cost is a tricky one. Enterprise BI tools are never cheap, but OBIEE came out as most expensive in terms of software license fees, and had the second worst "cost of ownership index" score, which factors in license fees, implementation costs and spreads this over the number of seats deployed.
  • At least in 10g, it's seen as tricky to administer, and developers think there's a long way to go in terms of proper SCM.
  • The BI Survey suggests a worrying lack of success in competitive evaluations; in fact, OBIEE is bottom of the table for these
  • Query performance seems to be an issue, even compared to other enterprise tools such as BO, Cognos and MicroStrategy
  • Functionality (at least in 10g, when the survey was carried out) appears "so-so", lacking some of the newer innovations from other BI tool vendors, though it does well in terms of scalability, data latency, web capabilities.

And what can Oracle do to address these issues, to ensure that OBIEE gets wider take-up in the marketplace? Again, just my opinions, but here's some constructive thoughts:

  • OBIEE 11g has delivered lots of welcome new BI functionality, but other areas that could be addressed include improving the visualizations, extending the types and numbers of charts, introducing what-if and predictive analytics into the dashboard, providing better in-memory analysis, improving the user experience, adding proper collaboration and annotation of reports, and sorting out the MS Office integration (BI Office is too limited, SmartView isn't ready yet for OBIEE)
  • To keep the developers and admins on side, improve software configuration management within the tool including out-of-the-box integration with a range of version control systems, keep up the work on improving patching, move the RPD to a proper database and simplify the whole MUD, online/offline RPD handling process
  • Address perceived query performance issues, either by full integration of Essbase and Oracle OLAP into the aggregate layer, or perhaps by leveraging products such as TimesTen or Hyperoll)
  • Reduce complexity and redundancy across the BI/EPM product stack in the areas of metadata, design, build and administration - basically, one place to define dimensions, metrics, KPIs, business rules and security, across BI, ETL and EPM
  • Recognize the attraction of tools such as QlikView, Tibco Spotfire and Microsoft PowerPivot for business "power-users" who want to build BI systems without (initial) IT involvement and an initial data warehouse

Anyway, if you're interested in reading more, The BI Survey 9 is available for purchase and download, and the OBIEE EMG Forum posting has full details on the member survey, including downloadable results and a full listing of the feature requests.