I see Google have announced their own web application platform:
http://www.techcrunch.com/2008/04/07/google-jumps-head-first-into-web-services-with-google-app-engine/#comments
which includes BigTable as part of it (remember this post?). There’s speculation over whether Microsoft has something similar up its sleeve:
http://blogs.zdnet.com/microsoft/?p=1320

This and the Panorama and Good Data stuff I blogged about over the last few weeks make me quite excited. What will the next generation of OLAP/BI tools be like? Surely it’s a mistake to think of them as hosted versions of what we’ve got today. As I’ve said before, the attribute-based approach of databases like BigTable remind me of Analysis Services dimensions; wouldn’t it be cool just to be able to grab data from a number of these stores and use them as dimensions and fact tables? Maybe through a front-end that was something like Lotus Improv (thanks to Andew Wiles for directing me to this in a recent conversation) on the web? But with an XMLA interface too? And since in the cloud hardware scale-out will presumably be just a matter of paying a bit more cash, you’d want an engine that could handle that transparently in the way I understand something like Teradata does? Ahh, if only I had a couple of million USD$s of VC funding to waste I’d hire Mosha and set him to work on this…

Possibly a little early to blog as this startup seems to be a while away from RTM, but it looks cool and BI as a service is very topical…

Yesterday a guy called Roman Stanek linked to my blog, so naturally I checked out his blog and his company:
http://www.gooddata.com/

What they’ve got is a "a complete, on-demand business intelligence platform combining analytics, reporting, data warehousing and data integration". The workflow seems to be that you upload your data, then you have an online environment where you have OLAP and collaboration tools; similar, I guess, to what Panorama are working on with Google docs? I guess they’re going to be making their money designing the initial data warehouse/ETL/OLAP design for each customer as well as subscription costs?

This raises some fundamental questions about BI as a service in my mind:

• Will companies be willing to upload their most valuable and secret data to a third party on the web? If they’re willing to use online data stores like Amazon S3 or SQL Server Data Services or other hosted solutions then I suppose so, but that’s a big if – and these online data stores aren’t necessarily positioned to hold the kind of ultra-sensitive data that you work with in a BI system.
• However complete a platform you build, the old adage that all data ends up in Excel needs to be considered. Remember it’s Excel I said here, not any old online spreadsheet. Will the guys in Finance abandon Excel to do spreadsheets on the net? I’m not sure. This is where Microsoft have a locked-in advantage, but it’s not an insurmountable problem for BI service-providers. Why not expose an XMLA interface for your hosted OLAP and let people connect to it directly from Excel? It’s possible for Analysis Services and SAP BW, and I’m sure companies like Simba will be only too happy to Excel-enable other OLAP tools.
• Unless BI-as-a-service companies let customers or partners design and build their own solutions on their platform, will these companies be able to scale out to meet the demand of designing enough apps for potentially hundreds of customers? That’s a lot of BI projects for any one consultancy to run. More importantly, will they be able to do an adequate job of designing each app? One of the arguments that’s always used against outsourcing BI projects is that face-to-face contact between designer and business users is essential, and that’s a view I tend to agree with.

Clever old Panorama… just when you thought that it was about time for them to roll over and die, having been jilted by Microsoft and SAP, they team up with Google to do BI. From their blog, here’s the press release:
http://www.panorama.com/blog/?p=98
and more information is here:
http://google-pivot-tables.blogspot.com/
and there’s a tutorial here:
http://www.panorama.com/google/pivot-table/tutorial/

This is part of the wider Google announcements on Google Gadgets:
http://googledocs.blogspot.com/2008/03/collaboration-goes-one-level-deeper.html
and the Google Visualization API:
http://google-code-updates.blogspot.com/2008/03/introducing-google-visualization-api.html
http://code.google.com/apis/visualization/documentation/gadgetgallery.html

I need some time to digest this, but my first impression is that this is BIG. The PowerApps stuff sounds very interesting indeed.

One of the benefits of being a blogger is all the free stuff that gets sent your way by people looking for some publicity. Unfortunately in my case the stuff I get isn’t all that sexy, ie no free XBoxes, but it’s still interesting – the books are always welcome, the free licences for AS client tools are useful too, and yesterday I got a freebie copy of the results of Nigel Pendse’s "BI Survey 7" with an invitation to blog about the contents. Now I’m sure over the next few weeks you’ll see the marketing machines of all the BI vendors crank into action, cherry-picking the findings so they can say things like "XSoft is the most reliable BI platform" and "YWare offers the best query performance" etc etc. But what does it actually have to say about Analysis Services?

Firstly, some characteristics of Analysis Services deployments. This year 34% of respondents were still using AS2K, with 66% now on AS2005 (with no-one using OLAP Services any more). This is almost a complete inversion of the usage figures from the previous survey, indicating most people migrated from AS2K to AS2005 in the last year. This is certainly what I’ve seen in my consultancy work, but I’m still surprised so many people have migrated given that the two platforms are so different. Another interesting point made is that while you’d expect AS to be the dominant OLAP tool when SQL Server is the data source, it’s also the most common OLAP tool for Oracle, Sybase and DB2 sites and it comes in second (behind Microstrategy) for Teradata. Oracle I can understand, because I work with Oracle data sources as much as I work with SQL Server (though in the past I used to work more often with Oracle, in fact) but I’ve never yet worked with anyone using DB2 or Teradata and only one person using Sybase, which isn’t in fact officially supported as a data source for AS. Maybe I don’t get around enough. I bet this is uncomfortable reading for Oracle, especially, since the only Oracle BI product that features in the top ten BI tools used against an Oracle data source is Essbase and that’s at #10.

One of the things that people often bring up about AS is that people only use it because it’s bundled free with SQL Server. That’s certainly a big part of why it’s chosen, and the survey shows it’s very common that when it’s used it doesn’t go through a formal evaluation process or that it’s the only product evaluated. But that certainly doesn’t explain all the Oracle, IBM and Sybase shops that use it and the survey also shows that when a formal, multi-product eval is conducted then AS wins 75% of the time – putting it ahead of all of its major competitors. Again, that’s consistent with my experience: I was working with a customer recently where they’d done an eval comparing AS with Oracle’s OLAP option and Essbase, and neither of the latter could handle the data volumes and dimension sizes that AS2005 could.

Regarding the problems faced by BI projects, it’s not surprising that slow query performance is the most common across the board for all products. AS is slightly above average in that 20% of respondents complained about this (maybe they need to get a specialised consultant on board to do some tuning? Now, who could do a job like that?); TM1 does the best at only 6.6%, SAP BI does worst at 37.5%. One thing that AS does worse on than average is ‘Security Weaknesses’, which to be honest is a bit strange given that I’ve never found any holes in the product that would lead to values being shown to people who shouldn’t see them. What I suspect is happening here is that although AS has a great set of options for security it’s still way too difficult to configure, especially for complex scenarios (see here for my experiences, although I wonder whether other products could meet these requirements at all) and for large numbers of users. There’s a real need for a better, more end-user friendly interface for managing roles, and perhaps it would be good to be able to use dynamic security right out of the box rather than have to implement it manually.

Lastly, there’s a good section on the client tools used with AS. I cannot believe that as many as 6.6% of people are still using Data Analyzer as a client tool – what planet are they living on? The high percentage using pivot tables I can understand, and there’s a very long tail of people using tools that have only got 0-4% of market share. The survey makes the interesting point that fewer AS users use any kind of Excel front-end than users of TM1, Essbase or SAP BI; maybe that’ll change as more companies move to Office 2007.

Anyway, hopefully I won’t get into trouble for divulging too many details but I can honestly say it’s a fascinating read and at 420 pages long there’s a lot that I haven’t mentioned here, and it’s more substantial than a lot of IT books. If you’re a vendor of any kind I’m sure you’ll be buying this anyway, but I’d also recommend it if you’re a consultancy (lots of juicy facts to quote to prospective customers to trash the competition) or you’re about to embark on a BI project in-house. Overall, AS comes out of it very well which is obviously good news for me and I guess the majority of people who read this blog. Of course Nigel has long been very positive about the MS BI stack, much more so than other analysts although Gartner has suddenly got very positive too. I don’t know about anyone else, but sometimes I get a bit miffed by the way some BI journalists talk about Cognos and Business Objects as serious platforms but never mention Microsoft at all… hopefully that’s starting to change.