One of my new year’s resolutions – or at least, something that got added to my list of stuff to do in the unlikely event I’ve got some time spare and can be bothered – was to learn more about statistics. I’ve only got a very basic grasp of the subject but, like data mining, it’s one of those things that seems to promise to be incredibly useful in my line of work. However it’s interesting to ponder that I’ve been working in BI for almost a decade and never so far needed to learn much beyond basic stats; my theory is that stats, like data mining, only tends to be used by highly skilled quantitative analysts, whereas the people I work with are business people whose maths skills are very average and who quite rightly don’t trust analysis done using methods they can’t understand.

Anyway, in my browsing on the subject I came across the all-of-a-sudden popular topic of R (see http://www.r-project.org/), the statistical programming language. I thought it might make an interesting blog entry, but today I saw John Brookmyre beat me too it so I’ll just link to him instead:
http://blogs.conchango.com/johnbrookmyre/archive/2009/01/14/are-you-r-ing-yet.aspx

I also got interested in learning about F#, the functional programming language that will be included in VS2010 (for a good overview, see http://www.developer.com/net/net/article.php/3784961). I was struck by some similarities with MDX and began to wonder about how it could be applied to BI; and yet again, a quick Google revealed Aaron Erickson had had the same idea and blogged extensively and intelligently on the subject:
http://blog.magenic.com/blogs/aarone/archive/2008/04/23/On-Business-Intelligence-and-F_2300_.aspx
http://blog.magenic.com/blogs/aarone/archive/2008/09/07/F_2300_-Business-Intelligence-Case-Study-_2D00_-XBox-Live-Trueskill.aspx
http://blog.magenic.com/blogs/aarone/archive/2008/12/11/One-step-closer-to-F_2300_-for-Business-Intelligence.aspx

It’ll be interesting to watch the uptake of F# in BI; from what I can see there’s already a lot of activity in the area of data manipulation and stats for F# (see for example Luca Bolognese’s blog) and I’m sure it’s only going to grow. The only complaint I’ve got is that here’s yet another addition to the Microsoft BI toolset and I’m yet to be convinced there’s any kind of company-wide strategy aimed at shaping all these tools into a coherent BI strategy. F# won’t be the language of BI in the way that Aaron wants; it’s more likely to end up as a technology island in the way Aaron specifically doesn’t want. But hey, the .NET guys have arrived at the party! The more the merrier.

Interesting press release from Netezza here:
http://www.netezza.com/releases/2009/release010609.htm

At the moment it only looks like there’s an OLE DB provider available, but the release says this is only the first part of the strategy. I wonder if Netezza is being considered as a supported data source for Analysis Services so it could be used with cubes in ROLAP mode, as with Teradata today?

Just had one of my occasional looks at Richard Tkachuk’s site, http://www.sqlserveranalysisservices.com and there’s some interesting new information on the home page. There’s an article on how to handle time intervals in AS (I’ve got some ideas on a different way of handling this, but I’d need to test them out to see how they perform), a note on how the Aggregate function works with calculated measures, and a draft of the AS 2008 Performance Guide that is a must-read:
http://sqlcat.com/files/folders/401/download.aspx

So the SQL marketing people have put together a website to plug SQL2008 share the SQL Server 2008 experience:
http://www.sqlserverexperience.com/

Actually, there are some quite interesting (in a Channel 9 sort of way) videos up there in the ‘For Developers’ section, including a few interviews with members of the AS dev team. Worth a look.

As you may have already seen, Microsoft has just announced it has closed its acquisition of DATAllegro:
http://www.microsoft.com/presspass/press/2008/sep08/09-16DAPR.mspx?rss_fdn=Press%20Releases

More details will be forthcoming at the BI Conference, CTPs will be available in the next year or so and the product is slated for release in the first half of 2010 (for SQL2010 then?). It’ll be interesting to see what form it actually takes exactly. As I’ve said before I suspect it would be easier to get Analysis Services working with this technology than many people think, and I also still wonder whether MS might also be thinking about buying or building a column-oriented database too.

As a counter to this euphoria, here are some interesting links… First, take a look in the discussion in the comments here:
http://www.dbms2.com/2008/07/24/how-will-oracle-save-its-data-warehouse-business/
Plenty of people with an axe to grind, I’m sure, but some points worth considering. Also worth checking out for a dissenting point of view is Kevin Closson’s Oracle blog, which discusses DATAllegro a few times, for example:
http://kevinclosson.wordpress.com/2008/07/07/i-know-nothing-about-data-warehouse-appliances-and-now-so-wont-you-part-ii-datallegro-supercharges-fibre-channel-performance/

It’s always good to take the blog-equivalent of a cold shower before getting too worked up about a new feature or technology. I’m sure that the guys at MS have done their due diligence on DATAllegro, and that come 2010 we’ll have a solution that is way, way more scalable than what we have right now, but let’s also set our expectations appropriately – it is going to be a version 1.0, and the competition isn’t going to stay still in the next two years either.

Here’s the press release from Kalido:
http://www.kalido.com/5a04ea36-c50e-4617-b25d-85ea56a22690/news-and-events-press-center-press-releases-detail.htm

I don’t have any direct experience with Kalido’s products (although I’ve heard good things) but I’d be interested to see the cubes it generates. I wonder to what extent it’s possible to optimise automatically generated cubes?

For your listening pleasure: the latest edition of the SQL Down Under Show is me talking to Greg Low about MDX and Analysis Services. You can download it here –

http://www.sqldownunder.com/PreviousShows/tabid/98/Default.aspx

Is it that time of year again? Yes, here’s the link for the latest BI Survey:
http://www.eu-survey.com/BI_8/BI.asp?lan=1&lin=23

Here’s the blurb:

We would very much welcome your participation in The BI Survey, conducted annually by Nigel Pendse. This is the largest independent survey of OLAP users worldwide. The Survey will obtain input from a large number of organizations to better understand their buying decisions, the implementation cycle and the business success achieved. Both business and technical users, as well as vendors and consultants, are welcome.

The BI Survey is strictly independent. While vendors assist by inviting users to participate in the Survey, the vendors do not sponsor the survey, nor influence the questionnaire design or survey results. You will be able to answer questions on your usage of a BI product from any vendor. Your data will only be used anonymously, and no personal details will be passed to vendors or other third parties.

As a participant, you will not only have the opportunity to ensure your experiences are included in the analyses, but you will also receive a summary of the results from the full survey. You will also have a chance of winning one of ten $50 Amazon vouchers.