SQLBits 8 Registration Now Open

We’ve just opened registration for SQLBits 8, which will be taking place in Brighton (in the UK) on the 7th-9th April. Full details can, of course, be found on the SQLBits website at http://www.sqlbits.com/ and you can register here.

It’s going to be a full three-dayer, and we’ve got loads of amazing stuff planned for it that we can’t talk about just yet, but there’s still plenty we can say publicly. For example, for anyone interested in Microsoft BI on the pre-conference seminar day we’ve got Marco and Alberto on their SQLBI methodology and Thomas Kejser on optimising the MS BI stack amongst other things (though not me this time). Also, take a look at the list of sessions that have been submitted – there’s some great stuff there too. It’s the SQL Server event of 2011 in Europe, without a doubt (especially since it doesn’t look like there will be a PASS European Conference this year).

Hope to see you there!

Google Docs can access data from BigQuery

I don’t know how I missed this news from last week (luckily I saw it relayed here), but this is important: Google have announced that you’ll be able to access data stored in BigQuery direct from Google Docs:
http://googledata.org/google-docs/bigquery-meet-google-spreadsheets/

I’ve been following BigQuery and Google’s BI efforts for a while, and it looks like it’s only a matter of time before they have something resembling a pivot table built into a Google Docs spreadsheet – and that would be a very significant development in the world of web-based BI. Think about it: vast amounts of data stored in BigQuery, accessible through a SQL-like interface and able to be queried very quickly; a spreadsheet on the front; all that’s needed is some kind of OLAP-like-layer in between to make it easy for end users to build their own queries. And Google have done similar things to what’s necessary here with Fusion Tables and their collaboration with Panorama

Microsoft really need an answer to this. We know SSAS in the cloud is coming, and we have the Excel web app, but again we need pivot table support in the web app to complete the picture (see here and here for more discussion on this topic). I hope it comes soon!

PowerPivot Videos from SQL Server Day 2010

Last month I had the pleasure of speaking at SQL Server Day 2010 in Belgium (I had a nightmare getting there and back because of the snow, but that’s another story). I’ve just seen that all the videos from the event are now available to view online and download here:
http://sqlserverday.be/video/

Both the sessions I gave are up: “Implementing Common Business Calculations in DAX” and “Comparing Analysis Services with PowerPivot”.

On a related note, we’ve aggregated all the content we’ve got for every single SQLBits on a new page on the SQLBits site:
http://sqlbits.com/content/
As far as I can see there are 152 videos on there alone!

Tuning the Currency Conversion Calculations created by the Add Business Intelligence Wizard

I see the ‘Add Business Intelligence’ wizard in BIDS as a bit of a missed opportunity on Microsoft’s part: it was a great idea to have functionality that would automate difficult stuff like adding MDX calculations to a cube, and it’s a shame that most of the MDX code it generates is so rubbish. Take, for example, the MDX currency conversion calculations that it creates. If you’re using Enterprise Edition the best way to do currency conversion is not to use MDX at all but to use measure expressions, but if you have Standard Edition you have no choice but to use MDX to do your currency conversion and if you use the calculations the wizard produces you’d probably think that this type of calculation has to be painfully slow. That’s not true, though – and in this post I’ll show you how to replace the currency conversion calculations created by the wizard with some that are not only much simpler but also much, much faster.

Let’s use a simple cube to illustrate this. In the Adventure Works database I created a cube with two measure groups: one based on the Internet Sales fact table, the other based on the Currency Rates fact table. I then added the Currency dimension and the Date dimension (joining on OrderDateKey in the Internet Sales table), and ran the ‘Add Business Intelligence’ wizard to add many-to-many currency conversion to the cube. Here’s what my Dimension Usage tab looked like after I’d run the wizard (note that it added a new Reporting Currency dimension to the cube):

image

And here’s the MDX that it adds to the MDX Script:

// <Currency conversion>
       
        // Currency conversion wizard generated script.
        // Currency conversion generated on: 11 January 2011 21:05:19   
        // by user: Chris   
        // Currency conversion type: ManyToMany   
        // Selected members to be converted: Sales Amount   
        // Please be aware that any changes that you decide to make to it may be
        // overridden the next time you run the Currency Conversion wizard again.   
   
        // This is the Many to One section   
        // All currency conversion formulas are calculated for the pivot currency and at leaf of the time dimension   
        Scope ( { Measures.[Sales Amount]} );
            Scope( Leaves([Date]) ,
                [Reporting Currency].[USD],   
                Leaves([Source Currency]));
       
               
             
              // Convert Local value into Pivot currency for selected Measures that must be
              //converted with Measure rate [Average Rate]
              Scope( { Measures.[Sales Amount]} );
                    
               This = [Reporting Currency].[Local] / Measures.[Average Rate];
   
              End Scope;
   

            End Scope;   
       
            // This is the One to Many section
            // All currency conversion formulas are calculated for
            //the non pivot currency and at leaf of the time dimension   
            Scope( Leaves([Date]) ,    
                Except([Reporting Currency].[Source Currency Code].[Source Currency Code].Members,
                {[Reporting Currency].[Source Currency Code].[Source Currency Code].[USD],
                [Reporting Currency].[Source Currency Code].[Source Currency Code].[Local]}));
       
               
           
            // This section overrides the local values with the Converted
            // value for each selected measures needing to be converted with Measure rate [Average Rate]…   
            // LinkMember is used to reference the currency from the
            // source currency dimension in the rate cube.
            Scope( { Measures.[Sales Amount]} );
                   This = [Reporting Currency].[Source Currency Code].[USD] * (Measures.[Average Rate], LinkMember([Reporting Currency].[Source Currency Code].CurrentMember, [Source Currency].[Source Currency Code])) ;
            End Scope;   
   
       
            End Scope; // Leaves of time, all reporting currencies but local and pivot currency   
        End Scope; // Measures

        // End of the currency conversion wizard generated script
    // </Currency conversion>

Scary, eh? I won’t explain what it does in detail, but basically it does the following:

  1. Converts the various local currency values held in the Internet Sales fact table into US Dollars
  2. Converts these US Dollar amounts into whatever Reporting Currency the user has selected

It’s a lot of code for something so straightforward, and the reason why so complex is because it needs to use the Currency Rates measure group for both sets of currency rates: from local currency to US Dollars, and from US Dollars to reporting currency. As we all know overly-complex code is usually poorly-performing code, and in this case the use of LinkMember in particular is a killer; also there are no checks for situations where exchange rate values don’t exist, which is a fairly basic mistake.

Take the following query:

SELECT [Date].[Calendar Year].MEMBERS ON 0,
NON EMPTY
[Reporting Currency].[Source Currency Code].MEMBERS ON 1
FROM [CurrencyTestWizard]
WHERE([Measures].[Sales Amount])

It displays all Years on columns and all Reporting Currencies on rows for the Sales Amount measure, so it’s doing quite a lot of work. On my quite beefy laptop, using the cube I’d just created, it ran in just under 9 seconds on a cold cache to give me this:

image

We can do a lot better than 9 seconds though. The first step to optimising this is, as always, to push complexity out of the MDX and back into the cube design. Given that a fact table containing exchange rate values is usually pretty small relative to other fact tables, what I decided to do was to add a second measure group to my cube based on exactly the same Exchange Rate fact table, use the original measure group for the conversion from local currency to US Dollars and then the new one for the conversion from US Dollars to reporting currency. This did increase both the size of the cube and the amount of time taken to process it, but only by a tiny amount.

BIDS didn’t let me create a second measure group from a fact table that was already in use, so I had to create a new named query in BIDS that did a SELECT * from the Exchange Rate fact table and then use that. Having done this, I was able to set up my Dimension Usage as follows, with Reporting Currency joining to this new measure group:

image

Now, I could replace the code above with the following MDX:

SCOPE({[Measures].[Sales Amount]});
    SCOPE([Date].[Date].[Date].MEMBERS);
        SCOPE([Source Currency].[Source Currency Code].[Source Currency Code].MEMBERS);
            SCOPE(EXCEPT([Reporting Currency].[Source Currency Code].[Source Currency Code].MEMBERS
                        ,{{[Reporting Currency].[Source Currency Code].&[2147483647]}}));

                THIS =  IIF([Measures].[Average Rate – Reporting Currency Rates]=0
                            , NULL
                            ,IIF([Measures].[Average Rate]=0
                                , NULL
                                ,([Measures].[Sales Amount] / [Measures].[Average Rate]))
                                * [Measures].[Average Rate – Reporting Currency Rates]);
            END SCOPE;
        END SCOPE;
    END SCOPE;
END SCOPE;

What I’m doing here is using a single set of nested scoped assignments to perform the calculation at the leaf level of the Date, Source Currency and Reporting Currency dimensions. With two different measure groups containing exchange rates there’s no need for the use of LinkMember – I just use two different measures for each step in the conversion – and I make sure I don’t bother doing any conversions where exchange rate values don’t exist.

When I ran my test query, I got exactly the same results but the query ran in around 0.5 seconds on a cold cache – 18 times faster than on the original version of the cube! It just shows you what a few tweaks to your cube design and MDX can achieve, doesn’t it? It’s a shame that MS didn’t invest a bit more time on the ‘Add Business Intelligence’ wizard to improve it – it would probably have improved the quality of many implementations no end. In the future, with PowerPivot and BISM being focused so much on ‘self-service’ BI, I think something similar to the wizard is a ‘must have’ to help users create complex DAX calculations, and I hope MS realise that functionality like this can be key to the overall success of a product.

PowerPivot Training in London (and elsewhere)

As you’re probably aware by now, PowerPivot and DAX are important parts of the Microsoft BI story going forward – self-service BI is the big new thing, while on the corporate side BISM will use DAX as its calculation language and BISM models will essentially be PowerPivot models. So it makes sense to learn PowerPivot and DAX as soon as possible to get a head start, right?

While that thought’s in your head, I’m pleased to mention that I’m helping Marco Russo and Alberto Ferrari (authors of the excellent book “PowerPivot for Excel 2010”) organise the London leg of their European PowerPivot training tour. Marco has more details on the course on his blog here, and you can find the course website here:
http://www.powerpivotworkshop.com/

Whether you’re a BI professional or an Excel power user, I think this represents an excellent opportunity to get training from acknowledged PowerPivot gurus. I’ll be there in the audience!

Report Portal

As I say whenever I talk about third party products here, I don’t do reviews on my blog, I just highlight products that look interesting and probably deserve closer inspection. So here is a non-review of a client tool I had a demo of today, Report Portal, a thin client, pure-html solution that includes both ad-hoc browsing capabilities and dashboarding/reporting. Although the UI looks a little dated, the fact that nothing needs to be installed on the desktop, that there are no requirements that might fall foul of corporate IT policies (which might, for example, rule out a Silverlight solution), and that it is licensed on a per-server basis do remove a lot of potential deployment headaches. 

Rather conveniently, Igor, the guy who gave me the demo, realised that I’ve seen hundreds of client tools for SSAS and just showed me the features that make this particular product stand out from the pack, so that’s what I’ll talk about. Here are the main points:

  • It automatically creates a number of date calculations and relative time selections (such as ‘current month’, ‘previous month’) for you, meaning you don’t have to develop them yourself.
  • On time dimensions you can set up selection by a date picker, and also do date range selections by selecting a start and end date.
  • It can also do cascading parameters rather like what’s possible in SSRS or with Excel 2010’s slicers.
  • For drillthrough it allows you to build your own drillthrough query, select which measures you want, which attributes/columns you want and the order you want the columns to appear
  • There are a wide range of charts and visualisations to choose from, including an interesting (although possibly not Stephen-Few-approved) pie-chart tree report.
  • There’s also a load of other features, such as the ability to embed SSRS, OWC, SQL-based and other report types in dashboards; it supports writeback; it allows you to save comments in cells; and there’s also a blog and a wiki inside the portal.

Overall, it’s a solid, mature product that’s been around for six years and does pretty much everything you’ll want. Definitely looks like one to add to the shortlist if you’re looking for a zero-footprint SSAS client tool.

Sixth Blog Birthday

Every year on December 30th (or thereabouts) I write a blog post summarising what I’ve been up to this year, to mark this blog’s birthday. Today is my blog’s sixth birthday which, to be honest, I find pretty amazing – but then every year I’m amazed at how long I’ve kept at this! I only wish I could apply the same kind of willpower to making cash (I’d be a billionaire by now) or dieting, exercise or anything else…

From a blog point of view, the big change this year was moving away from Windows Live Spaces, where I’d started out, to WordPress. I’d been wanting to make the move for ages but had been too lazy to do so, and also unwilling to lose all my old posts and traffic; luckily when MS made the decision to retire Live Spaces they made sure it was very easy to migrate all my old content to WordPress and for that I’m very grateful – it almost makes up for the shabby treatment Live Spaces users had been getting in the preceding years.

I also got one of my biggest ever days from a blog traffic point of view during the controversy over the future of SSAS at PASS; I’m not even going to link to that particular post because it still gets plenty of hits and it’s something I’d prefer to move on from. It’s better to look forward to the undoubtedly cool stuff that will be coming with Denali rather than brood over any disappointment about what has happened, and one thing I’m certainly relieved about is that in 2011 there will be lots of cool stuff to blog about – at some points in the last year I’ve struggled to find anything new or interesting to blog about for SSAS (although as Greg Galloway has just shown, there’s still lots of interesting SSAS topics to blog about if you look). Bring on BISM!

Professionally things go from strength to strength and I’m going to be busier in the first half of 2011 than I’ve ever been in January/February, which are usually relatively quiet times for me. Again, with significant new technology arriving in the BI stack in Denali I would expect a lot of consultancy and training work to follow; I’m going to be expanding my training activities in other ways next year too, and I’ll be making a separate announcement about that here any day soon. SQLBits also continues to grow and grown, and I’m really excited about the plans for SQLBits 8 in Brighton this April. Since it doesn’t look like there’s going to be a PASS Europe this year I would say that anyone who’s serious about SQL Server in Europe should plan to come to what’s going to be the biggest and best SQL conference in Europe bar none; and even if you live outside Europe, but fancy a vacation, why not come too? We have an ever increasing number of US attendees and speakers!

Anyway, it’s time for me to go and do some real work and stop being self-indulgent. I hope everyone reading this has a great 2011. Thanks for reading!

New Trace Events in SSAS2008 R2 CU5

Greg Galloway has just pointed out to me a new KB article outlining some major improvements/changes to trace events that are coming in CU5 for SSAS 2008 R2:
http://support.microsoft.com/kb/2458438/

This represents part of the new monitoring functionality that will be coming to SSAS in Denali. These changes will allow us to:

  • Monitor what happens during the execution of the MDX Script in more detail
  • Track resource usage for individual commands (this alone is very interesting)
  • Track locks more effectively

These changes are going to be very useful to anyone running a high-end SSAS implementation.

SQLBits 8–Beside the Sea!

The dates for the next SQLBits have been announced: it’s going to be on April 7th-9th at the Grand Hotel in Brighton (for those of you outside the UK, Brighton is a very cool town on the south coast of England, very easy to reach from London and Gatwick Airport). More details, as always, are to be found at http://www.sqlbits.com/

We’ve also posted most of the videos from SQLBits 7, which are available to view for free from the SQLBits site at the link above. Stay tuned for more announcements about the exciting stuff we have planned for Brighton!

Google Books Ngram Viewer

This has been blogged about quite a bit over the last few days, but if you’re looking to kill some time at work with something vaguely BI-related then take a look at Books Ngram Viewer, the latest thing on Google Labs:

http://ngrams.googlelabs.com/

It’s pretty simple: it allows you to visualise the number of occurrences of words in books held in Google books by year of publication, and therefore track the popularity of ideas and concepts over time (well, that’s the theory). You can even download the source data – if I’ve got some spare time it might be interesting to see this data in PowerPivot…!

There are already some interesting examples of things you can see with this data out there, for example on the Information Is Beautiful site, and on the Google Blog post announcing it. I can’t resist adding one more, though: Microsoft vs Oracle vs IBM vs Google – notice how IBM peaks in about 1988 then declines; Oracle and Microsoft have a similar peak in 2004 and then decline; while Google is still on the up…