The PASS Summit is over for another year and I’m just starting out on the long trip back home, so there’s plenty of time to get my thoughts together on what’s happened over the past week. In fact there’s not much to say about the event itself: it was, as ever, a lot of fun and totally worthwhile. Hey, within 30 minutes of arriving at the conference I learned I’d won an award for the best BI-related blog entry, for my post on implementing real SSAS drilldown in SSRS!
Attendance was up from last year although probably the recession still took its toll: remember that there was no BI Conference this year and I would have thought that a lot of people who would have gone to it would have gone to PASS instead. To be honest I think not having a BI Conference is a good thing, actually. I don’t like having to choose which conference to attend, and part of the benefit of a conference is to get as many members of a tech community together in one place. And this was certainly the largest gathering of Analysis Services people I’ve ever seen: all the usual crowd were there, I met a lot of people who I’d only met a few times before, and I finally got to meet Darren Gosbell in person after having known him by email for at least five years. One complaint I would make about the event was that the sessions weren’t scheduled particularly well. I know everyone always complains about this but in this case it did seem worse than usual: my session, for example, was up against two other SSAS-specific sessions, but in other cases there were time slots with no SSAS content at all.
The other benefit of PASS is that you get to talk at length about what’s going on in the world of SQL Server with other like-minded people. As a result you get to crystallise your thoughts on a lot of matters and – guess what – I’m going to share mine here.
First of all, the topic that was on everyone’s lips was PowerPivot. In fact everyone at the conference must have seen the standard demo at least five times and there were also a lot of advanced sessions on it too. Don’t get me wrong, I really think PowerPivot it cool from a technology point of view, I am going to take the time to learn it, and I also think from a make-money-by-getting-people-to-upgrade-to-Office-2010 point of view it is a very clever move for Microsoft. But my feelings about it remain ambiguous. Quite apart from the arguments about it discouraging ‘one version of the truth’ and encouraging spreadmarts that have already been discussed ad nauseam, I have another problem with it: I don’t honestly know whether I, as a consultant, will be able to make any money from it. The very nature of it, as a self-service tool, means no expensive outside consultancy is necessary. I don’t think it will take business away from me though; it will be widely used and it will be used instead of regular SSAS for more basic projects, but the more serious stuff will stay with SSAS I hope. I think the need for sophisticated security and more complex calculations will be the deciding factor when people choose between SSAS and PowerPivot; I’m not sure I see many people upselling from PowerPivot to SSAS either. We’ll see.
Something that worries me more about PowerPivot is the fact that it seems to have diverted the attention of the SSAS dev team. For SSAS 2008 we had few new features, although the performance improvements were very welcome. For 2008 R2 I can only think of one new feature in SSAS, and that’s the ability to use calculated members in subselects that will allow Excel 2010 to use time utility dimensions properly (I’ll blog about that at some point). Even though work on good old server-side SSAS will resume for the next major release of SQL Server I worry that PowerPivot will take priority in the future. If this happened it would be bad for me and other BI partners from a business point of view, and seems crazy given that SSAS has been such a successful product in the enterprise sector; it’s not like there aren’t a lot of new features and fixes that could be done. Shades of IE6 and Microsoft getting complacent once it’s cornered a market, I think.
Last of all on PowerPivot, I suspect that there is something new relating to it in the roadmap that hasn’t been announced yet. David DeWitt devoted his keynote on Thursday to it, the specifics of column-store databases and the Vertipaq engine (which is the new in-memory storage engine that PowerPivot uses), and at the end hinted at this saying that although he couldn’t make any announcements, those people who had been paying attention might have some ideas on what the future held for it. Of course I hadn’t been paying attention properly, but the obvious thing would be to integrate it with the relational database somehow. Given that PowerPivot is now being hosted inside Sharepoint, why not host it in SQL Server too? It’s already very table and join friendly, and I could imagine a scenario where it was used inside SQL, pointed at a schema, some kind of proactive caching kept the data in SQL in synch with the data in the Vertipaq store, difficult BI calculations could be expressed in DAX, but the whole thing was transparent to TSQL. Imagine integrating that with Madison too!
Moving on, the other thing that has become clear to me is that I really have to sit down and learn Sharepoint (or at least the relevant bits of it) properly. It’s at the heart of Microsoft’s BI strategy and there’s no avoiding it. I have to admit to some mixed feelings about this move though, and I know other people I talked to at the conference share them. Partly it’s because, in the past, there were BI specialists and there were Sharepoint specialists and we didn’t necessarily have much to do with each other; now, though, the two worlds are colliding and I’m outside my comfort zone. You might say that Sharepoint has been part of the MS BI strategy for ages now, what with PerformancePoint etc, but I see an awful lot of MS BI customers in my work and I very rarely seem to see any Sharepoint, although it could be because I’m not looking out for it. A more valid objection is that the need for Sharepoint Enterprise Edition CALs adds a lot of extra cost to a project; and from a technical standpoint Sharepoint itself carries a very big overhead – its installation and maintenance may put a lot of customers off if they don’t already have a company-wide Sharepoint strategy, and if they do have one they may not be willing to go to 2010 for some time. Sharepoint might be just too big for some customers to swallow, and be a difficult sell for BI partners.
I’d like to stress though, once again, that I see the considerable technical benefits for using Sharepoint for BI, and even if the reception of the latest wave of PerformancePoint has been somewhat muted (eg the realisation that the decomposition tree has been tacked on at the last minute and isn’t properly integrated) I am impressed with what’s coming with Excel 2010 and Excel Services too; for example I think the Excel Services REST API is very cool indeed, and as a SSAS client Excel 2010 is a big improvement on 2007 (which wasn’t all that bad either). I’ve decided I also need to learn Excel properly now as well – get to know all those advanced Excel functions, use Solver and all that. Once again two worlds are colliding: the Excel guys and the SSAS guys are going to have to learn a lot more about each others’ technologies for truly effective BI applications to get built.
Anyway, I think this post has gone on quite long enough now. As always, your comments on everything I’ve written here would be much appreciated.
Good post, some valuable considered opinions in there. Very interesting, especially whether the introduction of PowerPivot is going to be a zero sum game for BI consultancy.Certainly it may introduce competition coming up from more desktop-focussed developers, but for projects requiring scale and consistency I would imagine that Excel vs SSAS will continue to be the preferred platform.I think that PowerPivot will enable a greater number of BI projects to get started ; whether they get finished will be a different issue ! Certainly v1 of Excel Services promised way more than could be delivered.ref Sharepoint, I don\’t think that Microsoft are very good at showing what you can achieve using just the standard WSS, ie license free, implementation that ships with Server 2003 and up – integrated Reporting Services works nicely for instance (once you have it set up !).
Chris-Commenting on the last part of your post on Sharepoint. I to have been thinking about my development and career path recently and I also believe that I need to start learning more Sharepoint. I can be a fly on the wall in a Sharepoint meeting, but I don\’t have the knowledge to really add anything yet. I have been picking up tidbits from our Sharepoint dev team here and there, but I feel the need to take it to the next level. With PPS being fully integrated into Sharepoint in 2010 and with PowerPivot coming, I can just see clients asking me questions related to Sharepoint that I don\’t think right now I am ready to answer. Luckily, MS has decided to release a version of Sharepoint 2010 that will install on a 64-bit version of Windows 7. This will make it much easier to play around with and learn the framework instead having to run a VPC.Looking forward to your blog on Excel 2010 ability to handle time utility dimensions!
Hi Chris, Excellent observations, but i think we as BI professionals can have even more work with PowerPivot:Here’s how i see the BI Pros work in the following years- Sell Enterprise BI with SharePoint and PowerPivot- Build the enterprise DWH with default reports, cubes & PerfPoint to support the primary operation (like sales by salesperson and product in a store) with Single Version of the Truth, use the same data throughout the organisation (users have their own view, board has KPI, floormanager same data only from his department)- Support the DWH or Cubes as source for PowerPivot-Enable power users to do their own Ad Hoc BI with PowerPivot, you cannot possibly solve all BI problems with traditional BI- Discover new business needs by monitoring the PowerPivot applications- incorporate this need in the traditional DWH with reports and deploy it enterprise wideThis is the ultimate “Agile/Scrum” datawarehouse making the iterations very short and costs low (the ROI) because you are sure this is what they want and know where to get the data.I think we can sell the step from self service to traditional BI (Single version and enterprise BI ) like this: traditional BI enables enterprise dashboards, KPI’s and reports (perf point services, SSRS on OLAP) even using role based security (on OLAP).More on this discussion on http://powerpivotpro.com/2009/10/29/powerpivots-impact-on-bi-pros/ By the way i\’ll be at the sqlserverday on 3 december where you will closing, maybe we can catchup there and talk some more about PowerPivot.Kasper de Jonge
Chris, I think you are spot on with your observations and concerns. I feel the exact same way. I think Powerpivot will be an interesting product to watch. I have a client that is using the product, and it still doesn\’t connect to SSAS, so he told msft that he couldn\’t fully evaluate it until he could make that connection. Msft told him "you don\’t need cubes anymore, just create one from your star schema". I know that everyone is saying that Powerpivot is not going to replace cubes or data warehouses, but I think the urge will be there to do just that.I wish msft would have spent more time on an analytical front end. But oh well, I gotta go learn Sharepoint…. :>)
I agree on most points. Yes, we will need to learn SharePoint 2010 and Excel 2010, and I\’m very curious about whether a an early VPC is going to be available.My Analysis Services OLAP / MDX work around San Diego has mostly been on decent-sized datasets (200 – 400 million fact rows). What is PowerPivot\’s row limit again …100m? Regardless, why do it, ironically, now see a need now to focus more on upstream DW/BI work (data modeling, ETL and data quality)? Here\’s why: If the business-side is now going to get direct access to databases (warts and all) via PowerPivot (vs. output prettied-up or hidden by cubes and QA\’d dashboards, we need to make these new data marts better than ever: better dimensional models populated with cleaner data.On Chris\’ comments about "…imagine …Vertipaq with Madison…" lot\’s of people are imagining exactly that. In fact, a fortune 500 company I know of is currently planning to buy a DW appliance, dump (w/ little or no transformation) their myriad of ever-expanding source data (internal OLTP, many third party feeds) on it on a highly ad-hoc basis, slap QlikView (PowerPivot competitor) on top, and let the business go wild with it. Although, all by itself, this sounds like a disaster, but they consider it as an agile "BI Project Pressure Reducer" for the most frantically urgent, intentionally less-formalized, analytic needs. They also plan to continue work on formal data modeling and traditional DW/BI in the background, especially on less short-fuse requirements. As Kasper said, this paradigm can stimulate formal OLAP dev later on, when standardization needs to be re-introduced on data with sustained value.Gentlemen, do you think PowerPivot installation teams will try to tackle datasets too large for a single PowerPivot database by, perhaps, building more, smaller, simpler PowerPivot databases? If so, then they won\’t be able to compete analytically with Analysis Services on larger datasets with sophisticated dimensional schema (eg. many large, tightly integrated fact tables. Analysis Services, as we know, kicks butt in this regard (many measure groups, cascading M2M relationships, dimensions that are large, referenced and role-playing, MDX utility dimensions, ROLAP partitions.Plug: Speaking of sophisticated schema that can still be consumed (with some finesse) by Analysis Services, check out my recent technical article draft of the "Marketing Intelligence Pipeline" schema. http://www.decisionlab.net/Downloads.htmlNow that PASS Summit is over, I hope to get feedback from Chris and others on it. Thanks! In closing, I think we are still the \’Midas Muffler\’ guys. "Pay me now, Ma\’am, or pay me later".
A few points:Yes, in the last 18 months were mostly focused on Self-Services BI. But now that PowerPivot is about to be shipping the MS BI team will be back to focusing on the Corporate BI market. So no worry there. We have some big plans for all you BI pros.The PowerPivot push generated for us a bunch of incredible IP: VertiPaq, DAX, a new modeling paradigm, Excel and SharePoint integration, the Report Gallery…. All of these are going to be used for the Corporate BI scenario and will allow you to build some truly incredible large scale sophisticated corporate apps.BTW – VertiPaq is much more scalable than any of you can imagine. But I am already saying too much.The commonality between the underpinning of the Self-Service PowerPivot and the future Corporate BI should be viewed as a major boon. The work starting at the Business Unit level using PowerPivot should feed demand for new BI sources and will frequently evolve into full fledged BI apps. PowerPivot is a perfect organic demand generator for you guys. It is also a great selling tool – Include PowerPivot as part of you value proposition, learn how to do the demos well, and you’ll win new business left and right. Business users just fall over themselves to get the technology. Three more points:1. Learn SharePoint.2. Learn SharePoint.3. Learn SharePoint.I cannot emphasis enough how central SharePoint is to Microsoft’s BI roadmap and the unbelievable momentum this product has in the market. The 2009 SharePoint conference was larger than TechEd and PASS combined. And it sold out. In our BI breakout session in the SharePoint conference we had about as many people as we get in the PASS keynote. So learn SharePoint, or risk eating some dust.That’s it for now.
Amir: I\’m glad to hear you haven\’t forgotten about us corporate BI types; just don\’t get too distracted with those power users, and please let\’s make sure that PowerPivot isn\’t marketed as a substitute for SSAS in all cases. My doubts won\’t go away until I\’ve seen how Microsoft\’s strategy develops over the next few years…
Amir: I hope that you are not saying that we MS BI geeks should be experts in workflows, document lifecycle management, search and content management beside being experts in SQL Server, SSAS, SSAS data mining, SSRS, SSIS, Excel 2010 and Gemini? Sharepoint 2010 has 22 different applications, if I am not wrong. I think that Sharepoint is about 70 percent features that you do not need for MS BI low hanging fruits, unless you are are 100 percent MS shop.
Thomas – I am saying that you need to add SharePoint to your arsenal of well understood technologies. You don\’t need to know all aspects of SharePoint, but you do need to know enough to carry an intelligent convesation, understand the architecture and be able to configure and troubleshoot the BI technologies in SharePoint.SharePoint is a critical part of the BI stack and as a BI pros, I really recommend for you guys to study it. Believe me – there is a lot of business to be found in SharePoint BI.
OK Amir. I hope that Sharepoint 2010 will have good and clear wizards for setting up the BI-part of that platform. I will study it.