I’m a big fan of Kurt Buhler‘s work (indeed we like him so much at Microsoft that we get him to help out with our docs) and I have a lot of respect for his opinion. A few weeks ago, in a thread on Twitter/X, he asked an important question about Power BI Copilot: why would someone use it? When I saw this I realised it was something I hadn’t thought about properly but which deserved a detailed answer. Now, after ruminating for a while and talking to a few people (including Kurt), I will put my case for why someone might want to use Power BI Copilot.
[Just to be clear: in this post I am not going to discuss the related question of whether Copilot works well and delivers useful results – even though I think it is pretty good already – because that’s not what Kurt is asking in the thead]
Which Copilot are we talking about here?
Before I carry I need to define which Copilot is being referred to in the question. There are a lot of Copilot experiences in Power BI: the one in DAX Query View, the one that can create new report pages, the one that allows you to summarise and ask questions about data that is visible on report pages, the one that allows you to ask questions about data that is in your semantic model but not necessarily visible in a report, as well as others. Since Kurt and I agree on the usefulness of Copilot in DAX Query View for intermediate developers let’s focus on the last three – which can be summed up as the ability to answer data questions in natural language – which are what most people think about when they think about Copilot for Power BI anyway.
A solution looking for a problem?
To answer the question of why someone would use Power BI Copilot to answer data questions we need to look at who is asking for this functionality. But is anyone actually asking for it? I’ve heard a few people say that Copilot for Power BI is a solution looking for a problem.
Now I can see why you would think that because let’s face it, we can all imagine some VP in Microsoft ordering every program manager to add a Copilot to their product whether they think it’s useful or not because that’s what the market demands. However if you work in the Microsoft field or for a partner then I’m sure you’ll know how much genuine interest there is in Copilot from Power BI customers and users. In my opinion Power BI Copilot is actually the latest attempt to respond to a very real request that I have heard for the whole of my career in BI and which can be summed up as follows:
I want to ask questions in English (or whatever my native language is) about my data rather than have to look at a report someone else has built or build one myself
I’m old enough to remember SQL Server 2000’s English Query feature, which was an attempt to solve the same problem over twenty years ago, and there have been numerous other products and features from Microsoft (including Power BI’s own Q&A) and other BI vendors that have tried to do the same thing in the past. I am not saying that any of them succeeded in their aim because I don’t think any of them did. What I am saying is that they were attempts to do the same thing that Power BI Copilot is trying to do and that therefore Power BI Copilot is not a solution looking for a problem, it’s actually the latest attempt to solve a very real and very old problem.
So what is the problem?
I have never felt the need to ask questions about my data in natural language because I have always found it easy to:
- Look at an existing report or dashboard and interpret what’s in it
- Build my own report/dashboard/queries to get answer to questions that are not in existing reports or dashboards
That’s going to be true of anyone working as a data professional or a business analyst. So if the question is “why would I use it?” then the answer is I wouldn’t.
However I believe the people who are asking for natural language querying of data are people who:
- Find it hard or impossible to interpret data provided in numeric or visual form in a report
- Find it hard or impossible to build new reports or write queries to get answers that aren’t in the reports they do have access to
- Have to ask other people to get the answers they need
It’s tempting to say “read what’s in the report” or “learn Power BI” or “learn SQL” when faced with these people and their requests. But, clearly, they keep asking for an alternative and we’re not going to change their minds.
Think back to maths class at school and it shouldn’t be a surprise that some people just don’t enjoy staring at pages of numbers and graphs or learning database query languages in the same way your or I do. What’s more dyscalculia or “maths dyslexia” means that interpreting data – however well designed your visualisations are – doesn’t come easily to a significant number of people. While the imperative to be “data driven” is widespread in every industry and profession today, not being good with data or not enjoying using it is not a barrier to success in most careers. Power BI Copilot provides a different, and perhaps more approachable, way of getting answers to data questions for many people.
It seems to me that a lot of the excitement for Copilot for Power BI comes from C-level people who love the challenge of management, who need data to make decisions, but who don’t like working with data directly or don’t have the time to do so. Today they need to ask their data questions to a business analyst and this introduces latency: asking someone else to do something is always more time consuming than doing it yourself. What I suspect these C-level folks want is to cut the human out of the loop, get answers to their questions more quickly, while keeping the experience of interacting with a human. Power BI Copilot aims to be that virtual business analyst.
Why is Copilot better or more effective than non-AI approaches?
If the “non-AI” approaches mentioned here involve “reading the report” or “learning Power BI” I believe I’ve already dealt with them above. What about non-AI approaches to natural language querying? Clearly LLMs are vastly better at “understanding” natural language requests than any technology ever seen before but there’s more to say here.
As I already mentioned, while the problem of asking data questions in natural language has been around for a long time, no-one has solved it yet. Why? Thinking back to my experiences with Power BI Q&A the reason is that to get good answers you have to ask good questions. Half the challenge of learning programming or any database query language is understanding what you want to do before you even learn how to express that request in code. To get good results from Q&A you have to construct the English-language equivalent of a DAX or SQL query and that’s why it, like all its predecessors, failed. Users are bad at constructing clear, unambiguous questions.
One way that Copilot is an improvement on Q&A is that it can interpret a wider range of questions. For example, I have a demo with a table of books at their year of publication; I was pleasantly surprised that when I asked “Show me all 19th century literature” it gave me all the books published between 1800 and 1899. Q&A would never have been able to do this.
More importantly, Copilot is getting better at asking for clarifications. Going back to the previous question, when a C-level person asks their business analyst for the answer to a data question it will be something vague like “what was growth like for widgets in the UK last year?”. Part of the job of the business analyst is to know exactly what exact product “widgets” refers to and which particular geographies make up the UK without having to ask. Those are things that can be defined in a good Power BI semantic model. However part of the job of the business analyst is to ask follow-up questions like “do you mean month-on-month growth or year-on-year growth?” to clarify the original request and that’s exactly what you’ll see Copilot doing more of in the future.
The ability of LLMs to conduct conversations in near human-quality natural language and use this ability to turn a vague request into a data question that can be answered is, I think, why AI approaches to the problem we’re talking about are better than non-AI approaches.
Conclusion
I hope I have made a good argument for there being people who want to use Power BI Copilot and the reason why is that they can’t, don’t like or don’t want to learn to use existing methods of interacting with data. Will it succeed where so many others have failed? Let’s see where we are in a year or so’s time but I’m confident that the remarkable advances in generative AI mean that it will.
In my experience so far, Copilot is only marginally better than Q&A. I expect it to continue to improve, but it’s not worth bothering with yet unless you’ve devoted considerable energy into curating the linguistic model.
Q: Why would you use Copilot for anything except helping with DAX?
A: You wouldn’t, not yet anyway. Wait until it’s progressed beyond vaporware.
I concur with your thoughts. with advancements in genai models and power bi moving towards it (gpt4 to gpt4o) plus letting users to train how the copilot should answer from their semantic model (like that in AI skills), soon copilot in pbi will do a much much better job and the c-suite who wants answers in chatty format will love this over figuring our from the reports themselves.
Aside from the F64 capacity requirements, assuming you can justify this, the biggest problem for real world applications (if my understanding is correct) seems to be:
“Semantic models with implicit measures disabled: In scenarios where implicit measures are disabled in your semantic model, Copilot cannot create report pages.”
1) Real world models usually require explicit measures to handle complex business logic.
2) Calculation Groups (often required for time intelligence filters and other reasons) discourage/disable implicit measures.
3) XMLA/Excel pivot tables connected to PBI require explicit measures.
Complex real world scenarios essentially demand explicit measures but Copilot requires implicit measures…so forget about end users using Copilot to generate report pages (which could have been a strong use case).
Am I missing something?
wasn’t the 19th century from 1801 to 1900 not 1800 to 1899, or do literature people use a different definition?
Haha, no that’s my own stupid mistake!
As long as copilot use is classified as a backround operation for resource usage calculation, it’s simply to dangerous to enable it to a broader audience.
Why? Surely making it an interactive operation would be worse?
classifying it as an interactive operation at worst will cause only a temporary (in minutes) throttling of the service, heavy usage of copilot when classified as a background operation can potentially cause multi hour service unavailability.
That’s not correct. The nature of the operation has no effect on how throttling works. What is true is that background usage, because it is smoothed over 24 hours, has a way of building up slowly over time which makes burndown slower too.
For “summarise and ask questions about data that is visible on report pages”, can a user ask Copilot to fetch all text data in a particular column in a table visual (which is filtered by user) to summarise it?
Not exactly, you can summarise text data with some limitations. I want to blog about this but I need the ok from the relevant PM.