Power BI · Premium · Processing · Refresh

Troubleshooting Power BI Timeouts, Part 1: The Two/Five Hour Import Refresh Limit

If you’re working with a large Power BI dataset and/or a slow data source in Import mode it can be very frustrating to run into timeout errors after you have already waited a long time for a refresh to finish. There are a number of different types of timeout that you might run into, and in this series I’ll look at a few of them and discuss some of the ways you can work around them.

In this post I’ll look at one of the most commonly-encountered timeouts: the limit on the maximum length of time an Import mode dataset refresh can take. As documented here these limits are:

  • Two hours for an import-mode dataset in Shared capacity
  • Five hours for an import-mode dataset in a Premium or PPU capacity

If you are using Premium you can connect to your dataset using SQL Server Profiler and run a trace when the refresh starts and you’ll see the timeout set as a property of the TMSL command that runs the refresh (18000 seconds = 5 hours):

Here’s an example of the message you will see in the Refresh History dialog if your dataset refresh takes more than five hours in PPU:

Once again you get more details in a Profiler trace. When the refresh times out you’ll see an Error event with long message that starts with the following text:

Timeout expired. The timeout period elapsed prior to completion of the operation.. The exception was raised by the IDbCommand interface. The command has been canceled..

There is a way to work around this limit if you’re using Premium or PPU: it only applies to refreshes (scheduled or manual) that you start from the Power BI portal or via the REST API. If you start your refresh by running a TMSL script via the XMLA Endpoint (for example via SQL Server Management Studio or from PowerShell) then the limit does not apply. This is because you’re executing your own TMSL script rather than having the Power BI Service execute its own refresh command – with a timeout set – like the one shown in the screenshot above.

For example, here’s the same dataset from the screenshot above but refreshed successfully from SQL Server Management Studio (notice the Type column says “Via XMLA Endpoint”) and with a duration of just over five hours:

There are a couple of blog posts out there showing how you can implement a refresh strategy using the XMLA Endpoint; this post from Marc Lelijveld and Paulien van Eijk is a great example of how to do this using Azure Automation and Azure Data Factory.

You should also consider tuning any refresh that takes a long time and it could be that after tuning it you fall well within the two/five hour limit. There are a lot of things to consider when tuning dataset refresh; I did a conference session on this topic last year (you can watch the recording here) covering issues like data modelling, query folding in Power Query and the use of dataflows but it doesn’t cover everything and I have learned a lot even since then. In my opinion one of the most important things you can do to improve refresh performance for very large Import datasets is to partition your fact tables and increase the amount of parallelism, as I describe here. One customer I worked with was able to reduce their refresh time from four hours down to one hour using this technique:

As I said, though, there are many different types of timeout that you may encounter – so even if you refresh via the XMLA Endpoint it may still time out for another reason.

13 thoughts on “Troubleshooting Power BI Timeouts, Part 1: The Two/Five Hour Import Refresh Limit

  1. Triparna Ray – Passionate Business Intelligence Expert offering vast experience  leveraging software engineering and agile methodologies to deliver  highly effective and creative solutions to business and technology  challenges. Utilizes highly attuned analytical skills to develop BI solutions  employing cutting-edge technologies to increase productivity.  Consistently drives high standards of service through effective project  management, communication, and strategic planning to develop and  manage strong client relationships. Highly organized with strong capacity  to prioritize workload, delegate deliverables, and steer project  completion within established deadlines. 
    Triparna Ray says:

    Can’t imagine refreshing datasets spending 4-5 hrs per day? Are we not using incremental refresh. Or is it specific use case where we can’t use that?

    1. Chris Webb – My name is Chris Webb, and I work on the Fabric CAT team at Microsoft. I blog about Power BI, Power Query, SQL Server Analysis Services, Azure Analysis Services and Excel.
      Chris Webb says:

      There are some people with *very* large datasets out there…

  2. I just got this just now. Trying to use incremental refresh on Pro for large data for the initial load. Will have to address the model size, etc to get the speed down. Thanks for the info.

  3. There is also an oauth2 tocken timeout problem after an hour. This seems to impact pq running on gateway, but not on pbi desktop. This appears to be a well-known bug that Msft won’t fix. I opened a support case and Msft seems pretty apathetic about fixing this one. I’d guess that customers are impacted regularly.

    1. Hello,
      I’m finding exactly this timeout I guess. I’m trying to connect pbi to a azure gen2 datalake using a gateway (in an azure VM). Everything seems to work correctly after 1 hour when it triggers an authentication error.
      For me it seems that ‘SAS’ authentication doesn’t have this issue but I’m still investigating.

Leave a ReplyCancel reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.