Category: Operations Agent

Using Fabric Operations Agents And Workspace Monitoring With Power BI

This week, in the announcement about support for Fabric Pipelines in Workspace Monitoring, I noticed that it came with an Operations Agent that actively monitors and analyses Pipeline activity. And that got me thinking, since Workspace Monitoring also contains Power BI activity data, why not create an Operations Agent to actively monitor Power BI too?

I decided to test a really simple scenario. I created a semantic model that contained several measures, one of which returned an error. I then created a report from that semantic model with two pages: one with several working visuals and one with a visual that used the broken measure and which therefore returned an error.

I published this report to a workspace with Workspace Monitoring enabled.

I then created an Operations Agent in the same workspace and connected it to the KQL Database associated with Workspace Monitoring:

Here are my Agent Instructions:

*** Operational Instructions ***
1. Alert me when a DAX query on any of the Power BI semantic models in this workspace returns an error, for example when the DAX in a measure, except when the error is the result of a query being cancelled
2. When you alert me, give me the following information:
a) The username of the user that generated the error
b) The date and time of the error
c) The error message
d) The IDs of the semantic model, the report and the visual that generated the error
e) The OperationID of the query
*** Semantic Instructions ***
1. The SemanticModelLogs table contains information about Power BI activity
2. Every time there is an error, there will be an event with the OperationName "Error"
3. The EventText column for these Error events contains the error message
4. Errors generated by query cancellations, which can be ignored, have an error message in the EventText column that starts with the text "The operation was cancelled by the user"
5. The Timestamp column contains the date and time of the events
6. The ExecutingUser column contains the username of the user that generated the error
7. The ApplicationContext column contains a JSON fragment which gives the IDs of the semantic model (DatasetId), report (ReportId inside the Sources array) and visual (VisualId inside the Sources array)
8. The OperationID column contains the OperationID

Here’s the Playbook that the Operations Agent generated:

And here’s the KQL generated for the DAX Query Error Event Alert:

// DaxQueryErrorEvent: one row per non-cancelled DAX query error event from SemanticModelLogs
declare query_parameters(startTime:datetime, endTime:datetime);
let DaxQueryErrorFacts =
  ['SemanticModelLogs']
  // Restrict to DAX query error events and exclude user-cancelled operations
  | where ['Timestamp'] between (startTime .. endTime) and ['OperationName'] == "Error" and not(tostring(['EventText']) startswith 'The operation was cancelled by the user')
  // Parse ApplicationContext once for downstream extraction
  // Parse ApplicationContext once for downstream extraction
  | extend ApplicationContextParsed = todynamic(['ApplicationContext'])
  // Extract DatasetId directly from ApplicationContext
  | extend DatasetId = tostring(ApplicationContextParsed.DatasetId)
  // Extract first report/visual source, if present, from ApplicationContext.Sources[]
  | extend Sources = todynamic(ApplicationContextParsed.Sources)
  | extend FirstSource = iif(isnull(Sources) or array_length(Sources) == 0, dynamic(null), Sources[0])
  | extend ReportId = tostring(FirstSource.ReportId)
  | extend VisualId = tostring(FirstSource.VisualId);
let DaxQueryErrorEvent =
  DaxQueryErrorFacts
  | project
      // Entity identity and timestamp
      Id = ['OperationId'],
      Timestamp = ['Timestamp'],
      // Value requests
      DaxQueryErrorIsNonCancelledError = true, // by construction: filtered to non-cancelled errors
      ExecutingUser = ['ExecutingUser'],
      ErrorTimestamp = ['Timestamp'],
      ErrorMessage = ['EventText'],
      DatasetId = DatasetId,
      ReportId = ReportId,
      VisualId = VisualId,
      OperationId = ['OperationId'];
DaxQueryErrorEvent

I have to admit it took several iterations to get the Agent Instructions right so that working KQL was generated, and I would not have been successful without testing the KQL myself and understanding why it wasn’t working. It was not particularly hard to do this though.

I then saved and started the Operations Agent.

Next I opened the report that contained the broken visual, so that I saw an error in the report, and waited. After a few minutes I got the following message in Teams:

So I got a Teams notification when a report containing a broken visual was rendered, which is great and proves the point that Operations Agents can be used to monitor Power BI when used with Workspace Monitoring. You’ll notice that the Teams message doesn’t include the error message, the semantic model, report or visual IDs or any of the other information I requested (although the KQL query generated by the Playbook does return that information) but I guess I need to spend more time tuning the Agent Instructions. Other things I can imagine doing with an Operations Agent include monitoring for failed semantic model refreshes and slow DAX queries. Definitely something I need to spend more time investigating.