Monitor Real-time Data with Power BI Dashboards

2016-08-11_21h46_05

I’ve hear this a lot in my presentations that “Does Power BI support real-time data?”, And I answer yes. In fact this feature was available long time ago with Azure Stream Analytics and Power BI, however recently some new changes happened in this area which allows Power BI dashboards to monitor real-time data changes on the dashboard from a data coming from API, and also PubNub. In this post I’ll show you how easy is to create a dashboard that shows real-time data coming from API. If you want to learn more about Power BI, read Power BI online book; from Rookie to Rock Star.

Real-Time Data, Instant Monitoring

One of the key features of monitoring tools is the ability to monitor the real-time data instantly when the data refreshes. That means as a user you don’t need to click on refresh button to see the latest data. This single key feature brings heaps of value to Power BI as a monitoring tool. This ability isn’t new, in fact Amir NetZ showed an awesome demo of an IoT device working with Azure Stream Analytics and Power BI as a real-time monitoring. So what is new is that we can now have this feature in the new generation of Power BI with a dashboard tile connected to streaming data. Streaming data sets can be now either API, Azure Stream Analytics (coming soon), and PubNub. More streaming services will come very soon I believe.

In this post I want to show you only streaming data through API, and showing that in a dashboard tile in Power BI. This single streaming option by itself opens many gates to Power BI as a real-time data monitoring tool. So let’s start.

Prerequisite

For this example I will be using Microsoft sample .NET application named PBIRealTimeStreaming. You can download it from here.

You need Visual Studio (Not SSDT) for running this example solution.

Real-time Data Set

Start with logging in to Power BI service, create a new dashboard (or open an existing dashboard). And create a New Tile.

2016-08-11_21h01_08

In the New Tile pane you now can see an option to add a tile for real-time data.

2016-08-11_21h02_21

Select this option and click Next. if this is the first time you are using this service, then you probably don’t have any streaming data set existing there. So go to Manage Data to create one;

2016-08-11_21h03_42

Now you are in streaming data sets window. you can see list of existing data sets, or create a new one with click on Add Streaming Dataset option on top right hand side.

2016-08-11_21h05_34

Create Sample Streaming Data Set

As I’ve mentioned before at the time of writing this post only these three options for streaming data is available: API, Azure Stream Analytics, and PubNub (which is a data streaming service).

2016-08-11_21h07_49

Because for this example we are using API as a streaming option, choose API and continue. Now create a new data set with a name such as mySampleRTDS or any other names you like. You have to create two columns as below;

  • ts column with DateTime data type
  • value column with Number data type

These are columns that will be created and passed through a .NET application through Power BI API to this data set.

2016-08-11_21h12_09

Now click on create. This will create the data set with a Push URL that can be used in the application through API. copy this URL. We will be using that in the application next step.

2016-08-11_21h15_28

Sample Streaming Data API Application

As I mentioned before, download the sample project. and then select the project under this path:

samples/consoleapp/getting-started-for-dotnet-real-time/PBIRealTimeStreaming

And open the PBIRealTimeStreaming solution in Visual Studio. This project creates a data set including a datetime value and a numeric value and will pass that through Power BI API (which will be discussed later in another post) to Power BI service.

Open Program.cs file and in line 33 change the value of realTimePushURL to the URL which we’ve copied from above step.

2016-08-11_21h20_50

Run the application with pressing F5, or right click on project, Debug, Start New instance. The application will generate a data point in each second and send it through API.

2016-08-11_21h42_18

Create Dashboard Tile

Now, let’s create the dashboard tile for this streaming data set. Follow the steps and choose the mySampleRTDS as the data streaming source for the tile.

There are number of charts/visualizations that you can use with streaming data set. Choose Card for the first example. and set the value of that to “value” data field

2016-08-11_21h47_41

Add another dashboard tile for the same data set, and set this to be a Line chart with ts field in Axis, and value field in Values.

2016-08-11_21h49_10

After adding each chart you will immediately see the real-time dashboard working and changes every second (because streaming data from application updates every second);

2016-08-11_21h46_05

API Data Can Be Anything

The sample used here had only one datetime field and a value field. However you can create your own data stream from any event, notifications, or data set that you want to be monitored real time. With a simple application the stream of data can be generated. Or it can come from Azure Stream Analytics or PubNub. Don’t limit your options, think about where you can use this feature to solve a real-time monitoring challenge.

 

Save

Reza Rad on FacebookReza Rad on LinkedinReza Rad on TwitterReza Rad on Youtube
Reza Rad
Trainer, Consultant, Mentor
Reza Rad is a Microsoft Regional Director, an Author, Trainer, Speaker and Consultant. He has a BSc in Computer engineering; he has more than 20 years’ experience in data analysis, BI, databases, programming, and development mostly on Microsoft technologies. He is a Microsoft Data Platform MVP for 12 continuous years (from 2011 till now) for his dedication in Microsoft BI. Reza is an active blogger and co-founder of RADACAD. Reza is also co-founder and co-organizer of Difinity conference in New Zealand, Power BI Summit, and Data Insight Summit.
Reza is author of more than 14 books on Microsoft Business Intelligence, most of these books are published under Power BI category. Among these are books such as Power BI DAX Simplified, Pro Power BI Architecture, Power BI from Rookie to Rock Star, Power Query books series, Row-Level Security in Power BI and etc.
He is an International Speaker in Microsoft Ignite, Microsoft Business Applications Summit, Data Insight Summit, PASS Summit, SQL Saturday and SQL user groups. And He is a Microsoft Certified Trainer.
Reza’s passion is to help you find the best data solution, he is Data enthusiast.
His articles on different aspects of technologies, especially on MS BI, can be found on his blog: https://radacad.com/blog.

18 thoughts on “Monitor Real-time Data with Power BI Dashboards

    • Hi Brad,

      this works with push mechanism, SQL DB or Odata doesn’t support push by itself. So you need an application to read data from those and push it. which can be similar to the .NET application used here and pass the data to REST API.

      With SQL DB you can have DirectQuery which is different story from pushing real-time data.
      Cheers,
      Reza

  • I find your blogs excellent Reza, keep them coming.

    I saw this one a while ago and have been waiting for the mysterious “Azure Stream Analytics” streaming dataset option to pop up on my PowerBI desktop. Most people are seeing API and Pubnub only. Any idea when the Streaming Analytics option will be available – how do you see it and I don’t?

    It is tempting to migrate my Streaming Analytics source back to Pubnub just to get this feature.

    • Thanks Leigh for your kind words and feedback. Glad you liked it.
      For stream Analytics the process is different. Instead of creating Data set from Power BI and connect it to Stream Analaytics (like what you do for REST API and PubNub), you have to start process from Stream Analytics itself. there in Stream Analytics you can define the output to be Power BI Data set (you can define name of the data set, the work group of it, and name of table etc). Here is a blog post with a full example of Stream Analytics and Power BI I have written:
      https://radacad.com/stream-analytics-and-power-bi-join-forces-to-real-time-dashboard

      Cheers
      Reza

  • Which version of Visual Studio was this developed on? I’m using VS 2010 and when I try to open the SOLN or PROJ file it says:
    This project is incompatible with the current version of Visual Studio.

  • Hi

    Tried your sample but can’t make it work. I see that the timestamp ha a different format from the application compared to your screen dumps in the article.

    I get :
    “[{ \”ts\”: \”13.05.2017 06.14.57\”, \”value\”:39 }]”

    An the tile do not show anything with this format.

    Any ideas what is going on ?

    Regards

    • Hi Rolf,
      You can try and use Format String to something with YYYY MM DD HH mm SS. maybe this is because of your computer locale configuration.
      Cheers
      Reza

  • HI I am unable to see azure stream analytics data set option when we are creating a new data set how to to get that one to my account. why I am unable to find that one any idea on it ?

    • You won’t see this in Power BI.
      You need to set this up from Azure portal. Create azure stream analytics. and then an output for Power BI

      Cheers
      Reza

    • Hi,
      Streaming data: helps you to build a dashboard which changes any time a new data entry is available.
      Live query: means your dashboard data will be always up-to-date from the data source. but live query is not aware of changes in the data source. so dashboard refreshes every 15 minutes to give you up-to-date information.
      Cheers
      Reza

  • Hi i want to be create one dashboard to show all server DB size,DB growth for weekly and server space etc.how can i create like this kind of dashboard.please suggest.
    Regards,
    Saran

    • Hi Sara,
      Do you want such dashboard to be real-time?
      If so, then you need to create an application that calculate these information and send it through REST API to Power BI.
      otherwise if it is not necessary to be real-time. then create the report in Power BI Desktop with scheduled refresh.
      Cheers
      Reza

  • Hi there. I am trying to link Power BI to Project Online custom fields. Are you able to please help me do this? Cheers

  • Hi,

    I’d like to ask one thing. Is it possible to use Python? Can we write such an application in Python and push the data.

    Regards,
    Kamal

    • Hi Kamal
      I think (not sure) you should be able to do it from there too. for sending real-time data, you need a REST API URL, authentication configuration, and a mechanism to send JSON scripted data. if these are possible to do from Python, then you are good to go.
      Cheers
      Reza

Leave a Reply