Integrate Power BI Into Your Application: Part 6 – Real-time Streaming and Push Data

Facebooktwittergoogle_plusredditpinterestlinkedintumblrmailFacebooktwittergoogle_plusredditpinterestlinkedintumblrmail

2017-11-06_11h51_57

It’s been a while since the last post about Power BI REST API. I have written previously about how to get the REST API, and use it in your application, how the authentication works, and things such as embedding content, and data source management. In this post, I’m going to explain one of the functionalities of REST API which is pushing data to Streaming data set. I’ve previously explained how to do real-time streaming using Azure Streaming Analytics and Power BI. However, this post explains real-time functionality just with the REST API. You won’t need azure services besides it. If you want to learn more about Power BI, read Power BI book, from Rookie to Rock Star. If you want to learn more about REST API, follow this series:

Part 1: Register your Application
Part 2: Authenticate
Part 3: Embed Content
Part 4: Refresh Data Set
Part 5: Data Source and Gateway Management

Prerequisite

For running the sample in this post, you need to have an application that is registered and followed the authentication process. If you want to see how it works, here are the two steps you have to have prepared as the prerequisite:

Step 1: Register your Application
Step 2: Authenticate

You also need a Power BI account to create a data set in the service.

Streaming Dataset

In Power BI, you can create different types of dataset. The normal data set that everyone knows about it is a data set that automatically generates when you create a Power BI Desktop solution and publish it to the website. However, There is another very important type of data set, called Streaming data set. This dataset is used for pushing data into Power BI in a real-time scenario.

How to create a streaming data set

Go to Power BI Service (http://powerbi.microsoft.com) website. Log in to the website with your Power BI Account. When logged in create a new streaming dataset;

2017-11-06_11h44_03

There are three ways to create a streaming dataset. I have explained the Azure Stream Analytics previously. For this example, choose API;

2017-11-06_11h49_45

In the next part, is where you specify the name of your dataset, and add fields to it. Your fields can be of type DateTime, Text, or Number. with adding every new field, you will see a JSON format will be created underneath, which is the format that the data needs to be sent through REST API.

2017-11-06_11h51_57

 

Push, Hybrid or Streaming?

One very important option to choose is the Historic Data Analysis. By default, this option is off. It means the streaming data will not be stored anywhere. Service keeps up to 200,000 rows of data. when row # 200,001 comes in, the first row will be deleted. If you want the history to be saved, you have to switch this option to On. When you turn this option your dataset from a streaming dataset, changes to be a Hybrid dataset.

2017-11-06_12h11_42

Streaming dataset

Only streams the data rows. Keeps up to 200,000 rows. First In, First Out. When row #200,001 comes in, row #1 will be removed. Cannot be used easily in reports, because there is no historical data. You can create a report with that, however, because it only focuses on streaming data. not all options are available.

Hybrid dataset

Streams the data, and also keep the data rows. You will have the rich experience of Power BI Report. Because data is stored, you can interact with that easier from a report.

Push dataset

Created when you create a solution from Power BI Desktop with importing data into it. This dataset is not a streaming dataset. it just stores the data. the data should be then scheduled to refresh. we are not talking about this type of dataset in this post.

In this example, doesn’t matter which option to choose.

Create a Dashboard and Report

After creating the dataset, you need to create a report with a simple line chart with value and timestamp on it, and then save it, and pin it to a dashboard. The dashboard’s chart at the moment will be blank because there are no data entries in it for now. In the next step, we will be adding rows into this data set from an application.

Connecting to the dataset from REST API

As mentioned before, you need to prepare your application for this step as a prerequisite. In the application ready for this (after registration, and authentication), you can add few lines of code to pass this data through REST API. Here is a sample code;

Majority of the code above is about finding the dataset (which I explained it in details in data source management and refreshing dataset posts). After finding the dataset, I used a loop (highlighted lines) to create a random number from 0 to 100, and pass it to the service and dataset with a function named PostRowsInGroupAsync. This process is in a loop and happens every second for 5 minutes. as a result, now you can see the dashboard data changes;

2017-11-06_12h28_32

Summary

In this post, you’ve learned how easy is to create a streaming dataset and interact with that with REST API. You’ve learned that there are two types of datasets that you can work with for streaming; Hybrid and Stream only. One is storing data as well as streaming it. the other one is just streaming. If you are interested to learn more about REST API, read blog series below;

Part 1: Register your Application
Part 2: Authenticate
Part 3: Embed Content
Part 4: Refresh Data Set
Part 5: Data Source and Gateway Management
Facebooktwittergoogle_plusredditpinterestlinkedintumblrmailFacebooktwittergoogle_plusredditpinterestlinkedintumblrmail
rssyoutuberssyoutube
Reza Rad

Reza Rad is an Author, Trainer, Speaker and DW/BI Consultant. He has a BSc in Computer engineering; he has more than 15 years’ experience in databases, programming and development mostly on Microsoft technologies. He is a Microsoft MVP in Data Platform for seven continues years (from 2011 till now) for his dedication in Microsoft BI. He is author of some SQL Server and BI books, and also Power BI online book; from Rookie to Rock Star.


Leave a Reply

Your email address will not be published. Required fields are marked *