Refresh Power BI Queries Through Power Platform Dataflows: Unlimited Times with Any Frequency

FacebooktwitterredditpinterestlinkedintumblrmailFacebooktwitterredditpinterestlinkedintumblrmail
FacebooktwitterlinkedinrssyoutubeFacebooktwitterlinkedinrssyoutube

If you don’t have a Power BI Premium capacity license, then you are limited to refresh your dataflows up to eight times a day, with the frequency of 30 minutes. The good news for you is that you have a way to do unlimited refreshes and with whatever frequency you like using Power Platform Dataflows. Read the rest of this blog article to learn how.

What is Dataflow?

In the world of Power BI, You can separate the data transformation layer using Dataflows. Dataflows can run one or multiple Power Query scripts on the cloud and store the result into Azure Data Lake. The result of the dataflow (which is stored in Azure Data Lake) can then be used in a Power BI dataset using Get Data from Power BI dataflows. If you want to learn more about Dataflows, I highly recommend reading my dataflows series that starts with this article.

Creating Power Platform Dataflows

Dataflow is not only a concept for Power BI, but It is also available as Power Platform dataflow. You can check out one of the announcements of features added in Power Platform Dataflows by Miguel Llopis (one of the product managers of the Data Integration team at Microsoft), here. You can create Power Platform Dataflows, very similar to the way that you create Power BI dataflows. The only difference at the beginning is where you create it. Power BI dataflows are created in Power BI service, Power Platform Dataflows can be created in Power Apps (or Power Platform) portal.

Login to Power Platform Portal here.

Then go to Dataflows under the Data section,

Start a New dataflow

just provide a name for the dataflow, and then create

At this step, you can either go ahead and get data from anywhere you want or use Blank Query and copy a Power Query script from Power BI Desktop into here.

after preparing your queries, you can go to the step of loading it.

Load to Entity

The load to entity step in Power Platform Dataflows is a bit different from Power BI Dataflows. In Power BI Dataflows, the result will be loaded into CSV files into Azure Data Lake, so no more configuration is needed. However, Power Platform Dataflows stores the data into CDS (Common Data Services), and you have to select the entity you want to load this data into it.

You can either choose one of the existing entities or create a new entity, then map fields. Because these are CDS entities, there are some rules for having key fields, name fields, etc. If you don’t know what the CDS is, and where it stores the data, read my article about CDS here.

Refresh Settings

After setting up the load to entity and mapping fields correctly, you will get into the Refresh settings. This is where you can refresh your dataflow even with the frequency of a minute. You can refresh it as many times as you want. You are not limited to eight times or even 48 times a day.

Refresh as much as you want

Here you can see the frequency of the test dataflow that I set up, which ran every minute.

What about Licensing?

I know what you are going to ask now! What about licensing? What type of license do you need for this? Well, for Power Platform dataflows, you don’t need a Power BI license at all, not Premium, not even Pro. You do need, however, to have a Power Apps license. at the time of writing this blog post, there are two options, $10 a month, and $40 a month. Both of these can be cheaper than Power BI Premium if you are a single user (or even a few users). The main difference between the two plans of Power Apps, as I see is the database and file size for CDS;

How to use the results in Power BI?

You can use the result of dataflows in Power BI (similar to the way that you can use the result of Power BI dataflows in Power BI). You need to Get Data from Power Platform Dataflows.

However, at the time of writing this blog post, this feature is still beta and under development, and might not show the Power Platform dataflows.

Another option is also to get data from the Common Data Service. (Because Power Platform Dataflows stores the data into CDS)

Note that Power BI Dataset Refresh Limites Still Applies

Having a Power Query script that can refresh even every minute is great. However, you have to note that the dataflow refresh is different from the dataset refresh. If you use the result of the dataflow in a Power BI dataset, Still you need to get your dataset refreshed, and that means you are limited to eight times a day or 48 times a day or API refreshes depends on your Power BI license limits.

However, having the ability to refresh even the dataflow more frequent, still can be useful for some scenarios. I bet if you came to this post by searching for such functionality, you have some ideas about that already 😉.

Video

FacebooktwitterredditpinterestlinkedintumblrmailFacebooktwitterredditpinterestlinkedintumblrmail
FacebooktwitterlinkedinrssyoutubeFacebooktwitterlinkedinrssyoutube
Reza Rad on FacebookReza Rad on LinkedinReza Rad on TwitterReza Rad on Youtube
Reza Rad
Trainer, Consultant, Mentor
Reza Rad is a Microsoft Regional Director, an Author, Trainer, Speaker and Consultant. He has a BSc in Computer engineering; he has more than 20 years’ experience in data analysis, BI, databases, programming, and development mostly on Microsoft technologies. He is a Microsoft Data Platform MVP for nine continuous years (from 2011 till now) for his dedication in Microsoft BI. Reza is an active blogger and co-founder of RADACAD. Reza is also co-founder and co-organizer of Difinity conference in New Zealand.
His articles on different aspects of technologies, especially on MS BI, can be found on his blog: https://radacad.com/blog.
He wrote some books on MS SQL BI and also is writing some others, He was also an active member on online technical forums such as MSDN and Experts-Exchange, and was a moderator of MSDN SQL Server forums, and is an MCP, MCSE, and MCITP of BI. He is the leader of the New Zealand Business Intelligence users group. He is also the author of very popular book Power BI from Rookie to Rock Star, which is free with more than 1700 pages of content and the Power BI Pro Architecture published by Apress.
He is an International Speaker in Microsoft Ignite, Microsoft Business Applications Summit, Data Insight Summit, PASS Summit, SQL Saturday and SQL user groups. And He is a Microsoft Certified Trainer.
Reza’s passion is to help you find the best data solution, he is Data enthusiast.

9 thoughts on “Refresh Power BI Queries Through Power Platform Dataflows: Unlimited Times with Any Frequency

  • Interesting. But I can’t really see any valuable use cases without being able to refresh the dataset more often. Can you give some examples?

    • Hi Lars
      If you look at it from only the Power BI point of view, yes, not much of offer. However, if you look at CDS as the data warehouse middle layer, then Power Platform Dataflows gives us the ability to feed data into this middle layer on any frequency we want, as much as needed. Power BI is not the only place that uses that data, it can be even Power Apps, Power Automate, Power Virtual Agent…

      Cheers
      Reza

  • Reza,

    Thank you for writing this article about Dataflows, I’d seen an increase in chatter on social platforms regarding it, but wasn’t sure of the use case for my organization. This seems like a logical next step for us to increase the refresh frequency before making the jump to Premium!

  • thanks for the sharing. very useful. I have a question about the dataflow capacity. because so many people now use power bi dataflow which result in slow performance. if we start to use dataflow in power platform. what is the capacity limitation ?

    • Dataflows shouldn’t cause slow performance, in fact they help with the refresh speed sometimes.
      regarding the limitation: The limitation is based on your license.
      for example; if you are using Power BI dataflows with Pro license then consider the space used is from your 10GB limitation. with Premium, more space of course.
      if you use external Power BI dataflows with your own azure data lake storage, then the limitation is based on your data lake subscription.
      with Power Platform dataflows and CDS, the limitation is based on your PowerApps license as mentioned in this blog article.
      Cheers
      Reza

  • Brilliant stuff Reza
    9/10 clients cannot afford Premium and the merging of Dataflows so useful.

    Do you know if having a Power Platform licence enables the Dev to provide a free data warehouse to a report consumer on Power BI Desktop? Or does the consumer need a PP licence to consume a CDS for example?

    • Thanks Jude
      If you use CDS, then you do need a PowerApps account for creating dataflows, which is PowerApps license.
      but then using the CDS in Power BI just needs a normal Power BI pro license.
      Cheers
      Reza

  • Hi Reza,
    Thanks for creating this article on data flows. I do have some questions. What are the major differences between the data flows being created in power bi services against power platform. Do we have anything in terms or speed?

    Also I have been creating data flows in service. However, the refreshes are taking quite a long time to complete.Upon enquiry, I got to know that there are lot of user interactions on reports and priority is given to user interactions over backend data operations such as refreshes, data flows data ingestion etc. Is there a way to get rid of slowness in refreshes?

    • Hi Venkatesh
      Power Platform Dataflows and Power BI dataflows although very similar, but have some differences. here and here I wrote about two fundamental differences.
      I felt the Power Platform Dataflows is a bit slower, probably because it is writing into CDS which have rules, logs etc.

      Regarding the speed of your dataflow; I would try to run it at non-peak times to see how the performance is
      Cheers
      Reza

Leave a Reply

%d bloggers like this: