Live Streaming Weather Station with Cortana Analytics – Part 3

power2

In previous posts (part 1 and part 2), I have explained the process of data live streaming. In the first post, I have explained the process of setting up the Raspberry PI 3 and Weather station. In the second one, I have explained the process of setting up the Event Hub that helps us to. In this post, I will explain the process of creating and setting up the Stream Analytics.

iotprocess

 

What is Stream Analytics?

Stream Analytics is an event processing engine, which helps to send data from Event Hub to Power BI and Azure ML.

In this post I will explain how to use Stream Analytics to see live data in Power BI.

To create Stream Analytics, I login in new Azure Portal. In the search area I typed” Stream”, then I clicked on “Stream Analytics Job”

streamanalytics1

Then in “Name” area I clicked on “Stream Analytics Job”. By clicking on this icon, a new page will be appear. In this new page click on “Create” to start creation of New Stream Analytics.

streamanalytics2

 

By clicking on create a new analytics, new page will be shown that we have to provided some information such as “Job Name”, “Subscription” that you already selected. Moreover, we need to identify the new resource group. Finally, we should identify the location that we want to set up the stream analytics server.

stream3

Stream Analytics accepts input from Event Hub, Blob Storage and IoT Hub. Stream analytics able to received data from these inputs.

Then we should write a query to sent the data from inputs to outputs. Stream Analytics able to send data to : Power BI, Azure SQL Server, Azure Data Lake Store, Blob storage, Table Storage, and Document DB.

 

stream5

The query is “Stream Analytics Language” which is so similar to SQL statement. After creation a new stream analytics, we should specify the input and output for it. we specify the Input, Output, and Query by clicking on each of them.

 

streamanalytics7

after clicking on “Input” icon, we need to specify the input media. In our scenario “Event Hub” is the main input to get data. Hence, we specify the services and create the input.

stream8

After Creating Input, we also have to select the Outputs. The output will be Power BI. By selecting the Power BI. In output we just need specify the Group workspace, Dataset Name and Table Name. Stream analytics will create automatically table in Power BI with the name that we allocated here(see below image). However the data attributes will be identified in Stream analytics Query.

 

Powerbi4

Stream Analytics Query is so similar to SQL Statement (See below image).

In this scenario, we want to get all data from Event Hub. Hence, we use “Select *” then in from clause we choose “IoTEventHub” that means all data have been collected from weather station. Then, we insert data from “IoTPowerBI”.

stream9

In Stream Analytics Query, we can also use the Built-in Function. In the below image you can see the all possible functions.

stream10

stream12

 

After creation Stream Analytics, we need to run the program(.Net Program that we already have it, Part1) and then start the Stream Analytics. To fetch data from event hub we have to start the stream analytics service (see below image)

streamaanalytics3

 

Data will be send to the Power BI. As you can see in the below image. By running the Stream Analytics. We will see below diagram in Power BI.

power2

First we have to run the .Net code to get data from weather sensor. The data will be send from Net program to Event Hub. Below video shows the process. After that data will be send to the Stream Analytics. Stream Analytics will send data to Power BI. below video show the process.

After creation live data in Power BI.

There is a possibility to send the collected data to Azure ML and do more analytics. In the next post I will explain how to create prediction analytics in Azure ML for identifying the anomaly in the collected data from weather sensor.

[1]. https://docs.microsoft.com/en-us/azure/stream-analytics/stream-analytics-introduction.

[2]. https://www.hackster.io/windowsiot/build-hands-on-lab-iot-weather-station-using-windows-10-5b818f.

Save

Save

Leila Etaati on LinkedinLeila Etaati on TwitterLeila Etaati on Youtube
Leila Etaati
Trainer, Consultant, Mentor
Leila is the first Microsoft AI MVP in New Zealand and Australia, She has Ph.D. in Information System from the University Of Auckland. She is the Co-director and data scientist in RADACAD Company with more than 100 clients in around the world. She is the co-organizer of Microsoft Business Intelligence and Power BI Use group (meetup) in Auckland with more than 1200 members, She is the co-organizer of three main conferences in Auckland: SQL Saturday Auckland (2015 till now) with more than 400 registrations, Difinity (2017 till now) with more than 200 registrations and Global AI Bootcamp 2018. She is a Data Scientist, BI Consultant, Trainer, and Speaker. She is a well-known International Speakers to many conferences such as Microsoft ignite, SQL pass, Data Platform Summit, SQL Saturday, Power BI world Tour and so forth in Europe, USA, Asia, Australia, and New Zealand. She has over ten years’ experience working with databases and software systems. She was involved in many large-scale projects for big-sized companies. She also AI and Data Platform Microsoft MVP. Leila is an active Technical Microsoft AI blogger for RADACAD.

Leave a Reply