Tag: SSIS

Hybrid, End-to-End; Power BI, Azure SQL Database, Data Factory

Posted on by Reza Rad

1

With the announcement of Power BI a week ago and with great features in this product, there are many opportunities that this product and service can be used. Power BI can be used alongside some other Azure services, and even On-Premises services to build an end-to-end enterprise BI and Data Analysis Solution. In this post I’ll just explain one example scenario of Hybrid and End-to-End BI solution that uses below technologies;

  • Data Sources On-Premises
  • Azure Data Factory or SSIS with Azure Pack for Data extraction and transformation
  • Azure SQL Database or Azure SQL Data Warehouse as the database / data warehouse engine in cloud
  • Power BI as the data analysis and front-end tool

[…]

I’ll Speak in SQL PASS Summit 2015; 3 Years in a row

Posted on by Reza Rad

It is a honor for me that I’ve been selected to speak in SQL PASS Summit 2015. SQL PASS Summit is the largest SQL Server conference and event in the world. Last year about 6ooo people from more than 55 countries attended this great event. I’ve been honored previously to speak in this great conference Read more about I’ll Speak in SQL PASS Summit 2015; 3 Years in a row[…]

SSIS Demos in Action: Tutorial Videos

Posted on by Reza Rad

podcast

SQL Server Integration Services is not a new technology, this technology is a mature data transfer and data consolidation tool which has been in the market since 2005. Prior than that SSIS had another name; DTS. However SSIS nowadays is capable of doing far more than what DTS had to offer.

You can transfer data from any source to any destination, you can upload or download files, you can set priority on execution of tasks, you can call third party applications or command lines, you can get part of the data from Web Service, you can zip or unzip result set, change structure of the result set and load it wherever you want.

[…]

Incremental Load: Change Data Capture in SSIS

Posted on by Reza Rad

15

Incremental Load is always a big challenge in Data Warehouse and ETL implementation. In enterprise world you face millions, billions and even more of records in fact tables. It won’t be a practical practice to load those records every night, as it would have many downsides such as;

  • ETL process will slow down significantly, and can’t be scheduled to run on small periods.
  • Performance of the source, and destination server will be affected badly, downtime of these systems would be longer.
  • More resources will be required to maintain the process. such as better processors, more RAMs… and adding these won’t help so much at the end, because the amount of data is increasing as times passes.
  • and many other issues.

So what would be the solution? Solution is Incremental Load approach. In this approach data will be loaded partially, preferably only part of the data that has been changed. A change set will be much smaller than the total amount of data. As an example in a 200 million records fact table which stored data for 10 years, only 10% percent of that data might be related to the current year and changes frequently, so you won’t usually required to re-load the rest 180 million records.

[…]

TechDays Hong Kong 2015: Azure Data Factory vs. SSIS

Posted on by Reza Rad

6

It has been long time passed from my presentation in Hong Kong TechDays 2015 on Mid February, I’ve been really busy so far and hadn’t chance to upload my presentation files here. I would like to thank all audience of my session on Azure Data Factory vs. SSIS, and provide you the link to my presentation slides. In this session you will see comparison between SSIS and Azure Data Factory on different factors such as developments, features, deployments, user experience, environment, and etc. For each comparison factor you will see a table comparison of these two products and their pros and cons for different situation.

[…]

Referring SSIS Variables from SSIS Script Task

Posted on by Reza Rad

Today I got a very valued feedback from a reader of my SSIS 2012 book, and based on that feedback I’ve found that I had a mistake in one of the exersizes of the book. an important note was not mentioned there. So in this post I want to mention that and applogize for any Read more about Referring SSIS Variables from SSIS Script Task[…]

Speaking at SQL Saturday 352 Sydney: ETL Architecture for Solving Data Warehouse Challenges

Posted on by Reza Rad

Hi, Thanks for all attendees to my session at SQL Satuday 352 Sydney, It was great to meet Grant, and Iman and also some Persian folks in this event. I was talking about my favorite subject: ETL Architecture for Solving Data Warehouse Challenges Session Description: Data Warehouses are heavily in use nowadays in most of Read more about Speaking at SQL Saturday 352 Sydney: ETL Architecture for Solving Data Warehouse Challenges[…]

SQL Saturday 337: Speaking about Foreach Loop Enumerators in SSIS

Posted on by Reza Rad

Hi, Thanks for attending in my session in SQL Saturday 337 Oregon. My session was about: Foreach Loop Enumerators in SSIS Description: There are different types of enumerators in the SSIS Foreach Loop Container which helps to implement different loop scenarios. In this chapter these different types explained with few case studies, with resort of Read more about SQL Saturday 337: Speaking about Foreach Loop Enumerators in SSIS[…]

Speaking at TechEd New Zealand 2014

Posted on by Reza Rad

It is my pleasure to speak in TechEd New Zealand 2014.  TechEd is happening between 9th to 12th of September, and it will be held in SkyCity, Auckland, New Zealand. the topic that I will talk about is:   ETL Architecture for Solving Data Warehouse Challenges Description: Data Warehouses are heavily in use nowadays in Read more about Speaking at TechEd New Zealand 2014[…]