SQL Server 2016 is under a heavy improvement plan, The latest CTP version (Community Test Preview) of it is 2.3 which released as preview three days ago. CTP 2.3 has some new features across all components of SQL Server, such as SSRS, SSAS, SSIS, MDS and etc. Previously I wrote about some new changes in Read more about SSIS 2016; What’s New in CTP 2.3[…]
It is a honor for me that I’ve been selected to speak in SQL PASS Summit 2015. SQL PASS Summit is the largest SQL Server conference and event in the world. Last year about 6ooo people from more than 55 countries attended this great event. I’ve been honored previously to speak in this great conference Read more about I’ll Speak in SQL PASS Summit 2015; 3 Years in a row[…]
SQL Server Integration Services is not a new technology, this technology is a mature data transfer and data consolidation tool which has been in the market since 2005. Prior than that SSIS had another name; DTS. However SSIS nowadays is capable of doing far more than what DTS had to offer.
You can transfer data from any source to any destination, you can upload or download files, you can set priority on execution of tasks, you can call third party applications or command lines, you can get part of the data from Web Service, you can zip or unzip result set, change structure of the result set and load it wherever you want.
Incremental Load is always a big challenge in Data Warehouse and ETL implementation. In enterprise world you face millions, billions and even more of records in fact tables. It won’t be a practical practice to load those records every night, as it would have many downsides such as;
So what would be the solution? Solution is Incremental Load approach. In this approach data will be loaded partially, preferably only part of the data that has been changed. A change set will be much smaller than the total amount of data. As an example in a 200 million records fact table which stored data for 10 years, only 10% percent of that data might be related to the current year and changes frequently, so you won’t usually required to re-load the rest 180 million records.
It has been long time passed from my presentation in Hong Kong TechDays 2015 on Mid February, I’ve been really busy so far and hadn’t chance to upload my presentation files here. I would like to thank all audience of my session on Azure Data Factory vs. SSIS, and provide you the link to my presentation slides. In this session you will see comparison between SSIS and Azure Data Factory on different factors such as developments, features, deployments, user experience, environment, and etc. For each comparison factor you will see a table comparison of these two products and their pros and cons for different situation.
Today I got a very valued feedback from a reader of my SSIS 2012 book, and based on that feedback I’ve found that I had a mistake in one of the exersizes of the book. an important note was not mentioned there. So in this post I want to mention that and applogize for any Read more about Referring SSIS Variables from SSIS Script Task[…]
Hi, Thanks for all attendees to my session at SQL Satuday 352 Sydney, It was great to meet Grant, and Iman and also some Persian folks in this event. I was talking about my favorite subject: ETL Architecture for Solving Data Warehouse Challenges Session Description: Data Warehouses are heavily in use nowadays in most of Read more about Speaking at SQL Saturday 352 Sydney: ETL Architecture for Solving Data Warehouse Challenges[…]
Hi, Thanks for attending in my session in SQL Saturday 337 Oregon. My session was about: Foreach Loop Enumerators in SSIS Description: There are different types of enumerators in the SSIS Foreach Loop Container which helps to implement different loop scenarios. In this chapter these different types explained with few case studies, with resort of Read more about SQL Saturday 337: Speaking about Foreach Loop Enumerators in SSIS[…]
It is my pleasure to speak in TechEd New Zealand 2014. TechEd is happening between 9th to 12th of September, and it will be held in SkyCity, Auckland, New Zealand. the topic that I will talk about is: ETL Architecture for Solving Data Warehouse Challenges Description: Data Warehouses are heavily in use nowadays in Read more about Speaking at TechEd New Zealand 2014[…]
I am honored that I will speak in SQL Saturday 316 Christchurch on 14th of June 2014. Presentation topic is : ETL Architecture for Solving Data Warehouse Challenges presentation Description: Data Warehouses are heavily in use nowadays in most of businesses around the world. Data Warehousing brings higher performance, faster and more insightful data analysis Read more about SQL Satuarday 316: Christchurch: ETL Architecture for Solving Data Warehouse Challenges[…]