Dynamic Row Delimiter and Column Delimiter in SSIS

FacebooktwitterredditpinterestlinkedintumblrmailFacebooktwitterredditpinterestlinkedintumblrmail
FacebooktwitterlinkedinrssyoutubeFacebooktwitterlinkedinrssyoutube

There are times that you want to make Row Delimiter and Column Delimiter dynamic in Flat File source, and you want to change these values dynamically with package variables.

usually you can use EXPRESSION properties for making things dynamic in SSIS, such as ConnectionString and bind it to a package variable.

As you may know there is no Column Delimiter in Flat File Connection manager, I think this is because limitation of SSIS Data flow task which is STATIC METADATA. actually SSIS can not solve columns on the fly in Data flow task, so It needs to define columns exactly at design time.

So there is no way to make Dynamic Column Delimiter in Flat File Connection manager .

On the other hand you can find RowDelimiter Expression property in Flat File Connection Manager, But after a simple test you will find that this will not work correctly, maybe this is a hidden bug in SSIS.

So what is work around?

If you want to read data from flat file into SQL Server table, use BULK INSERT TASK instead of Data flow.

in the BULK INSERT TASK you can set these EXPRESSION properties:(you can find them in the expression tab of Bulk Insert Task Editor)

ROWTERMINATOR
FIELDTERMINATOR

and this will work with binding to variables,this is I think because of physical layer behind of bulk insert task which probably use Bulk Insert ( T-SQL command ).

If you want to read data into other Data Source, You need Scripting for this. you can use Split(‘delimiter character’) .NET Method for creating dynamic delimiters in Script task .

FacebooktwitterredditpinterestlinkedintumblrmailFacebooktwitterredditpinterestlinkedintumblrmail
FacebooktwitterlinkedinrssyoutubeFacebooktwitterlinkedinrssyoutube
Reza Rad on FacebookReza Rad on LinkedinReza Rad on TwitterReza Rad on Youtube
Reza Rad
Trainer, Consultant, Mentor
Reza Rad is a Microsoft Regional Director, an Author, Trainer, Speaker and Consultant. He has a BSc in Computer engineering; he has more than 20 years’ experience in data analysis, BI, databases, programming, and development mostly on Microsoft technologies. He is a Microsoft Data Platform MVP for nine continuous years (from 2011 till now) for his dedication in Microsoft BI. Reza is an active blogger and co-founder of RADACAD. Reza is also co-founder and co-organizer of Difinity conference in New Zealand.
His articles on different aspects of technologies, especially on MS BI, can be found on his blog: https://radacad.com/blog.
He wrote some books on MS SQL BI and also is writing some others, He was also an active member on online technical forums such as MSDN and Experts-Exchange, and was a moderator of MSDN SQL Server forums, and is an MCP, MCSE, and MCITP of BI. He is the leader of the New Zealand Business Intelligence users group. He is also the author of very popular book Power BI from Rookie to Rock Star, which is free with more than 1700 pages of content and the Power BI Pro Architecture published by Apress.
He is an International Speaker in Microsoft Ignite, Microsoft Business Applications Summit, Data Insight Summit, PASS Summit, SQL Saturday and SQL user groups. And He is a Microsoft Certified Trainer.
Reza’s passion is to help you find the best data solution, he is Data enthusiast.

Leave a Reply