Random SSIS Advice: Limit using data flow tasks that sort, union, join


When using a Data Flow Task in Integration Services, try to avoid data manipulations such as unions, sorts and joins using data flow tasks. Try to include all join, sort and union operations within your source queries itself. This way, you can do away with run-time issues that crop up when running your package with large amounts of data. Usually various memory restriction errors.

Then, why do they have all these tasks like Sort, Union All and Merge Join you may ask. Well, you could use them when you need to join or combine data coming from disparate sources such as an Excel file and a SQL Server database, or if you would like to sort data coming from a text file etc.

I just fixed a couple of memory errors and brought down the overall execution time of a certain production package, just by following this practice. :)

About these ads

2 thoughts on “Random SSIS Advice: Limit using data flow tasks that sort, union, join

  1. Woah! I’m really loving the template/theme of this website. It’s simple, yet
    effective. A lot of times it’s tough to get that “perfect balance” between usability and visual appeal. I must say you’ve done a awesome job with this.
    Also, the blog loads extremely quick for me on Opera.
    Outstanding Blog!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s