Object variables in SSIS are incredibly versatile, allowing the storage of almost any type of data (even .NET objects). In my last post on this topic, I demonstrated how an SSIS object variable containing a .NET DataSet object could be used by the for each loop container as an iterator. In this post, I’ll continue the discussion by showing how…
Some time back I wrote about how to use the script component to parse out ragged data files in SSIS. In this post, I’ll continue the discussion to describe how to handle mixed format data files – specifically, those with several different record types in each file. In a perfect world, a flat file will contain a single record type….
When I think about Baltimore, I think about Edgar Allan Poe. Baltimore was Poe’s one-time home, and in fact there is a museum located in a home in which he once lived. I’ve been a Poe fan since high school, and am looking forward to seeing some of the Poe-related sights when I’m in town next month. Why am I…
“He’s a screw-up. Always trying things that don’t work.” I’ll be honest: I used to be afraid of being the person described above. I didn’t want to be known as someone whose ideas didn’t work. And to that end, I was successful: most of what I tried was successful. The bad news was that I wasn’t doing much. I was…
The answer: It Depends. One of the more common problems I encounter when managing data quality, especially in an ETL process, is the proper handling of null, empty string, or zero values. When I put on my preaching shoes to talk about bad data, this is one the areas I have to spend a lot of time covering because it…
This week, PASS announced the Best of PASS Summit 2014, a collection of the 10 highest-rated sessions from last fall’s PASS Summit in Seattle. I was pleasantly surprised to learn that my session, “Building Bullet-Resistant SSIS Packages”, was among these. All of these sessions (including lots of other content) are available for viewing on the PASS TV channel on YouTube….
Earlier today, the fine folks at the F. Oobar Corporation released a revolutionary product: a universal data integration utility. This software component, known as the Baseline Ongoing Generic Utility for Synergy, will run on any platform and can convert data to and from almost any format automatically. It also reads the semantics of the data to determine exactly how it…
Yesterday I read an article entitled “Why I Don’t Want to Have Coffee With You”, in which the author writes that he doesn’t have the time or the desire to simply “have coffee”. While I empathize with some of the author’s justifications for his position, I was disappointed at the hard line he took on this. Personally, I prefer a…
How many times have you said to yourself, “Someone should build an application that does [x]…”, or “Wouldn’t it be easy to add automation to [y]”, or “It would be a lot of fun to work on a project to build [z]”? For me, this has happened a lot, and seems to occur more frequently the longer I’m in this…
When processing data files using SQL Server Integration Services, it is not uncommon to find files with different end-of-line markers for each line in the file. In this post, I will demonstrate how to fix inconsistent line terminators in SSIS to avoid ETL errors. Fix Inconsistent Line Terminators in SSIS In every text file, there are unprintable characters called line…