ETL Architecture

A Better Way to Execute SSIS Packages with T-SQL

There are several ways to execute SSIS packages that have been deployed to the SSIS catalog, and my favorite way of performing this task is to execute SSIS packages with T-SQL. There are a couple of challenges that come up when executing catalog-deployed packages using the default settings in T-SQL, but I have workaround for those issues which I’ll cover…


ETL Atomicity

I still remember the first real ETL process I developed. I was working for a hospital at the time, going through a major system implementation as we replaced a 17-year-old UNIX-based system with a more modern healthcare application suite with a SQL Server back end. I was tasked with building, testing, and executing the ETL processes for this conversion. While…


Get Your Email Out of my ETL

Question from someone in one my recent classes: “What tool do you use to send email from ETL processes?” My response: “I don’t.” The tl;dr version of this post is I let my extract-transform-load processes do just ETL, and leave notifications to the scheduling system where they belong. Get Your Email Out of my ETL Before you read any further,…


ETL Modularity

Imagine for a moment that you’ve built a software thing. In fact, we’ll call it The Thing. You put a lot of work into The Thing, and it does exactly what you wanted it to. You put The Thing into play as part of a larger solution and, after a couple of revisions, its behavior is verified and it is…


How To Get Fired from an ETL Developer Job

Through the course of my 8-someodd years of building and fixing ETL processes, I’ve had the opportunity to see a lot of ETL code. Some of that code was really good, well-thought-out and carefully executed. Other load processes were – well, let’s just say that they provide plenty of consulting opportunities (and I include much of my early code in…


New Blog Series: Parent-Child Architecture in SSIS

I’m kicking off a new series of blog posts discussing the topic of parent-child architectures in SQL Server Integration Services.  The links to the posts in this series are below. Parent-Child SSIS Architecture SSIS Parent-Child Architecture in Package Deployment Mode SSIS Parent-Child Architecture in Catalog Deployment Mode I still remember the first SSIS package I ever deployed to a production environment. …