Knowing Your Data

Is it possible to be an effective data professional without knowing anything about the domain of data you support? I remember having a conversation a few years ago with a friend of mine, a fellow data practitioner. In his role as a database administrator, he asserted, he could do his job without knowing anything about the underlying data. His argument…


Using the SSIS Script Component With Multiple Outputs

One of the more common questions I find in SQL Server SSIS forums is how to split apart a single text file into multiple outputs. Files extracted from foreign sources often arrive in nonstandard formats, and are too often unchangeable at the source and must be parsed during the import process. For unconventional custom transformations such as this, using the…


Upcoming Full-Day SSIS Class in Dallas

This year I’ve delivered my full-day Building Better SSIS Packages course a half dozen times across the country, but I hadn’t scheduled this class in the Dallas area this year. As of today, that has changed! I’m happy to announce that I’ll be delivering this class at the Microsoft office in Las Colinas next month, on Friday, August 25th. Registration…


Create a Staging Load with Biml

When designing the ETL architecture for new or changing systems, one of the more common needs is to create a process that copies all of the data from a set of tables (perhaps even all tables) from a source system to a destination database. This is especially prevalent when building an operation data store (ODS), or building a set of…


Running SSIS in an Azure VM

In the previous post in this series, I addressed how to use an on-premises instance of SSIS to move data to and from Azure databases. If you’re running in a pure Azure environment without on-prem SQL Server, that load architecture would present some challenges. However, by running SSIS in an Azure VM, you can communicate move data into or out…


Creating the SSIS Catalog

If you are building SQL Server Integration Services (SSIS) packages, using the SSIS catalog as a deployment target is usually the easiest and most efficient solution. However, the SSIS catalog is not created by default, even when you select the SSIS components during the SQL Server installation process. The good news is that creating the SSIS catalog is a quick…


Trying to Solve the Wrong Problem

Some time back, while pulling into a local coffee shop, I spotted a stranded motorist in the parking lot. Per the Texan code of ethics, I was duty bound to walk over and offer my assistance, and in doing so I discovered that someone else had already stopped to help. The motorist was in the driver’s seat turning the key…


My SQL Server Development Toolbox

Having the right tools for the job makes the work much more efficient. However, for those just starting out in SQL Server land, it may not be clear which tools are useful or appropriate for a given task. I regularly field questions about which tools I use on my development machines, and why. To that end, I’ve compiled a brief…


Using ETL Staging Tables

Most traditional ETL processes perform their loads using three distinct and serial processes: extraction, followed by transformation, and finally a load to the destination. However, for some large or complex loads, using ETL staging tables can make for better performance and less complexity. As part of my continuing series on ETL Best Practices, in this post I will some advice…


Deleting a Package from the SSIS Catalog

Among the new features of SSIS 2016, one of my favorite is incremental package deployment. This new functionality allows the deployment of a single package to an SSIS catalog without having to deploy the entire project. Since the release of this version, I’ve been asked the following question a few times: “How does one go about deleting a package from…