SSIS

Retrieve A List Of Files From FTP Using SSIS

Retrieve a List of Files from FTP using SSIS

The FTP protocol is one of the oldest methods for sharing and moving files. Although frequently considered to be an “old-school” way to transfer data, FTP is still relied upon in most every data movement architecture. Sadly, the functionality around FTP is very limited in SQL Server Integration Services. A common project requirement is to retrieve a list of files from…


SSIS custom logging levels

SSIS Custom Logging Levels

In my ongoing series on ETL best practices, I recently wrote about the importance of logging in extract-transform-load processes. For users of later versions (2012 and beyond) of SQL Server Integration Services, adding logging to those ETL processes is very simple. Since logging is managed by the catalog, adding logging to an ETL process is a one-step process and requires…


SSIS Execution Status

SSIS Execution Status Lookup

The SSIS catalog comes packaged with a rich set of built-in reports that give those monitoring Integration Services a window into what has been happening inside the catalog. However, if you work with SSIS long enough, you’ll eventually need to write your own queries against the underlying tables. Of the many things the built-in catalog tables and views do well,…


Unzip Files with SSIS

Unzip Files with SSIS

In my last post I shared how to build a package in SSIS to perform simple file archive (zip) operations. In this post, I’ll show you how to go the other direction and unzip files with SSIS. Unzip Files with SSIS Similar to the file zip operation, this solution uses the following tools: 7Zip: This free utility manages the creation…


Zip Files in SSIS

Of the very many things SQL Server Integration Services does well, one of its shortcomings is that it does not have a built-in way to create compressed archive (ZIP) files. Generating such files is a common need in ETL processes, so it’s surprising that the fifth generation of SSIS still does not have a native task to manage this operation….


SSIS Package Validation

SSIS Package Validation in the Catalog

Built into the SQL Server Integration Services catalog is the ability to run a validation without actually executing the package. Running a package validation in the SSIS catalog performs a high-level check against the underlying metadata to check for common points of failure (especially those related to data flows). SSIS package validation is not designed to capture every metadata issue,…


SSIS Data Taps

SSIS Data Taps

One of my favorite testing features of SSIS is also one of the most underutilized. SSIS data taps were introduced with the SSIS catalog in SQL Server 2012 as a way to capture data within one leg of a data flow task and write it out to a file for testing or auditing purposes. In this brief post, I’ll show…


Parameterizing Settings in SSIS

A Shortcut for Parameterizing Settings in SSIS

I’ve written quite a bit about the benefit of externalizing changing values in SSIS packages. Moving static values such as connection strings and file paths to a configurable input makes easier the tasks of testing, changing, and auditing the process in the future. The short and generic story here is: don’t hard code values that can change. In SQL Server…


A Better Way to Execute SSIS Packages with T-SQL

There are several ways to execute SSIS packages that have been deployed to the SSIS catalog, and my favorite way of performing this task is to execute SSIS packages with T-SQL. There are a couple of challenges that come up when executing catalog-deployed packages using the default settings in T-SQL, but I have workaround for those issues which I’ll cover…


DevConnections 2016

Speaking at DevConnections 2016

I am excited to share that I will be presenting at the DevConnections conference in Las Vegas in October of this year. This is my fourth (if I remember correctly) time to have spoken at DevConnections, and the first time I will have delivered a full-day workshop there. This year I will present one workshop and two regular presentations: Building…