Articles by Tim Mitchell

Retrieve A List Of Files From FTP Using SSIS

Retrieve a List of Files from FTP using SSIS

The FTP protocol is one of the oldest methods for sharing and moving files. Although frequently considered to be an “old-school” way to transfer data, FTP is still relied upon in most every data movement architecture. Sadly, the functionality around FTP is very limited in SQL Server Integration Services. A common project requirement is to retrieve a list of files from…


SQL Saturday Pensacola

Upcoming Class: Building Better SSIS Packages in Pensacola

I’ll be teaching my popular day-long course, Building Better SSIS Packages, on Thursday, June 1 in Pensacola, Florida. This class is being delivered in conjunction with SQL Saturday Pensacola. Registration is open to the public, and early-bird pricing is in effect until early May. Let me know if you have any questions about this class. I look forward to seeing…


Why Data Warehouse Projects Fail

Why Data Warehouse Projects Fail

Data warehouse projects are among the most visible and expensive initiatives an organization can undertake. Sadly, they are also among the most likely to fail. At one time, Gartner reported that more than 50% of data warehouses would fail to make it to user acceptance. Because of the size of investment (both time and money) required, the success of such…


What is ETL

What is ETL?

In my ongoing series on ETL Best Practices, I am illustrating a collection of extract-transform-load design patterns that have proven to be highly effective. In the interest of comprehensive coverage on the topic, I am adding to the list an introductory prequel to address the fundamental question: What is ETL? What Is ETL? ETL is shorthand for the extraction, transformation,…


Side Project

The Side Project

When I first started working as a technologist some 17 years ago, I was doing fairly low-level stuff: moving and reimaging computers, network cabling, and basic desktop support were the extent of my responsibilities. I decided early on that I wanted to move into a more advanced role, but faced a common quandary: I couldn’t get the job I wanted…


ETL Error Handling

ETL Error Handling

In designing a proper ETL architecture, there are two key questions that must be answered. The first is, “What should this process do?” Defining the data start and end points, transformations, filtering, and other steps must be done before any other work can proceed. The second question that must be answered is “What should happen when the process fails?” Too…


It's a Lonely Road

Survey Results: Skipping the Holidays

Last week I posted a brief survey about working the holiday. To those who normally are not required to work the holidays, I asked if they would be willing to do so if the price was right, and if so, what that price would be (relative to what they normally make). There were 46 responses to this survey, and I’ve…


It's a Lonely Road

Survey: Skipping the Holidays?

For a little pre-holiday fun, I’ve got a simple scenario to present to you, and I am interested in your response. Your holiday plans are made. All of your family and friends expect to see you at the annual holiday gathering. Your kids are expecting you there on Christmas morning as they open presents. You unexpectedly get a call from…


SSIS custom logging levels

SSIS Custom Logging Levels

In my ongoing series on ETL best practices, I recently wrote about the importance of logging in extract-transform-load processes. For users of later versions (2012 and beyond) of SQL Server Integration Services, adding logging to those ETL processes is very simple. Since logging is managed by the catalog, adding logging to an ETL process is a one-step process and requires…


KPIs in SSRS

KPIs in SSRS 2016

Key performance indicators, or KPIs, are at-a-glance metrics for simple insight into the business. These are each designed to provide a glimpse into one aspect of a measurable business activity. KPIs are present in various implementations in most every reporting tool, including SQL Server Reporting Services. Until recently, KPIs in SSRS were not a native part of the SSRS reporting…