In SQL Server Integration Services, the lookup component is one of the most frequently used tools for data validation and completion. The lookup component is provided as a means to virtually join one set of data to another to validate and/or retrieve missing values. Properly configured, it is reliable and reasonably fast. To get the most out of this component,…
SQL Server Integration Services (SSIS) has been a staple of the Microsoft business intelligence stack since 2005. Descended from Data Transformation Services (DTS), SSIS is an enterprise-ready, fast, and easy-to-use ETL system built on top of Microsoft SQL Server. Most anyone can learn the SSIS basics in just a few hours. SSIS Basics While SSIS makes it relatively easy to…
One of the most common questions I am asked about ETL configuration is how much memory SSIS requires. When configuring a new instance of SQL Server running SSIS, or adding SSIS to an existing SQL Server installation, it is important to understand how SSIS uses memory, and what factors in your package design will impact the amount of memory used…
When SQL Server 2016 was introduced, Microsoft added support for temporal tables. SQL Server temporal tables (also referred to as system-versioned tables), which follow the guidelines defined in the ANSI SQL:2011 specification. Temporal tables represent a time-based row versioning system for persisting the full history of changes. Temporal tables in SQL Server Keeping track of the history of changes in…
Call me pedantic, but I can be a stickler for standardized naming conventions. Whether it’s application code, database objects, documentation, or other digital assets, I find a tremendous amount of value in establishing a set pattern for naming and formatting. Especially for organizations with a large digital landscape, having a documented and agreed-upon standard for object names can make developers…
Temp tables are very handy when you have the need to store and manipulate an interim result set during ETL or other data processing operations. However, if you use SQL Server Integration Services as your ETL tool, you may find some challenges when trying to work with temp tables in SSIS packages, especially in the SSIS data flow. In this…
Encapsulating business logic into data movement and presentation is a critical part of a stable information management strategy. Too often, though, business logic is built and added late in the process, forcing it into whatever nooks and crannies are available. While this duct-tape approach sometimes works, it makes the resulting system difficult to maintain when the business logic is spread…
One of the most underutilized assets in SQL Server Management Studio is the SSMS project. Within Management Studio, one can create code projects that make organizing SQL scripts much simpler, and more easily integrate with source control systems. In this brief post, I’ll show the essentials of using SQL Server Management Studio projects for T-SQL code management. T-SQL Code Management…
We’re all creatures of habit, and work more efficiently when we know exactly where to find the tools we need. The same holds true with digital tools, and those of us who work with SQL Server will spend a lot of time using SQL Server Management Studio. SSMS has a lot of features, but each data professional will likely use…
SQL Server Integration Services is one of the most popular extract-transform-load (ETL) tools on the market. Learning how to use SSIS to move and transform data is an essential skill for every data professional, whether you’re a DBA, SQL Server developer, data analyst, or architect. Classroom Training: Introduction to SSIS This 3-day hands-on course will teach you what you need…