ETL

Connecting to SharePoint Lists with SSIS

January 8, 2021

Connecting to SharePoint Lists with SSIS

SharePoint lists are popular and simple tools for storing modestly-sized discrete sets of data. SQL Server Integration Services (SSIS) includes a source component to allow reading data from SharePoint lists. In this post, I’ll demonstrate how to use SSIS to read from a SharePoint list using the OData source. SharePoint lists A SharePoint list is a simple collection of related…

Read More
ETL antipattern using full load instead of incremental load

December 14, 2020

ETL Antipattern: Performing Full Loads Instead of Incremental Loads

In my last post in the ETL Antipatterns series, I wrote about the common antipattern of ingesting or loading more data than necessary. This brief post covers one specific case of loading more data than necessary by performing a full data load rather than using a smaller incremental load. ETL Antipattern: performing full loads instead of incremental loads Earlier this…