I am honored to have been selected to be a presenter at this year’s PASS Summit coming up this November. I will be presenting a half-day talk entitled Build a Metadata-Driven ETL Repository with Biml and SSIS:
Your boss tells you that a new field will be added to one of your vendor’s data files, and asks how long it will take to update the ETL to support this change. You start thinking through all of the pieces: the source packages, staging tables, stored procedures, and business rules. It could take weeks just to add a single column!
Managing metadata is one of the most important and time-consuming challenges an ETL developer will face. Simple changes can translate into many hours of discovery, code changes, and testing. However, there are ways to more easily manage this using tools you already have access to, which will ease the burden of SSIS package creation and maintenance. In this demo-packed session, we will discuss ETL metadata management using SSIS and Biml. We will review the challenges posed by “metadata guessing”, and will show a more predictable and robust way by building an end-to-end metadata repository to manage data types, sources and destinations, and transformations.
Using Biml to automate the process of ETL development and management is one of my favorite topics to teach, so I am very much looking forward to this session. Since we get a double time slot (2.5 hours) for this topic, we’ll get to dive deeply into metadata management patterns in Biml. I hope to see you there!