Data Pipeline orchestration is a challenge to effective data engineering yet there is no perfect solution that can meet all needs. However, if you are using Databricks, then the Workflows service is an excellent answer to this problem. In this video, I'll explain what workflows are, when and how to use them, and discuss powerful new features added to this service!

See my YouTube video content guide:
https://bpc-global-solutions-llc.gitb...

Support Me on Patreon Community and Watch this Video without Ads!
https://www.patreon.com/bePatron?u=63...

Slides and Code/Data
https://github.com/bcafferky/shared/b...