The optimisation of data pipelines can happen with dbt and Delta Live Tables. These tools focus on testing, version control, documentation and data quality.