Multiproject architecture - DEV & PROD

Hello Keboola and community,

how do you use DEV and PROD environment in multiproject architecture? What are your best practices?

Our story: We are using multiproject architecture heavily. So one pipeline from source to destination can be across 3-5 Keboola projects. (One for data extraction, few of them for transformation and last one for writers.) It helps us to lower the complexity.

Problem: There is no easy way how to test whole pipeline, currently there are only Development Branches inside same project. So we are doing DEV inside production by renaming objects now to be able to share them between projects. But it is not safe, reliable and clean method.

What do you suggest?

 

2 replies