Hello Keboola and community,
how do you use DEV and PROD environment in multiproject architecture? What are your best practices?
Our story: We are using multiproject architecture heavily. So one pipeline from source to destination can be across 3-5 Keboola projects. (One for data extraction, few of them for transformation and last one for writers.) It helps us to lower the complexity.
Problem: There is no easy way how to test whole pipeline, currently there are only Development Branches inside same project. So we are doing DEV inside production by renaming objects now to be able to share them between projects. But it is not safe, reliable and clean method.
What do you suggest?
@Pavel Chocholouš Good idea, thank you. Please contact me at petr.simon@nova.cz with some time proposals to plan a call. Renaming columns is not our problem.
Would you mind to spend 30 minutes on a call and explain your use case in detail? From what I see here I'd suggest not renaming columns, but adding them, which keeps the data catalog "backward compatible".