ETL development and operations for a customer data platform (CDP) product

Background

A CDP brings together customer data collected through many channels and shapes it into a usable form. The types and volume of data differ from one customer deployment to the next, and the ETL (extract, transform, load) processes grow accordingly.

Products like this need more than building the processes: those processes have to run reliably, every single day. And when the data platform architecture is reviewed, the client also needs someone to consult on the technical side.

What we did

RG took charge of building the ETL workflows, assembling the processes from data ingestion through transformation and hand-off. We also advised on data platform design, taking into account how to split up the processes and how easy they would be to operate.

Rather than handing over what we built and walking away, we took on operations as well. Because the people who built the processes also handled additions and fixes, the client never had to explain the background from scratch.

Result

Since the engagement began in 2022, the same team has handled ETL development, design consulting, and operations. We did not simply hand over the ETL workflows we built; we took on running them as well.