Data Engineer & Data Platform Engineer
Glovo is the app that allows you to get the best products of your city in a few minutes. We connect users, businesses and couriers to make that possible. The project was born with the aim of transforming the way users acquire what they need, making cities more accessible.
Originally joining the central Data Engineering team to manage all data pipelines for the company using Luigi and Jenkins as orchestration stack, and Redshift as our data warehouse. The technical decisions taken at the time were not the best fit for a company that was growing as fast as Glovo did, and soon enough the team started evaluating new technologies to replace them. Moreover, the amount of time dedicated to support stakeholders and maitain existing pipelines was too high, and I made it my personal objective to reduce this as much as possible. This lead to the creation of a first support team that would take over triaging of requests and gave our team room to grow and engage in the transition to the next stack.
In this way, and with the introduction of a data mesh architechture, the team moved on to build the platform that allowed
all newly created data domains to build and manage their own data products, using Airflow for orchestration and Trino and Glue as data lake,
thus becoming the Data Platform team. A side effect of the transition to a platform team was the introduction of knowledge in new technologies,
in particular Kubernetes, ArgoCD and Terraform for infrastructure as code.
My main objective within this team was to design, implement and oversee
all Trino infrastructure. This infra continues running until today, with several clusters serving hundreds of thousands of queries per day.