Senior Data Engineer

About the role

As a Senior Data Engineer, you will lead the end-to-end design and delivery of complex data pipelines and cloud infrastructure for Kpler's new Marine Insurance initiative. You will act as a technical leader, setting data architecture directions and mentoring peer engineers to ensure the platform scales reliably. The role involves ingesting, processing, and serving large-scale data to global customers.

Responsibilities

• Design and own end-to-end data pipelines from architecture through maintenance.
• Deploy and containerize applications within AWS using Python, Docker, Kubernetes, Airflow, and Terraform.
• Architect PostgreSQL database structures and event-streaming flows using Kafka and Flink.
• Lead complex pipeline design, peer reviews, and implement logging, monitoring, and alerting frameworks.
• Collaborate cross-functionally with product managers and data scientists while mentoring peer engineers.
• Research and implement modern data infrastructure improvements to elevate platform capabilities.

Required skills

• Python
• Docker
• Kubernetes
• AWS
• Airflow
• PostgreSQL
• Kafka
• Flink
• Terraform
• CI/CD
• Agile

Nice to have

• Large-scale analytics platforms
• Commodities market data
• Shipping market data
• Energy market data
• Marine insurance market data

About the Company

Kpler simplifies global trade information and provides valuable insights to help organizations make informed decisions in commodities, energy, and maritime sectors. Founded in 2014, the company employs over 850 experts from 69 countries to transform intricate data into actionable strategies.