Senior Data Engineer (Azure Databricks)

About the role

Callista Group AG is seeking two experienced Data Engineers to build a new Lakehouse data platform for an insurance company in the Frankfurt area. You will be responsible for developing data pipelines from source systems to the core model while providing technical leadership and mentoring the client's team. The role involves working in a tandem model to transfer Databricks expertise and ensure the successful implementation of the data platform.

Responsibilities

• Connect source systems, external providers, and CRM systems with automated loading processes
• Develop ETL/ELT pipelines using Delta processing, error handling, and recovery procedures
• Perform data deduplication and consolidation of customer and intermediary data
• Implement the core data model with historization using the Medallion architecture
• Establish data quality rules, monitoring, and alerting routines
• Provide data exports and feedback loops to operational systems
• Contribute to development pipelines, CI/CD, access management, and data lineage
• Document processes and conduct knowledge transfer with the client team

Required skills

• Databricks
• PySpark
• SQL
• Delta Lake
• Azure Data Factory
• Data Modeling
• Git
• CI/CD
• Automated Testing
• German
• English

Nice to have

• Data Vault
• Dimensional Modeling
• AI-assisted development
• Insurance industry data

Qualifications

• Professional experience as a Data Engineer in Data Warehouse or Lakehouse solutions

Certifications

• Databricks Certified Data Engineer
• Azure Data Engineer Associate

About the Company

Callista Group AG is a service provider currently supporting a client in the insurance sector with the development of a new data platform.