Senior Data Engineer

About the role

You will build the data foundation for Refractal, including the evidence store used by customer auditors, analytics for compliance reporting, and training datasets for models. This role involves designing data models, building streaming and batch processing layers, and ensuring data quality and residency for regulated customers.

Responsibilities

• Design the data model for our append-only, cryptographically signed evidence store.
• Build the streaming and batch layer that turns ingested activity and detections into queryable data, working with our integrations engineers on schemas.
• Own data quality, retention, residency and deletion controls for regulated customers.
• Build training and evaluation datasets with our data scientists and security engineers.
• Make compliance reporting against the EU AI Act, NIST AI RMF and ISO/IEC 42001 quick to produce from our data.

Required skills

• Python
• SQL
• Postgres
• BigQuery
• ClickHouse
• Kafka
• Pub/Sub
• Airflow
• Dagster
• Data modelling
• Data lineage

Nice to have

• Banking
• Insurance
• UK GDPR
• Data residency

Benefits

• Equity
• Pension contribution
• Private medical cover
• Private dental cover
• Visa support
• Relocation support

About the Company

Refractal is an AI security company building the infrastructure organisations need to defend against autonomous AI threats. Founded on cybersecurity and AI expertise from MIT, Microsoft, NASA and the U.S. Navy, the company provides tools to detect AI-enabled threats at low cost.