Senior Data Engineer – Lead Scalable Data Pipelines

About the role

The Senior Data Engineer will lead data integration across the holding, delivering reliable batch and real‑time pipelines that feed reporting, analytics, machine learning and generative AI. You will own the data foundation, work closely with AI engineers and data scientists, and mentor a growing engineering team.

Key responsibilities

• Design and implement scalable ETL/ELT pipelines, using Apache Airflow for orchestration.
• Integrate databases, APIs, files, event streams and documents; manage incremental loads, change‑data capture and schema evolution.
• Ensure data integrity, platform scalability and database performance in partnership with DBAs.
• Maintain data models, source mappings, lineage, documentation and automated tests in version control.
• Collaborate with analysts, AI engineers and data scientists to deliver curated datasets and consistent business definitions.
• Monitor data quality, freshness and pipeline failures; implement alerts, retries, replay mechanisms and root‑cause fixes.
• Optimize queries, transformations, storage and processing costs for high‑volume workloads.
• Develop reusable connectors, automation scripts and AI‑assisted development tools while reviewing generated code for security and quality.
• Evaluate new data technologies and architectures against business, security and cost criteria.
• Mentor junior engineers, review code and enforce engineering standards.
• Build reproducible feature pipelines for model training, evaluation and inference, including retrieval‑augmented generation (RAG) pipelines.
• Provide governed data access for AI applications, enforcing permissions and traceability.

Required profile

• Bachelor’s degree in Computer Science, Computer Engineering or a related field.
• 4+ years of hands‑on data engineering experience, including production pipelines, data warehousing and SQL.
• Professional proficiency in the local language and English.
• Strong expertise in Apache Airflow, Python and SQL.
• Experience with relational and analytical databases such as PostgreSQL, SQL Server or ClickHouse, and familiarity with NoSQL solutions.
• Batch and streaming integration experience using Kafka or equivalent, including deduplication and incremental processing.
• Knowledge of dbt and automated data testing.
• RAG or vector‑search experience is a plus.

Required skills

• Apache Airflow
• Python
• SQL
• PostgreSQL
• SQL Server
• ClickHouse
• NoSQL databases
• Kafka
• dbt
• Git (version control)
• Data modeling
• Data lineage