Group Data Engineer I

AI in this role

databricks

KEY ACCOUNTABILITIES

Technical Leadership & Architecture:

Define target architectures, engineering patterns and standards for batch/streaming integration, lakehouse design, data modelling, sharing and serving.

Lead design reviews and technical decisions for complex or high-impact initiatives, ensuring scalability, security, resilience and reuse.

Data Engineering & Platform Delivery:

Design and build production-grade pipelines and reusable frameworks using SQL, Python, Spark and cloud data platform technologies.

Establish reusable ingestion/transformation components, CI/CD and infrastructure-as-code to accelerate onboarding and reduce delivery risk.

Reliability, Performance & Cost:

Set standards for observability, SLAs, performance tuning, disaster recovery, incident prevention and root-cause resolution.

Optimize compute, storage and workload design to improve platform performance, reliability and unit cost.

Data Quality, Governance & Security:

Embed automated data quality, lineage, metadata, access control, privacy and retention requirements into the engineering lifecycle.

Partner with Governance and Security teams to ensure critical data products are trusted, auditable and compliant.

Engineering Excellence & Automation:

Drive automated testing, code quality, version control, deployment automation, coding standards and technical debt reduction.

Evaluate emerging technologies, lead proofs of concept and convert proven capabilities into scalable enterprise standards.

Collaboration & Mentoring:

Mentor engineers, raise technical capability and provide hands-on support for complex troubleshooting and engineering decisions.

Collaborate with product, analytics, AI/ML, platform and source-system teams to deliver reusable, trusted data capabilities.

QUALIFICATIONS, EXPERIENCE AND SKILLS

Qualifications:

Bachelor's degree in Computer Science, Engineering, Information Technology or a related discipline; a Master's degree is desirable.
Minimum of 7+ years of experience in data architecture, data engineering, or a similar role, with a strong focus on designing large-scale data platforms.
- Advanced hands-on expertise in SQL, Python, Spark, distributed data processing and data modelling.
- Strong experience with cloud data platforms (Azure, AWS or GCP); Databricks/lakehouse experience is preferred.
- Deep knowledge of batch and streaming ingestion, CDC, orchestration, APIs, data lakes/warehouses and modern data architecture patterns.
- Strong experience with Git, CI/CD, infrastructure-as-code, automated testing, observability and production engineering practices.
- Working knowledge of data governance, security, privacy, lineage, metadata management and data quality controls.
- Demonstrated ability to lead architecture/design reviews, resolve complex technical issues, mentor engineers and influence senior stakeholders.

Key Skills:

Strong leadership, collaboration, and communication skills.
Expertise in cloud platforms and services (Azure preferred).
Proficiency in data pipeline orchestration tools (e.g., Apache Airflow, Azure Data Factory).
Knowledge of containerization and microservices architecture.
Familiarity with data visualization and BI tools (e.g., Power BI, Tableau).
Experience with infrastructure-as-code tools (e.g., Terraform, CloudFormation).
Ability to think strategically while balancing business needs and technical solutions.
Experience with Agile methodologies and working in a fast-paced, collaborative environment.

Desirable Qualifications:

Certifications such as Microsoft Certified: Azure Solutions Architect Expert or Google Cloud Professional Data Engineer.
Experience with machine learning and AI workloads on data platforms.
Knowledge of DevOps practices and CI/CD for data pipelines.

#LI-AA6

How we rate this

Group Data Engineer I at DP World rates 67 out of 100 for how much of the daily work is AI. That makes it Works on AI (AI Level 3 of 4). The level is about AI in the job, not seniority.

Classification

Works on AI. The daily work is on AI products, without building the model.

●●●● Builds AI80 to 100
●●●○ Works on AI60 to 79
●●○○ Uses AI40 to 59
●○○○ Little AI0 to 39

Levels come from how often the tools, models and workflows of the role are named in the posting itself. Open the description and count.

Prepare for this job

A free preview built only from this posting: what it asks for, what you could be asked in an interview, and how to adjust your resume.

Skills and AI tools this role asks for

Databricks

Questions you could be asked

What's a project where you used Databricks hands-on?
Describe a typical day in a role like this one: which parts run through AI directly?
If you removed AI from this role, what would be left, and how do you decide what still needs a human?

Adapt your resume

List these exact terms on your resume: Databricks. An applicant tracking system matches the wording, not the idea.
Attach one line of real, concrete experience to at least one of them — a tool named with nothing behind it rarely survives a human read.
Show where AI is part of your daily process, not a one-off project — this role expects it to be a running habit.

Want an expert to read your CV for this job?

Free. Send your CV and the role you want next. We reply by email within 2 to 4 business days.

Get a free CV review

Get new data engineer jobs (Works on AI ●●●○ or higher) by email

One email a week with the new data engineer jobs (Works on AI ●●●○ or higher), each rated for how much AI is in the work. No recruiter spam, unsubscribe in one click.

Also alert me for new jobs at DP World

Free. One email a week. Unsubscribe in one click.