Senior Data Engineer(6+ years with MS Fabric, Azure, ADF, Data Platforms only)

Job Summary

We are looking for a Senior Data Engineer to design, build, and operate modern data solutions that power our data products, analytics, integrations, reporting, and emerging AI capabilities.

This is a hands-on engineering role for someone who combines strong data engineering fundamentals with experience in modern cloud data platforms, APIs, integration patterns, and production-grade engineering practices.

You will work across technologies such as Databricks, Microsoft Fabric, Snowflake, and cloud-native services, depending on the needs of the solution. We value engineers who understand the underlying architecture and engineering principles rather than being tied to a single technology.

We are also looking for an AI-first engineer who actively uses AI-assisted development to improve how solutions are designed, developed, tested, documented, and supported while maintaining strong engineering quality and accountability.

Key Responsibilities

Build Modern Data Solutions

• Design, develop, and maintain scalable data pipelines supporting operational, analytical, reporting, and AI use cases.
• Build batch, incremental, near-real-time, and event-driven data processing solutions.
• Develop data ingestion, transformation, orchestration, and delivery workflows using modern cloud data platforms.
• Design and implement data models optimized for performance, maintainability, and consumption.
• Develop reusable engineering components and patterns rather than one-off solutions.
• Optimize pipelines and data processing for reliability, scalability, performance, and cost.
• Contribute to technical design discussions and architecture decisions.

Build Enterprise Data Integrations

• Design and implement integrations across enterprise applications, data platforms, and third-party systems.
• Develop reusable integration services and data-access capabilities.
• Build secure REST and GraphQL APIs to expose data and platform capabilities.
• Implement event-driven and asynchronous integration patterns where appropriate.
• Integrate systems using APIs, connectors, messaging, events, and modern data-sharing techniques.
• Apply appropriate authentication, authorization, versioning, monitoring, and API lifecycle practices.

Engineer for Quality & Reliability

• Build automated data validation, reconciliation, and quality controls into pipelines and data products.
• Implement appropriate unit, integration, and data testing as part of development.
• Build monitoring and observability into pipelines, APIs, and integration services.
• Troubleshoot complex production issues and perform root-cause analysis.
• Implement preventive improvements rather than repeatedly addressing symptoms.
• Continuously improve the reliability, resiliency, maintainability, and performance of production solutions.

Apply AI-First Engineering Practices

• Use AI-assisted development throughout the engineering lifecycle, including design, coding, testing, documentation, troubleshooting, and optimization.
• Apply appropriate technical review and validation to AI-generated code and engineering artifacts.
• Practice spec-driven development by defining clear requirements, data contracts, API contracts, acceptance criteria, and test expectations before implementation.
• Use AI to accelerate engineering delivery while maintaining security, maintainability, and quality.
• Contribute to the development of data solutions supporting AI agents, RAG, semantic models, machine learning, and other AI-enabled applications.
• Experiment with emerging engineering capabilities and share effective practices with the broader team.

Enable Secure & Governed Data

• Implement data quality, lineage, metadata, security, and auditability requirements within data solutions.
• Build data pipelines and integrations appropriate for sensitive and regulated data.
• Follow established governance, risk, security, and compliance requirements.
• Support secure and auditable processing of KYC, UCC, public-record, and other business data.
• Maintain clear technical documentation for data flows, APIs, transformations, and operational processes.
• Partner with architecture, security, governance, and platform teams to resolve technical and compliance requirements.

Operate What You Build

• Take ownership of solutions through development, deployment, and production operation.
• Monitor pipeline execution, API health, data quality, and platform performance.
• Investigate incidents and integration failures and drive issues through resolution.
• Participate in code reviews and continuously improve engineering practices.
• Automate deployment, testing, configuration, and operational processes wherever practical.
• Contribute to improving team standards based on lessons learned from production.

Collaborate & Contribute

• Work closely with Lead Data Engineers, product teams, architects, analysts, AI teams, and business stakeholders.
• Translate requirements into practical and scalable technical solutions.
• Participate actively in architecture and technical design discussions.
• Communicate technical decisions, dependencies, risks, and tradeoffs clearly.
• Share knowledge and reusable solutions across the engineering team.
• Support and mentor less-experienced engineers when appropriate.
• Demonstrate ownership, curiosity, collaboration, and a continuous-learning mindset.

Required Qualifications

·  Bachelor’s degree in Computer Science, Engineering, Information Systems, or a related discipline, or equivalent practical experience.

·  6+ years of experience in data engineering, integration engineering, software engineering, or related roles.

·  Strong SQL skills with experience in data modeling, database design, and performance optimization.

·  Hands-on experience designing and developing scalable ETL/ELT pipelines and data-processing solutions.

·  Experience with one or more modern cloud data platforms, such as Databricks, Microsoft Fabric, Snowflake, or comparable technologies.

·  Programming experience with Python, PySpark, Scala, Java, C#, or a comparable language.

·  Experience developing and consuming REST APIs and working with modern API-based architectures.

·  Experience with event-driven, messaging, or asynchronous integration patterns.

·  Experience with cloud-native technologies across Azure, AWS, GCP, or comparable environments.

·  Experience with source control, automated testing, CI/CD, monitoring, and modern DevOps practices.

·  Understanding of data quality, governance, security, lineage, and cloud identity concepts.

·  Experience supporting production data systems and troubleshooting complex technical issues.

·  Practical experience using AI-assisted development tools in day-to-day engineering.

·  Strong problem-solving, communication, and collaboration skills.

Preferred Qualifications

· Experience working across multiple modern data platforms or cloud technologies.

· Experience with Apache Spark, Delta Lake, Apache Iceberg, lakehouse architectures, or comparable technologies.

· Experience with GraphQL and enterprise API management platforms.

· Experience with streaming and messaging technologies such as Kafka, Event Hubs, Event Grid, Service Bus, or comparable technologies.

· Experience with serverless computing and cloud-native integration services.

· Experience with data products and data-as-a-product principles.

· Experience with modern data-sharing and interoperability patterns.

· Experience with Infrastructure as Code and deployment automation.

· Experience with AI-related data solutions such as RAG, semantic models, data agents, machine learning, or agent/tool integrations.

· Experience working with sensitive or highly governed data in financial services, legal, compliance, or regulatory environments.

· Familiarity with concepts such as KYC, UCC, public records, GRC, or SOC 2.

Key Skills

Modern Data Platforms | Databricks | Microsoft Fabric | Snowflake | Cloud Data Services | SQL | Python / PySpark | Apache Spark | Data Modeling | ETL/ELT | REST & GraphQL APIs | Event-Driven Architecture | Data Integration | Data Quality & Observability | Data Governance & Lineage | CI/CD & DevOps | AI-Assisted Engineering | AI Agents & RAG

Our Interview Practices

To maintain a fair and genuine hiring process, we kindly ask that all candidates participate in interviews without the assistance of AI tools or external prompts. Our interview process is designed to assess your individual skills, experiences, and communication style. We value authenticity and want to ensure we’re getting to know you—not a digital assistant. To help maintain this integrity, we ask to remove virtual backgrounds and include in-person interviews in our hiring process. Please note that use of AI-generated responses or third-party support during interviews will be grounds for disqualification from the recruitment process.

Applicants may be required to appear onsite at a Wolters Kluwer office as part of the recruitment process.