State StreetData Engineering Developer - Assistant Manager
State Street is hiring a Data Engineering Developer - Assistant Manager in Bengaluru, India. Level rates it ; you can apply on Level.
AI in this role
Role Summary
We are seeking a highly skilled Data Engineer with strong expertise in Databricks, Apache Spark, and Scala to design, develop, and optimize large-scale data processing solutions. The ideal candidate will have hands-on experience building distributed data pipelines, implementing scalable ETL frameworks, and delivering cloud-native data solutions on Azure.
Key Responsibilities
Design, develop, and maintain scalable data pipelines using Databricks, Apache Spark, and Scala.
Build ingestion, transformation, and validation frameworks for high-volume data processing.
Develop and optimize Spark jobs for performance, scalability, and reliability.
Implement and manage Databricks workflows, job orchestration, and scheduling.
Troubleshoot production issues, performance bottlenecks, and pipeline failures.
Optimize cluster utilization, Spark execution plans, and data processing efficiency.
Work with cross-functional teams to design data architectures and integration solutions.
Implement best practices for coding, testing, CI/CD, and deployment.
Support production environments and participate in incident resolution and root cause analysis.
Collaborate in Agile teams to deliver high-quality data products.
Required Skills
Strong hands-on experience in:
Databricks
Apache Spark
Scala
Spark SQL
PySpark
Experience in:
Distributed data processing frameworks
ETL/ELT pipeline development
Data modeling and data warehousing concepts
Performance tuning and optimization of Spark applications
Cloud experience:
Microsoft Azure
Azure Data Lake Storage (ADLS)
Azure Data Factory (ADF)
Strong SQL development and query optimization skills.
Experience with source control systems such as Git/GitHub.
Strong analytical and debugging skills.
Preferred Skills
Delta Lake
Unity Catalog
CI/CD pipelines (Azure DevOps, Harness)
Snowflake
Data Quality and Validation Frameworks
Financial Services / Reference Data domain knowledge
Experience with AI-assisted development tools such as GitHub Copilot
Education & Experience
Bachelor's or Master's degree in Computer Science, Engineering, or related field.
10+ years of experience in Data Engineering.
6+ years of hands-on Databricks and Spark development experience.
Strong experience developing production-grade Scala applications.
Nice-to-Have
Databricks Certification
Azure Certification
Experience with Real-Time Streaming (Kafka, Spark Structured Streaming)
Exposure to Lakehouse architecture patterns
About State Street
Across the globe, institutional investors rely on us to help them manage risk, respond to challenges, and drive performance and profitability. We keep our clients at the heart of everything we do, and smart, engaged employees are essential to our continued success.
We are committed to fostering an environment where every employee feels valued and empowered to reach their full potential. As an essential partner in our shared success, you’ll benefit from inclusive development opportunities, flexible work-life support, paid volunteer days, and vibrant employee networks that keep you connected to what matters most. Join us in shaping the future.
As an Equal Opportunity Employer, we consider all qualified applicants for all positions without regard to race, creed, color, religion, national origin, ancestry, ethnicity, age, disability, genetic information, sex, sexual orientation, gender identity or expression, citizenship, marital status, domestic partnership or civil union status, familial status, military and veteran status, and other characteristics protected by applicable law.
Discover more information on jobs at StateStreet.com/careers
Read our CEO Statement
How we rate this
Data Engineering Developer - Assistant Manager at State Street rates 46 out of 100 for how much of the daily work is AI. That makes it Uses AI (AI Level 2 of 4). The level is about AI in the job, not seniority.
Uses AI. An ordinary role that requires AI tools.
- ●●●● Builds AI80 to 100
- ●●●○ Works on AI60 to 79
- ●●○○ Uses AI40 to 59
- ●○○○ Little AI0 to 39
Levels come from how often the tools, models and workflows of the role are named in the posting itself. Open the description and count.
Prepare for this job
Built from this posting and from similar jobs on Level.
Skills and AI tools this role asks for
Also common in data engineer jobs at this level
Pay
- Typical data engineer job
- $147k
This posting doesn't list pay. Use the typical data engineer figure when pay comes up. See data engineer jobs
Questions you could be asked
- What's a project where you used Copilot hands-on?
- Walk me through how you've used Databricks in your day-to-day work.
- This role expects you to use AI tools as part of the job. Which ones have you used, and for what?
- Tell me about a time an AI tool got something wrong. How did you catch it?
Adapt your resume
- List these exact terms on your resume: Copilot and Databricks. An applicant tracking system matches the wording, not the idea.
- Attach one line of real, concrete experience to at least one of them. A tool named with nothing behind it rarely survives a human read.
- Put the AI tool in a bullet point about what you did, not just in a skills list. This role treats it as a required part of the job.
Want an expert to read your CV for this job?
Free. Send your CV and the role you want next. We reply by email within 2 to 4 business days.
Get new data engineer jobs (Uses AI ●●○○ or higher) by email
One email a week with the new data engineer jobs (Uses AI ●●○○ or higher), each rated for how much AI is in the work. No recruiter spam, unsubscribe in one click.
Free. One email a week. Unsubscribe in one click.
Similar roles
Software Engineering roles that use AI, at other companies.
What kind of AI work fits you?
Answer 12 practical questions in about three minutes. Get a simple profile, the work it points to, and live roles to explore next.
Find my next step