Skip to content

Amazon

Sr. Business Intelligence Engineer, AOP (Analytics Operations and Programs), AOP

Amazon is hiring a Sr. Business Intelligence Engineer, AOP (Analytics Operations and Programs), AOP in Bengaluru, India. Level rates it ; you can apply on Amazon.

Level's read on this job

  • On Level's scale, this role is Works on AI ●●●○: the daily work is on AI products, without building the model.
  • Tools and skills the posting cites include AI evaluation.
  • Level classifies it as a senior role.
  • Level lists 20,000+ other open roles at Amazon.
  • The role was posted in the last few days.

AI in this role

ai-evaluation
Amazon strives to be Earth's most customer-centric company where people can find and discover virtually anything they want to buy online. By giving customers more of what they want - low prices, vast selection, and convenience - Amazon continues to grow and evolve as a world-class e-commerce platform. The AOP team is an integral part of this and strives to provide Data and Science Capabilities to fulfill all customer processes in the APAC, LATAM, IN and MENA Ops businesses.

This BIE role sits at a rare intersection and has high growth prospects. On one side, you have cutting-edge ML/OR models making split-second cost saving decisions across Ops network. On the other, you have the analytical depth required to understand why those models succeed, where they fall short, and what needs to change. You won't just be observing these models from the outside, you'll own them along with the Scientists. Their performance, their enhancements, their evolution over time, that's your responsibility. When the model needs a new feature, a re-calibrated threshold, or a fundamentally different approach to a particular corridor or condition, you're the one driving that forward in partnership with the scientists who built it. In some instances you will also directly own these models.

The technical challenges here are genuinely complex and rewarding. You'll work with large-scale, low-latency data flowing through real-time pipelines. You'll design automated reporting frameworks that need to be as robust as the system they monitor. You'll apply statistical rigor to evaluate model performance in production, design experiments to validate enhancements, and quantify impact in an environment where conditions shift daily. But beyond monitoring, you'll be the one identifying what's broken, proposing what to build next, and owning the analytical path from hypothesis to deployed improvement. This is a role without a single well-worn toolkit, you'll have the opportunity to continuously sharpen your skills across data engineering, statistical analysis, and model evaluation to keep pace with the problems in front of you.

What makes this role particularly compelling is the combination of ownership and impact. The products you will work on drive millions in cost savings for Amazon Ops. You will also be involved in cutting-edge research. You'll need to be comfortable with ambiguity. Many of the most important questions in this space haven't been cleanly articulated yet, they emerge from patterns in the data, from model behavior that doesn't match expectations, from edge cases that only surface under certain network conditions. You'll need strong statistical foundations, fluency in SQL and Python and AWS Services, and experience building scalable BI solutions. Your ability to move between deep technical analysis and clear stakeholder communication, and to own outcomes end-to-end rather than just deliver reports, is what will make you exceptional in this role.

Key job responsibilities
1. Own End-to-End Analytics: Independently design and deliver complex analytical solutions for ML/OR science products, translating ambiguous business requirements into scalable, actionable insights.
2. Drive Science Model Performance: Develop frameworks to monitor model accuracy, operational effectiveness, and business impact. Identify performance gaps, investigate root causes, and drive improvements.
3. Lead Complex Deep Dives: Analyze large-scale operational datasets, identify trends and anomalies, and provide data-driven recommendations that influence product and science decisions.
4. Build Scalable BI Solutions: Design and develop robust data models, ETL pipelines, automated reporting systems, and self-service dashboards using SQL, Python, AWS, and BI tools.
5. Partner on ML/OR Enhancements: Collaborate with scientists to evaluate new features, optimize model thresholds, design experiments, and validate improvements. Directly own analytical or statistical models where appropriate.
6. Establish Measurement Frameworks: Define KPIs, apply statistical methodologies, and develop rigorous approaches to measure model effectiveness, incremental impact, and financial benefits.
7. Influence Cross-Functional Stakeholders: Work with Operations, Product, Engineering, and Science teams across geographies to align analytical priorities and drive adoption of data-driven solutions.
8. Drive Technical Excellence: Establish best practices for data quality, analytical accuracy, automation, and solution scalability while improving existing BI processes and frameworks.
9. Provide Technical Guidance: Mentor junior BIEs, review analytical approaches, share best practices, and contribute to raising the technical bar across the team.
10. Deliver Business Impact: Develop mechanisms to quantify and track cost savings, productivity improvements, and operational efficiency resulting from science products.

About the team
The AOP Science Team is the central science team of Amazon APAC, LATAM, IN and MENA Operations. We are responsible for creating core science capabilities to scale decision making across domains such as Truck Capacity Optimization, Short-term capacity planning, Network Abuse Detection, Impact analyses, Productivity improvement Models and so on. As a team our goal is to support our customers to make better decisions by leveraging applied statistics, machine learning, OR, deep learning architecture and algorithms. Team owns end-to-end life cycle for science model development which includes data exploration, experimentation, automating near real-time data pipelines, data visualization and model productionization.

Basic qualifications

- 10+ years of professional or military experience
- 5+ years of SQL, ETL or Oracle experience
- Experience programming to extract, transform and clean large (multi-TB) data sets
- Experience with theory and practice of design of experiments and statistical analysis of results
- Experience with AWS technologies
- Experience in scripting for automation (e.g. Python) and advanced SQL skills.
- Experience with theory and practice of information retrieval, data science, machine learning and data mining

Preferred qualifications

- Experience working directly with business stakeholders to translate between data and business needs
- Experience managing, analyzing and communicating results to senior leadership

Our inclusive culture empowers Amazonians to deliver the best results for our customers. If you have a disability and need a workplace accommodation or adjustment during the application and hiring process, including support for the interview or onboarding process, please visit https://amazon.jobs/content/en/how-we-hire/accommodations for more information. If the country/region you’re applying in isn’t listed, please contact your Recruiting Partner.

How we rate this

Sr. Business Intelligence Engineer, AOP (Analytics Operations and Programs), AOP at Amazon rates 65 out of 100 for how much of the daily work is AI. That makes it Works on AI (AI Level 3 of 4). The level is about AI in the job, not seniority.

Classification

Works on AI. The daily work is on AI products, without building the model.

  1. ●●●● Builds AI80 to 100
  2. ●●●○ Works on AI60 to 79
  3. ●●○○ Uses AI40 to 59
  4. ●○○○ Little AI0 to 39

Levels come from how often the tools, models and workflows of the role are named in the posting itself. Read the description and count the AI tools, models and workflows it names.

Prepare for this job

Built from this posting and from similar jobs on Level.

Skills and AI tools this role asks for

AI Evaluation

Questions you could be asked

  1. How do you decide that one model's output is better than another's for a given task?
  2. Describe a typical day in a role like this one: which parts run through AI directly?
  3. If you removed AI from this role, what would be left, and how do you decide what still needs a human?

Adapt your resume

  • List these exact terms on your resume: AI Evaluation. An applicant tracking system matches the wording, not the idea.
  • Attach one line of real, concrete experience to at least one of them. A tool named with nothing behind it rarely survives a human read.
  • Show where AI is part of your daily process, not a one-off project. This role expects it to be a running habit.

Get your CV reviewed before you apply

Send your CV and the role you want next. Your review arrives by email within 3 days.

Get new data jobs (Works on AI ●●●○ or higher) by email

One email a week, each job rated for how much AI is in the work. No recruiter spam, unsubscribe in one click.

Free.

Similar roles

Data roles that work on AI, at other companies.

What kind of AI work fits you?

Answer 12 practical questions in about three minutes. Get a simple profile, the work it points to, and live roles to explore next.

Find my next step

More jobs at Amazon

Related searches

Same AI level

Jobs by city