Senior Software Engineer - DevOps

JOB DETAILS

JOB BAND: D
CONTRACT TYPE: Permanent - Full Time
DEPARTMENT: Media Tech
LOCATION: London and UK Wide - Hybrid
PROPOSED SALARY RANGE: Up to £74,338 depending on relevant skills, knowledge, and experience. The expected salary range for this role reflects internal benchmarking and external market insights.

We're happy to discuss flexible working. If you'd like to, please indicate your preference in the application – though there's no obligation to do so now. Flexible working will be part of the discussion at offer stage.

PURPOSE OF THE ROLE

The BBC’s future depends on delivering seamless, 24/7 online services without disruptions. We build business-critical, highly available observability and alerting platforms for massive-scale performance tracking and corrective automation. We are migrating legacy influx systems to a modern open-source stack (Mimir, Prometheus, Alerta, Alloy, Grafana, LibreNMS) and integrating Loki and Tempo. As a Senior Engineer, you will architect, design, and develop these core cloud and network systems, underpinning our audience products and so directly preventing outages for millions of users.

WHY JOIN THE TEAM

Join an innovative, tight-knit team where your growth matters. You will collaborate across the BBC to shape our future observability roadmap, engineering critical high-scale services using AWS. This role offers total freedom to pitch ideas, innovate at a massive scale, and define our future enterprise monitoring ecosystem. Beyond delivering high-impact platforms, you will enjoy tailored learning opportunities to develop your skills alongside a supportive community of experienced engineers and our internal Academy ensuring currency in a rapidly evolving technical landscape.

YOUR KEY RESPONSIBILITIES AND IMPACT

• Architect and scale robust, highly available, and user-focused infrastructure using cloud services and automation as well as lead the design and development of software that meets complex product and business needs which includes integrating monitoring and support tools.
• Be able to think out of the box and proactively diagnose and resolve operational issues whether triggered by system alerts or raised by stakeholders, contribute to team rituals and participate in a supportive agile on-call rotation.
• Collaborate with stakeholders to gather requirements to ensure platforms scale effectively and meet business demands as well as product and operational teams to ensure services are suitable and supportable.
• Own engineering quality by driving continuous improvement in engineering practices and processes such as appropriate tests using automated test-driven techniques, daily best practices, code reviews, enhancing workflows and processes and solid SDLC processes bringing a focus to automation, infrastructure as code, and operations.
• Guide the team’s long-term technical strategy and plan future observability roadmaps.
• Mentor and support junior team members to accelerate their professional learning and growth in current and new technologies

YOUR SKILLS AND EXPERIENCE – KEY CRITERIA

ESSENTIAL CRITERIA

• While you do not need expertise in everything listed, expertise in some, familiarity with several and a willingness to learn is key:
o Tools and Services: Git / GitHub, Atlassian Tools, Telegraf, InfluxDB, Consul, LibreNMS, Docker, Kubernetes, Jenkins, Full stack: Database (Relational & Time series), Web / HTTP, backend
o Strong experience in Alerta, Grafana, Prometheus, Mimir, Alloy, Jenkins
o Infrastructure as Code: Terraform, Ansible, CloudFormation, AWS CDK
o Strong experience in Programming / Scripting Languages: Python, Bash/Shell.
o Strong production experience in AWS and Cloud: EC2, Load Balancing, Lambda, SQS, SNS, S3, RDS, CloudWatch, Route 53.
• Knowledge of system metrics and their impact on availability and performance.
• Experience with Linux Operating Systems (e.g. RedHat, CentOS, Rocky, Amazon Linux).
• Understanding of CI/CD practices and large-scale infrastructure design and support.

DESIRABLE CRITERIA

• Experience of high throughput metric ingestion at scale and distributed metrics storage / clustering
• Optional Network aspects such as HTTP, TLS, DNS, SNMP
• Working in an Agile development environment
• Enjoy collaborating in cross-functional teams and being comfortable asking questions, seeking feedback, and learning from others.

If you are experienced in some of these areas and eager to grow your expertise in others, we would love to hear from you!