Skip to main contentA logo with &quat;the muse&quat; in dark blue text.

Sr. Middleware Engineer - Kafka

AT Broadridge
Broadridge

Sr. Middleware Engineer - Kafka

London, United Kingdom

At Broadridge, we've built a culture where the highest goal is to empower others to accomplish more. If you're passionate about developing your career, while helping others along the way, come join the Broadridge team.

We are looking for a Sr. Middleware Engineerwho will be part of the design, deployment, and continuous improvement of our real-time data streaming solutions.

In this role, you will leverage your expertise with Apache Kafka, Confluent Cloud, and related technologies to architect scalable, resilient, and secure solutions that empower our business and clients.

Primary Responsibilities

Architecture & Design

  • Architect, design, and implement Kafka-based solutions using Confluent Cloud and Confluent Platform, ensuring they are highly scalable, resilient, and future-proof.
  • Provide technical leadership in designing event-driven architectures that integrate with on-prem systems and multiple cloud environments (AWS, Azure, or GCP).
Platform Management

Want more jobs like this?

Get jobs in London, United Kingdom delivered to your inbox every week.

By signing up, you agree to our Terms of Service & Privacy Policy.
  • Oversee administration and operational management of Confluent Platform components: Kafka brokers, Schema Registry, Kafka Connect, ksqlDB, and REST Proxy.
  • Develop and maintain Kafka producers, consumers, and streams applications to support real-time data streaming use cases.
Deployment & Automation
  • Lead deployments and configurations of Kafka topics, partitions, replication strategies in both on-prem and cloud setups.
  • Automate provisioning, deployment, and maintenance tasks with Terraform, Chef, Ansible, Jenkins, or similar CI/CD tools.
Monitoring & Troubleshooting
  • Implement robust monitoring, alerting, and observability frameworks using Splunk, Datadog, Prometheus, or similar tools for both Confluent Cloud and on-prem clusters.
  • Proactively troubleshoot Kafka clusters, diagnose performance issues, and conduct root cause analysis for complex, distributed environments.
Performance & Capacity Planning
  • Conduct capacity planning and performance tuning to optimize Kafka clusters; ensure they can handle current and future data volumes.
  • Define and maintain SLA/SLI metrics to track latency, throughput, and downtime.
Security & Compliance
  • Ensure secure configuration of all Kafka and Confluent components, implementing best practices for authentication (Kerberos/OAuth), encryption (SSL/TLS), and access control (RBAC).
  • Collaborate with InfoSec teams to stay compliant with internal and industry regulations (GDPR, SOC, PCI, etc.).
Cross-Functional Collaboration
  • Work with DevOps, Cloud, Application, and Infrastructure teams to define and align business requirements for data streaming solutions.
  • Provide guidance and support during platform upgrades, expansions, and new feature rollouts.
Continuous Improvement
  • Stay current with Confluent Platform releases and Kafka community innovations.
  • Drive continuous improvement by recommending new tools, frameworks, and processes to enhance reliability and developer productivity.
Qualifications

Technical Expertise
  • 5+ years of hands-on experience with Apache Kafka; at least 2+ years focused on Confluent Cloud and Confluent Platform.
  • Deep knowledge of Kafka Connect, Schema Registry, Control Center, ksqlDB, and other Confluent components.
  • Experience architecting and managing hybrid Kafka solutions in on-prem and cloud (AWS, Azure, GCP).
  • Advanced understanding of event-driven architecture and the real-time data integration ecosystem.
  • Strong programming/scripting skills (Java, Python, Scala) for Kafka-based application development and automation tasks.
DevOps & Automation
  • Hands-on experience with Infrastructure as Code (Terraform, CloudFormation) for Kafka resource management in both cloud and on-prem.
  • Familiarity with Chef, Ansible, or similar configuration management tools to automate deployments.
  • Skilled in CI/CD pipelines (e.g., Jenkins) and version control (Git) for distributed systems.
Monitoring & Reliability
  • Proven ability to monitor and troubleshoot large-scale, distributed Kafka environments using Splunk, Datadog, Prometheus, or similar tools.
  • Experience with performance tuning and incident management to minimize downtime and data loss.
Security & Compliance
  • Expertise in securing Kafka deployments, including Kerberos and SSL configurations.
  • Understanding of IAM best practices, network security, encryption, and governance in hybrid environments.
Leadership & Collaboration
  • Demonstrated experience leading platform upgrades, migrations, and architecture reviews.
  • Excellent communication skills, with ability to articulate complex technical concepts to diverse audiences (developers, architects, executives).
  • Comfortable collaborating with cross-functional teams-product owners, system engineers, security, and business stakeholders.
Education & Preferred Experience
  • Bachelor's or Master's degree in Computer Science, Information Systems, or related field (or equivalent experience).
  • Experience with container orchestration (Docker/Kubernetes) is a plus.
Hybrid Flexible at Broadridge

We are made up of high-performing teams that meet in person to learn and collaborate as needed. This role is considered hybrid, which means you'll be assigned to a Broadridge office and given the flexibility to work remotely.

#LI-Hybrid

#LI-LM1

We are dedicated to fostering a collaborative, engaging, and inclusive environment and are committed to providing a workplace that empowers associates to be authentic and bring their best to work. We believe that associates do their best when they feel safe, understood, and valued, and we work diligently and collaboratively to ensure Broadridge is a company-and ultimately a community-that recognizes and celebrates everyone's unique perspective.

Client-provided location(s): London, UK
Job ID: broadridge-JR1072024
Employment Type: Full Time

Perks and Benefits

  • Health and Wellness

    • Health Insurance
    • Dental Insurance
    • Vision Insurance
    • FSA
    • FSA With Employer Contribution
    • HSA
    • HSA With Employer Contribution
    • Life Insurance
    • Short-Term Disability
    • Long-Term Disability
    • Fitness Subsidies
    • Pet Insurance
    • Mental Health Benefits
  • Parental Benefits

    • Family Support Resources
    • Birth Parent or Maternity Leave
    • Non-Birth Parent or Paternity Leave
    • Adoption Leave
    • Fertility Benefits
  • Work Flexibility

    • Flexible Work Hours
    • Remote Work Opportunities
    • Hybrid Work Opportunities
    • Work-From-Home Stipend
  • Office Life and Perks

    • Commuter Benefits Program
    • Casual Dress
    • Happy Hours
    • Company Outings
    • On-Site Cafeteria
    • Holiday Events
  • Vacation and Time Off

    • Paid Vacation
    • Paid Holidays
    • Personal/Sick Days
    • Leave of Absence
    • Volunteer Time Off
    • Unlimited Paid Time Off
  • Financial and Retirement

    • 401(K)
    • 401(K) With Company Matching
    • Performance Bonus
    • Financial Counseling
    • Relocation Assistance
  • Professional Development

    • Tuition Reimbursement
    • Promote From Within
    • Mentor Program
    • Access to Online Courses
    • Lunch and Learns
    • Leadership Training Program
    • Internship Program
  • Diversity and Inclusion

    • Diversity, Equity, and Inclusion Program

Company Videos

Hear directly from employees about what it is like to work at Broadridge.