Senior Distributed Systems Engineer, Multi-Chip Simulation

Role Overview

Running one AI model across many chips is a distributed systems problem: a host and many chips exchanging data over links, waiting on each other, and failing in ways no single chip would. As a Senior Distributed Systems Engineer, you will help us simulate that system before the hardware exists. You will work with the chip modeling, SDK, and runtime teams to build the simulators that stand in for the host, the links, and the protocols between chips.

This role is about building and scaling simulation infrastructure, not testing someone else's code. You will own the software architecture of the multi-chip simulation stack, decide how much fidelity each layer needs, and keep it fast enough that distributed software is exercised every day in CI.

Key Responsibilities

Multi-Chip Simulator Design

• Design and build the simulators for the host-to-chip and chip-to-chip layers of our system, including the drivers, links, protocols, and transfers that connect many chips into one machine.
• Model the communication fabric, including transfer timing, collectives, ordering guarantees, back-pressure, congestion, and partial failure, calibrated against real hardware.

Architecture and Scale

• Own the architecture of the multi-chip simulation stack as new chips, interconnects, and deployment topologies arrive.
• Compose simulators of different fidelity (fast functional, cycle-level, and hardware emulation) into one coherent multi-chip simulation.
• Scale the simulation from a few chips to full racks while keeping runs fast enough for daily use in CI.

Correlation and Quality

• Correlate simulated behavior against emulation and hardware measurements, with regression coverage and clear documentation of assumptions and limitations.

Required Qualifications

• 5+ years of experience as a distributed systems engineer building and operating software that coordinates many machines or accelerators through RPC, message passing, consensus, or collective communication.
• A strong understanding of the failure modes of distributed software, including races, partial failure, ordering and consistency, timeouts and retries, back-pressure, and deadlock.
• Demonstrated experience debugging timing-dependent bugs across hosts, NICs, or accelerators using traces and logs.
• Experience building simulators, emulators, or test harnesses for distributed or networked systems is strongly preferred.
• Sound judgment in choosing the right level of fidelity for a question and balancing accuracy against run time.
• Strong Rust skills, or expert-level C/C++ skills and a willingness to work in Rust day to day, along with proficiency in Python for tooling and analysis.
• Excellent communication skills, with the ability to explain what the simulation shows, what it does not, and what the software or hardware team should change.

Preferred Qualifications

• Hands-on experience with multi-GPU or multi-accelerator inference or training, including NCCL-style collectives, tensor, pipeline, and expert parallelism, and host-device runtimes.
• Familiarity with hardware emulators such as Cadence Palladium or Synopsys ZeBu, or with FPGA prototyping.

Leveling & Scope

While this role is currently posted at a specific level, we are a growth-oriented organization and are open to hiring at a more senior level for the right candidate. Please note that this job description serves as a focused but generalized overview of the role; specific responsibilities and impact expectations will be tailored to the experience and seniority of the final hire.

Why Join Us?

• You will own the architecture of the simulation stack that lets our distributed software run across racks of chips long before the hardware exists.
• You will work across chip modeling, SDK, and runtime teams on one of the hardest problems in AI inference: making many chips behave as one machine.

Compensation & Benefits

The base salary range for this role is $220,000 – $270,000.

Please note that the figures provided represent the base salary range only and do not include other elements of our total compensation package, equity, or comprehensive benefits.

At Positron AI, we value the unique expertise each candidate brings. While the range above reflects our typical expectation for the position, we reserve the flexibility to exceed this range for candidates whose specialized skills, significant experience, or unique qualifications fall outside the standard scope of the role. Final offers are determined based on a variety of factors, including internal equity, and individual impact.

Benefits & Perks

We want you to do your best work and feel confident that you and your family are taken care of. That means comprehensive coverage, real time to rest, and support for your future.

Health and wellness

• Fully company-paid medical, dental, and vision insurance for you and your dependents
• Company-paid life and disability coverage, with voluntary options to add more
• Supplemental hospital, critical illness, and accident coverage available

Time off and flexibility

• Unlimited paid time off; we encourage everyone to truly unplug and recharge
• 13 paid company holidays
• Remote-first culture with a company-provided computer and home office setup

Compensation and future

• Competitive salary and equity
• 401(k) with company matching, eligible from day one

Visa Support

This position is open to candidates currently authorized to work in the U.S. We cannot provide new visa sponsorship for this role but are open to facilitating H-1B visa transfers for eligible candidates.

Equal Opportunity Employer. If you're excited about the role but don't meet every bullet, we'd still love to hear from you.