Featured

Rust Engineer

LiteLLM·Senior·On-site · San Francisco·$200k–$260k USD

Posted Jul 26, 2026·litellm.ai

Overview

LiteLLM is an AI Gateway used by companies such as Adobe, Netflix, and NASA. The platform provides access to LLMs and adjacent services. The role involves owning the migration of LiteLLM from Python to Rust to build a low-latency, high-throughput AI gateway.

About The Role

The position owns LiteLLM's migration to Rust end to end. Responsibilities include planning the migration alongside the CTO, structuring the codebase for Rust, and executing a gradual, stability-first rollout. The objective focuses on performance, latency reduction, and throughput optimization at high scale without compromising platform reliability.

This is a single-owner mandate to lead the complete Python-to-Rust migration.

Responsibilities

Own the end-to-end migration of LiteLLM from Python to Rust, from architecture through rollout.
Plan the migration alongside the CTO, sequencing work to minimize risk to live, high-volume traffic.
Design and execute a gradual rollout that holds stability as the top constraint while the migration progresses.
Restructure the existing codebase so it is positioned for a clean, incremental move to Rust.
Drive performance gains across latency, throughput, and resource efficiency at 1B+ requests/month.
Collaborate cross-functionally to ensure the migration aligns with backend priorities across performance, reliability, and security.

Requirements

Deep expertise in writing production Rust.
Focus on latency, throughput, and performance optimization.
Track record of balancing architectural change against system reliability.
Experience with backend infrastructure, databases (PostgreSQL, Redis), and high-volume production systems.
Bonus: Prior experience leading codebase migrations from another language to Rust.
Bonus: Demonstrated history of shipping performance-critical systems.

All Rust jobs.

Senior Embedded C++/ Rust Software Engineer (Safety Critical)

Principal Rust Developer

Rust Systems Engineer

Staff Backend Engineer Rust

Rust Engineer

Staff Backend Engineer (Rust)

Rust Engineer

Overview

About The Role

Responsibilities

Requirements

Rust Engineer

Overview

About The Role

Responsibilities

Requirements

Senior Embedded C++/ Rust Software Engineer (Safety Critical)

Principal Rust Developer

Rust Systems Engineer

Staff Backend Engineer Rust

Rust Engineer

Staff Backend Engineer (Rust)

Rust Engineer

Overview

About The Role

Responsibilities

Requirements

Rust Engineer

Overview

About The Role

Responsibilities

Requirements