Backend Engineer (API)
Summary
the pitchJoin our team as a Backend Engineer and play a crucial role in building the xAI API that serves our models to developers worldwide. You will own the end-to-end system responsible for high-throughput inference, handling billions of tokens per minute with low latency and high availability. This includes model serving infrastructure, request routing, SDK development, rate limiting, observability, and efficient scaling. As a Backend Engineer, you will benefit from comprehensive health insurance, life and family benefits, flexible vacation, visa sponsorship, and a 401(k) plan. Ideal candidates should have a good understanding of highly scalable and reliable production infrastructure, experience in designing and maintaining distributed systems, and expertise in Rust or C++.
Role
posted by company- As an ideal candidate you have a good understanding of how highly scalable and reliable production infrastructure is built
- Most of our backend infrastructure is written in Rust. So familiarity with a compiled language such as C++, Rust, or Go is highly beneficial
- Experience in designing, implementing, and maintaining reliable and horizontally scalable distributed systems
- Experience in operating commonly used databases such as PostgreSQL, Clickhouse, and MongoDB
- Expert knowledge of either Rust or C++
- Knowledge of service observability and reliability best practices
- Experience with LLM inference engines and serving frameworks (e.g., SGLang, TensorRT, vLLM)
- Experience designing or building with agent SDKs and agent orchestration frameworks
- Experience with Docker, Kubernetes, and containerized applications
- Expert knowledge of gRPC (unary, response streaming, bi-directional streaming, REST mapping)