Software Engineer - API Gateway

📋 Description

• • Own and evolve the beating heart of Featherless.ai – the API gateway that routes every inference request from thousands of AI builders, startups, and Fortune-500 enterprises. Your code will authenticate users, enforce subscription entitlements, and expose a clean, lightning-fast surface for text, image, audio, and multi-modal models.
• • Ship daily improvements that directly impact GPU utilization and end-user latency. You’ll add new endpoints as fast as open-source models drop, patch regressions before customers notice, and instrument every hop so we can prove “done” with real usage data.
• • Triage and crush gnarly cross-stack bugs that span DNS, Kubernetes ingress, Node.js, Redis, MongoDB, Python micro-services, and Cloudflare edge rules. When a 3 A.M. pager fires because a new 70 B parameter model suddenly spikes tail-latency, you’ll be the calm engineer who traces it to a mis-configured context-length limiter and rolls a fix within the hour.
• • Design and harden subscription management logic that turns feature flags into real-world guardrails: context windows, rate limits, and concurrency caps must adapt in real time as customers upgrade, downgrade, or burst during product launches.
• • Build rock-solid observability: wire OpenTelemetry traces from Fastify handlers through Redis queues to GPU nodes, surface golden signals in Elastic Cloud, and create Sentry alerts that page only when humans need to act.
• • Right-size infrastructure so that autoscaling GPU pools spin up fast enough for viral launches but never waste precious H100 hours. You’ll pair with the infra team to tune Kubernetes HPA thresholds, warm-pool sizes, and Cloudflare cache rules.
• • Collaborate in a tight-knit, remote-first Platform Team that swarms pull requests, debates design in Notion, and celebrates weekly demos. We bias to action: if a fix unblocks a customer today, we ship it and measure tomorrow.
• • Champion the developer experience: write clear OpenAPI specs, maintain Postman collections, and cut SDK snippets so external builders can integrate a new LLM in under five minutes.
• • Influence product direction by synthesizing support tickets, Discord chatter, and usage dashboards into crisp proposals. When the community begs for vision-language model support, you’ll scope the gateway changes, break them into two-week bets, and deliver.
• • Mentor junior engineers through pair programming and thoughtful code reviews, raising the bar for testing, documentation, and operational rigor across the entire stack.
• • Stay curious about the AI ecosystem: run open LLMs locally, experiment with LangChain, replicate a trending tweet-to-video pipeline, then bring that empathy back to the team so we never lose sight of the builders we serve.