Senior Staff AI/ML System Software Engineer

d-Matrix Corporation

Job Overview

Location

Santa Clara

Job Type

Full-time

Full Job Description

📋 Description

• Design, develop, and maintain next-generation AI deployment software for d-Matrix’s proprietary AI compute engine, ensuring seamless integration between hardware and software layers.
• Collaborate with compiler experts to build and enhance compiler infrastructure optimized for custom AI hardware architectures.
• Work cross-functionally with hardware teams including mixed-signal, DSP, and CPU engineers to co-design and optimize system-level performance.
• Architect and implement distributed, high-performance software systems that enable efficient execution of deep learning workloads across heterogeneous computing environments.
• Develop and optimize ML inference pipelines, including model preprocessing, quantization, sparsity, and deployment workflows within MLOps frameworks.
• Integrate and extend open-source ML compiler frameworks such as MLIR to support novel AI compute primitives and hardware-specific optimizations.
• Deploy and scale ML models for computer vision (e.g., ResNet), natural language processing (e.g., BERT, GPT), and recommendation systems (e.g., DLRM) on distributed, multitenant infrastructure.
• Implement and tune deep learning runtimes including ONNX Runtime, TensorRT, and similar frameworks to maximize throughput and latency efficiency.
• Utilize inference server frameworks such as Triton, TensorFlow Serving (TFServ), and KubeFlow to serve models in production-grade environments.
• Design and deploy distributed systems collectives using NCCL and OpenMPI to enable efficient model parallelism and data communication across nodes.
• Build and maintain Linux-based software toolchains using C/C++ and Python, adhering to industry-standard development and debugging practices.
• Lead technical initiatives with ownership across the full-stack AI software toolchain, from model training to edge deployment, in fast-paced, high-pressure development cycles.
• Contribute to software documentation, code reviews, and mentorship of junior engineers to ensure high-quality, maintainable, and scalable codebases.
• Iterate rapidly on software deliverables under tight timelines while maintaining system reliability and performance targets.
• Engage in direct, collaborative communication with team members to solve complex problems and improve system architecture through shared accountability and humility.
• Participate in the end-to-end lifecycle of AI software products, from prototyping and validation to production rollout and ongoing optimization.
• Apply deep understanding of computer architecture, data structures, and machine learning fundamentals to make informed trade-offs in hardware-software co-design.
• Support the productization of AI software stacks for commercial deployment, ensuring alignment with customer use cases and operational requirements.
• Stay current with advancements in AI systems, compiler technologies, and distributed computing to continuously improve d-Matrix’s software platform.

🎯 Requirements

• BS in Computer Science, Engineering, Math, Physics or related field with 7+ years of industry software development experience; MS in same fields preferred with 5+ years
• Strong grasp of computer architecture, data structures, system software, and machine learning fundamentals
• Proficient in C/C++/Python development in Linux environment using standard development tools
• Experience with distributed, high-performance software design and implementation
• Self-motivated team player with strong sense of ownership and leadership
• Experience implementing SIMD algorithms on vector processors

🏖️ Benefits

• Hybrid work model with 3 days per week onsite at Santa Clara, CA headquarters
• Equal opportunity workplace with inclusive, collaborative culture
• Opportunity to work at the forefront of generative AI hardware-software innovation
• Direct communication environment valuing humility, dedication, and learning together

Skills & Technologies

Python

Linux

TensorFlow

PyTorch

Senior

Hybrid

Degree Required

Ready to Apply?

Apply Externally

You will be redirected to an external site to apply.

AI Job Fit Analysis

Pro

See exactly how your profile matches this role — strengths, skill gaps, and what to do about them.

d-Matrix Corporation

Visit Website

About d-Matrix Corporation

d-Matrix designs silicon for high-efficiency AI inference at scale. Its Corsair compute platform combines in-memory computing with a digital approach to slash latency and energy use in transformer and generative workloads. Targeting hyperscale data centers and edge deployments, the company offers hardware and software stacks that integrate into existing AI pipelines. Founded in 2019 and headquartered in Santa Clara, California, d-Matrix serves cloud and enterprise customers seeking cost-effective alternatives to GPUs for large language model serving.

View Company Profile

Get more remote jobs like this

Subscribe to the weekly newsletter for similar remote roles and curated hiring updates.

Weekly remote jobs and featured talent.

No spam. Only curated remote roles and product updates. You can unsubscribe anytime.