Principal Engineer

Nexus Cognitive LLC

Job Overview

Location

Atlanta HQ

Job Type

Full-time

Full Job Description

📋 Description

• The Principal Engineer is the highest individual contributor on the NX1 platform engineering team, holding architectural authority and serving as the primary technical decision-maker below the Head of Engineering, responsible for defining how core platform components are built, integrated, and evolved.
• Day-to-day responsibilities include owning the architecture of NX1's core platform (Spark, Trino, Iceberg, Ranger, Gravitino, Keycloak), leading multi-tenant security model design, driving Iceberg catalog strategy, authoring technical design documents, owning integration posture with upstream open-source projects, leading technical investigations of complex L3 escalations, defining Kubernetes operator strategy, mentoring senior engineers, partnering on technical roadmap planning, and interfacing directly with enterprise customers on complex architectural questions.
• Nexus Cognitive LLC is a team of builders and problem-solvers focused on turning complex data into actionable insight, helping clients accelerate innovation by managing the heavy lifting of modern data platforms, governance, and cloud transformation across U.S. and India operations, with a culture rooted in curiosity, ownership, collaboration, and care.
• In this role, the individual will deepen expertise in enterprise-scale distributed data systems, shape the technical direction of a strategic platform, lead open-source integrations and version migrations, mentor engineering talent, and gain direct influence over enterprise customer architecture decisions while working at the forefront of cloud, data, and AI innovation.

🎯 Requirements

• 10+ years of software engineering experience with at least 3 years at staff or principal engineer level on distributed data systems
• Expert-level knowledge of Apache Spark, including Spark on Kubernetes, Spark SQL engine, custom extensions, and performance tuning at scale
• Deep experience with Trino (formerly PrestoSQL): connector architecture, query planning, cost-based optimization, and integration with Hive Metastore or Iceberg REST catalog
• Hands-on production experience with Apache Iceberg: table format internals, partitioning strategies, compaction, snapshot expiry, and multi-engine compatibility (Spark + Trino)
• Strong experience with enterprise authentication and authorization: Keycloak (OIDC/JWT), Apache Ranger (policy engine, service definitions, plugins), and LDAP/Active Directory integration
• Experience with Apache Gravitino or comparable metadata/catalog platforms: catalog API design, Iceberg REST catalog implementation, and credential vending patterns
• Kubernetes and OpenShift expertise: operators, CRDs, RBAC, network policies, and multi-tenant namespace management; experience with YuniKorn or equivalent schedulers is a strong plus
• Proficiency in Java and/or Scala for Spark/Trino work; Python for tooling, automation, and data engineering integrations
• Experience with Helm chart authoring, Terraform module development, and GitOps-based platform lifecycle management
• Track record of leading engineering teams through complex open-source integrations, major version migrations, and enterprise customer deployments

🏖️ Benefits

• Collaborative team culture built on curiosity and respect
• Challenging work where contributions clearly matter
• Leadership team that invests in learning and development
• Opportunity to work at the intersection of cloud, data, and AI innovation

Skills & Technologies

Python

Java

Scala

Kubernetes

Terraform

Senior

Onsite

Ready to Apply?

Apply Externally

You will be redirected to an external site to apply.

Nexus Cognitive LLC

Visit Website

About Nexus Cognitive LLC

Nexus Cognitive LLC is a Delaware-registered data and AI consultancy that designs cloud-native data platforms, implements real-time analytics, and delivers machine-learning solutions for Fortune 1000 enterprises. The company focuses on modernizing legacy data estates through Microsoft Azure, Databricks, and open-source technologies, offering advisory, implementation, and managed services across retail, healthcare, and financial services sectors. Founded in 2018, it operates with distributed teams across the United States and maintains Microsoft Solutions Partner status for Data & AI and Digital & App Innovation.

View Company Profile

Get more remote jobs like this

Subscribe to the weekly newsletter for similar remote roles and curated hiring updates.

Weekly remote jobs and featured talent.

No spam. Only curated remote roles and product updates. You can unsubscribe anytime.