Senior Software Engineer (big data)

📋 Description

• • As a Senior Software Engineer (big data) at Bitsight Technologies, you will play a pivotal role in advancing the company’s mission to make the online world safer by designing, building, and scaling the data pipelines that power its industry-leading Security Ratings product. Your work will directly impact how organizations understand and mitigate cybersecurity risks by enabling the ingestion, processing, and analysis of terabytes of security intelligence data from global sources, ultimately delivering daily ratings for over 1,000,000 organizations.
• • You will architect and evolve Bitsight’s core data pipeline on AWS, ensuring it remains highly scalable, reliable, and performant as data volumes grow. This involves integrating new data sources, refining data transformation logic, optimizing query performance, and collaborating closely with data science and research teams to surface actionable insights that enhance the accuracy and responsiveness of cyber risk ratings.
• • Day to day, you will write clean, maintainable Java code to develop and operate large-scale data pipelines, leveraging modern software engineering practices including continuous integration and delivery (CI/CD), containerization with Kubernetes, and infrastructure-as-code principles. You will participate in code reviews, contribute to technical design discussions, and help establish best practices for data quality, monitoring, and observability across the pipeline.
• • You will work with big data technologies such as Apache Spark, Hadoop, and emerging data lakehouse solutions like Apache Iceberg to process and store massive datasets efficiently. Additionally, you will engage with search and analytics engines such as Elasticsearch or OpenSearch and OLAP systems like StarRocks to enable fast, interactive querying of security data for both internal teams and customer-facing applications.
• • Collaborating within a distributed, cross-functional team spanning Boston, Raleigh, New York, Lisbon, Singapore, and remote locations, you will partner with product owners, designers, and fellow engineers to translate business requirements into technical solutions. Your strong communication skills will be essential in documenting complex systems, sharing knowledge, and fostering a culture of transparency and inclusion.
• • You will contribute to the continuous improvement of Bitsight’s data infrastructure by evaluating and adopting new tools and technologies that improve scalability, reduce latency, and increase the value derived from security data. This includes experimenting with incremental processing frameworks and optimizing data models to support evolving rating algorithms.
• • About the team and company: Bitsight is a cyber risk management leader with over a decade of innovation, having invented the cyber ratings industry in 2011. Trusted by over 3,000 customers and supported by more than 750 teammates globally, the company blends the agility of a startup with the stability of a mature industry leader. Engineering teams operate with a strong emphasis on collaboration, technical excellence, and mission-driven impact.
• • In this role, you will have the opportunity to deepen your expertise in large-scale data systems while making a tangible difference in global cybersecurity. You will grow as a technical leader by influencing architectural decisions, mentoring peers, and contributing to a culture of belonging and inclusion where diverse perspectives are valued and innovation thrives.