Mid System Engineer (Storage team)

📋 Description

• • Own and evolve Hostinger’s global storage backbone: 12 data-centres, tens of petabytes, 3 million customers. You will architect, deploy, and tune distributed storage clusters (CEPH, GlusterFS, and emerging open-source stacks) to guarantee 99.9 %+ availability and millisecond-level latency for e-commerce sites, SaaS apps, and media platforms.
• • Provision and scale storage capacity ahead of demand. You will spin up new CEPH OSDs, rebalance PGs, and integrate NVMe-oF or CephFS gateways while maintaining zero-downtime growth. Capacity planning is data-driven: you will model growth curves, forecast 6–12 months out, and automate elastic expansion so that new regions come online in hours, not weeks.
• • Design resilient, cost-efficient storage topologies. From erasure-coding policies to multi-site replication and geo-fencing, you will craft blueprints that survive rack, DC, or regional failures. You will benchmark throughput/IOPS, tune CRUSH maps, and select hardware tiers (SAS, SATA, NVMe, SMR) to hit price/performance targets set with Finance and Product teams.
• • Automate everything. Ansible playbooks, Terraform modules, and CI/CD pipelines will be your daily language. You will codify server provisioning, OSD replacement, pool resizing, and secret rotation so that every change is repeatable, peer-reviewed, and rolled back in minutes. Expect to contribute upstream patches when gaps appear.
• • Monitor, alert, and observe. You will extend Prometheus, Grafana, Loki, and Alertmanager to surface cluster health, RGW latency, RBD queue depth, and pool utilisation. SLIs/SLOs (durability, availability, recovery time) will be defined, tracked, and continuously improved. On-call rotation is lightweight thanks to the automation you build.
• • Troubleshoot at scale. When a PG goes inconsistent or a client reports slow reads, you will dive into ceph health detail, perf counters, and tcpdump traces to pinpoint root causes. Post-mortems are blameless and feed directly into runbook and code improvements.
• • Collaborate across tribes. Storage underpins Shared Hosting, Cloud, VPS, and Kubernetes products. You will consult with SRE, Backend, and Data teams to expose block, object, or file storage that meets their latency, throughput, and security requirements. You will also mentor junior engineers and run internal workshops on Ceph internals.
• • Drive innovation. Evaluate emerging tech—Crimson OSD, RGW multisite sync, CephFS snapshots, or alternative engines like SeaweedFS—and run proof-of-concepts in our lab. Successful experiments graduate to production and are shared at meetups or conferences.
• • Champion security & compliance. You will enforce encryption at rest, in-flight TLS, key rotation, and GDPR data-locality rules. Regular audits and chaos-engineering drills ensure we can restore any dataset within agreed RPO/RTO.
• • Continuous learning is in our DNA. Expect dedicated time each sprint for deep-dives, conference attendance (e.g., Cephalocon, SREcon), and internal guild sessions. Your growth path can lead to Senior System Engineer, Staff Engineer, or Product-focused Storage Architect roles.