The Wikimedia Foundation is looking for a Senior Site Reliability Engineer to support and develop the platform serving the world’s favorite encyclopedia, Wikipedia, to millions of people around the globe. Wikimedia’s Site Reliability Engineering (SRE) team is principally responsible for ensuring our global top-10 website and its underlying infrastructure is healthy and developing further in support of Wikimedia’s mission: to help everyone share in the sum of all knowledge.
The SRE team at Wikimedia is a globally distributed and diverse team of engineers with a drive to explore, experiment, and embrace new technologies. We work in the open by publishing all of our documentation, code, and configuration as open source, and all our production systems are powered by open source software.
You are responsible for:
- Performing day-to-day operational/DevOps tasks on Wikimedia’s public facing infrastructure (deployment, maintenance, configuration, troubleshooting)
- Implementing and utilizing configuration management and deployment tools (Puppet, Kubernetes)
- Leading continuous improvement, by automating the installation, configuration and maintenance of services on our platform
- Working closely with product teams helping them bring scalable functionality to our users by assisting in the architectural design of new services and making them operate at scale
- Participating in a 24/7 on-call rotation shared across the broader SRE team. This includes taking part in incident response, diagnosis and follow-up on system outages or alerts across Wikimedia’s production infrastructure.
- Collaborating with a global, cross-functional team in an asynchronous communication environment
- Mentoring peers in your areas of technical and operational strength
About the Wikimedia Foundation
The Wikimedia Foundation is the nonprofit organization that operates Wikipedia and the other Wikimedia free knowledge projects. Our vision is a world in which every single human can freely share in the sum of all knowledge.
The Wikimedia Foundation is a remote-first organization. Salaries at the Wikimedia Foundation are set in a way that is competitive, equitable, and consistent with our values and culture. The anticipated annual pay range of this position for applicants based within the United States is US$ 116,633 to US$ 181,243 with multiple individualized factors, including cost of living in the location, being the determinants of the offered pay. For applicants located outside of the US, the pay range will be adjusted to the country of hire. U.S. Benefits & Perks
Skills and Experience:
- 6+ years experience in an SRE/Operations/DevOps role as part of a team
- Experience with shell and any scripting language used in an SRE context (Python, Go, Bash, Ruby; we primarily use Python) and configuration management tools (Puppet, Ansible; we use Puppet)
- Experience with distributed caching systems: including their underlying algorithms and how to optimize their performance
- Experience with package management on Linux systems (we use Debian)
- Strong Linux system-level troubleshooting skills
- History of automating tasks and processes, identifying process gaps, and finding automation opportunities
- Strong English language skills (verbal and written) and ability to work independently, as an effective part of a globally distributed team working across multiple time zones
- Experience leading and participating in incident response and post-incident review rituals, with the goal of conducting root cause analysis and implementing preventive measures
Additionally, we’d love it if you have:
- Experience with Linux kernel tuning
- Experience with the use, maintenance and configuration of monitoring, metrics and logging infrastructure (Prometheus, Grafana, etc.)
- Developing/contributing to Free and Open Source software, or being part of an open-source community
- Experience with LAMP stack technologies (PHP/HHVM, memcached/Redis) -- MediaWiki experience is a definite plus
- Experience with defining cross-team SLOs and their implementation
- Experience operating an on-premise filesystem or object store at scale, preferably OpenStack Swift or Ceph
- Experience with other advanced distributed storage and database systems (Cassandra, MariaDB etc.)
- Experience managing backups as an SRE with some experience with Bacula
