Data Pipeline Engineer Seattle
Build the core of Stripe products, services and APIs in spectacularly robust fashion.
Stripe sits at the heart of our merchant’s financial operations, these operations generate copious amounts of data that merchants need to analyze, summarize and report on. At Stripe, we built Sigma to help businesses quickly gain access to and analyze the data they create on the Stripe platform. Sigma has evolved from a web based query tool to a holistic data platform used by several products at Stripe. Many engineering teams at Stripe work to load data into Sigma and use it as a central source of truth to power dashboards, reports and insights. As the Stripe platform continues to expand its capabilities, the permutations of data insights Stripe can provide are rapidly increasing, and as such are becoming a critical component of business operations for our users.
- Work with Stripe engineers across multiple teams to design, build and evolve Sigma data flows
- Lead the design and building of new pipeline infrastructure components and primitives
- Provide mentorship and technical guidance to other Sigma engineers
- Engage directly with stakeholders across Stripe to help with planning and general technical direction
You may be a fit for this role if you:
Have extensive experience designing and building large scale data pipelines Are experienced in developing and maintaining distributed systems built with open source tools Enjoy optimizing the end to end performance of distributed systems Have experience in writing and debugging ETL jobs using a distributed data framework (Hadoop/Spark etc…)
- Have experience with Scalding or Spark
- Have experience with Airflow or other similar scheduling tools
- Worked with Presto or similar large scale distributed SQL query engine
- It’s not expected that you’ll have deep expertise in every dimension above, but you should be interested in learning any of the areas that are less familiar.
You might work on:
- A real-time system to model the dependency graph of Stripe’s entire data set
- Custom Presto query connectors
- Complex event driven DAG monitoring
What’s it like to work at Stripe?
Stripe is helping the internet fulfill its potential as a platform for economic progress by building software tools that accelerate global economic access and technological development. Stripe makes it easy to start, run and scale an internet business from anywhere in the world.
Stripe is, at its heart, an engineering company. To provide a missing pillar of core internet infrastructure, we hire people with a broad set of technical skills (and from a wide variety of backgrounds) who are ready to take on some of the most challenging problems in the industry – from reliably handling 100M API requests per day, to building adaptive machine learning as a result of years of data science and infrastructure work, and enabling entrepreneurs worldwide to start a global internet business. We look at Stripe as a constant work in progress and the same is true of our people; for all of us, we believe the best is yet to come. We’re here to support each other in our curiosity and creativity – which we pursue through thoughtful discussion and knowledge-sharing among a diverse set of peers and colleagues. We encourage all engineers to transition teams once every year and a half and also take on short-term projects with other teams across Stripe. This enables engineers to learn how different parts of Stripe work while also establishing stronger ties and cross-pollination between groups.
We contribute to existing open-source projects and the people working on them, and we release several tools as open-source. We want to work in a company of warm, inclusive people who treat their colleagues exceptionally well. The kind of people who are committed to going out of their way to help other Stripes in the short-term and pushing them to improve over the long-term (by helping them to get better at what they do).
We’re a highly cross-functional organization and view that as part of the fun: we design our space to encourage as much collaboration as possible. We have long tables in the kitchen for a reason (to enable everyone to meet new people and learn from them). We also have a culture of transparency that we carry through to email communication, ensuring that Stripes all around the world have the information they need to make good local decisions. In both our products and our people, we aim to reflect, represent and advocate for all of our users, globally. Our users transcend geography, culture and language; what we share, collectively, is a drive to create a fairer, more economically interconnected world.#LI-AH1
We look forward to hearing from you.