We are developing a SaaS product that simplifies financial planning and analysis of cloud billing data for large enterprises with complex cloud spending requirements. We're looking for a data engineer to wrangle complex cloud billing data by designing the pipelines that power our product.
We have fascinating technical challenges around data modeling and continuous quality control. We're analyzing massive amounts of semistructured data at scale: processing cloud bills with constantly evolving schemas—complexity that only increases as we expand functionality and provider support. On the frontend, customers use the data to drive large financial decisions, so full data product ownership and quality is key.
Build and maintain ETL pipelines processing hundreds of millions of rows of cloud billing data
Work with ClickHouse, Parquet files, and S3 to design efficient data storage and retrieval systems
Develop data validation and quality control systems using Python and SQL
Design data models for complex, evolving cloud billing schemas (AWS CUR and beyond)
Build and optimize Airflow workflows for reliable data processing
Collaborate with the entire engineering team to investigate and resolve data quality issues
Scale data infrastructure as we expand to new cloud providers and use cases
3+ years experience with data products: warehouses/lakehouses/OLAPs, ETL pipelines, or job queues
Software engineering experience, with intermediate Python experience
Strong SQL skills including CTEs, window functions, and query optimization
Experience with data validation and quality control systems
Comfortable with columnar databases, Parquet, and cloud storage (S3)
Ability to deliver results in hours instead of days
Some experience in a startup environment, or ability to work well in a startup environment
Fastidiousness about data quality and comfort when there's no answer key
Experience with ClickHouse or other OLAP datastores
Past experience with cost management tools and/or cloud billing data
Experience with Airflow or similar workflow orchestration tools
Backend engineering experience beyond data pipelines
You'll tackle genuinely complex technical challenges at scale while building expertise in cloud financial data—a rapidly growing and specialized field. You'll have end-to-end ownership of data products that directly impact customer success, working in a small team where your contributions immediately matter.
We are a small and growing team (less than 10 people!), which means you get the opportunity to be on the ground floor of building the product and company. Our founders are the founders of The Duckbill Group, who bring their wealth of domain expertise and deep industry and customer connections in cloud cost management to the product. We're currently in a semi-stealth mode while we're focusing on building the initial product.
We work together in the office in San Francisco three days per week, so you must be located in the SF Bay Area and willing to work in the office on a regular basis.
duckbill