NBCUniversal Advertising Products & Solutions (AP&S) is responsible for the product development and project management of NBCUniversal鈥檚 full advertising technology suite. From sales support to campaign execution, delivery, and billing, our division services both internal and external customers in support of NBCUniversal鈥檚 $7B+ annual advertising business. Together, we鈥檙e building鈥痶he鈥痯latform that powers the future of advertising at NBCU.
We are seeking a Senior Data Engineer, Engineering & Operations to lead the Engineering & Automation pillar. Reporting to the Sr. Director of Engineering & Operations, you will lead the engineering and operational excellence behind our data collaboration ecosystem. You will be instrumental in designing and implementing scalable data architectures and driving automation through AI agents and self-service tooling, translating prototypes into robust production solutions.
Duties and Responsibilities
The person in this role will:
- Architecture Leadership: Define partner onboarding and clean room architecture patterns across Snowflake, LiveRamp, and Databricks that are secure, scalable, and repeatable.
- Infrastructure Setup & Library Deployment: Configure and manage partner-specific clean room environments; deploy and manage Python-based libraries within the platform ecosystem.
- MLOps Integration: Establish and maintain MLOps practices, including model serving, monitoring, and pipeline orchestration for AI/ML features deployed within the platform ecosystem.
- Security & RBAC: Own design and enforcement of granular RBAC policies and least-privilege service accounts.
- Partner Onboarding: Serve as the technical lead for onboarding new partners, implementing privacy-preserving controls (e.g., aggregation thresholds and anonymization techniques).
- Pipeline & Data Product Ownership: Design, build, and operate scalable ELT pipelines using Snowpark and/or PySpark and advanced SQL to provision Gold datasets.
- Identity Resolution: Implement and evolve identity resolution logic mapping internal data to 3P identifiers (including LUIDs, RampIDs, TransUnion IDs), ensuring privacy-safe practices.
- Scalable Architecture: Design and operate scalable data architectures across Snowflake and Databricks supporting batch and near real-time processing patterns.
- Data Quality by Design: Build robust automated checks (e.g., Great Expectations or custom SQL assertions) and define quality standards to detect schema drift, null rate spikes, and volume anomalies.
- FinOps & Operational Excellence: Lead performance optimization across platforms (query tuning, caching, incremental processing) and define and implement query tagging and chargeback models for accurate cost attribution.
- Operational Maturity: Establish monitoring, alerting, runbooks, and standard operating procedures to improve platform reliability and reduce incident time-to-resolution.
- Testing & Validation: Validate that output data adheres to privacy and business requirements, and define test strategies for partner-facing releases.
- Technical Escalation: Serve as the escalation point for diagnosing connection failures, data discrepancies, or latency issues with partner technical teams.
- Agentic Workflows & Platform Enablement: Design and build internal AI agents (using frameworks like LangChain, Snowflake Cortex) and mentor other engineers through code reviews, design discussions, and operational best practices.