We are looking for a Senior Data Architect to design the end-to-end data platform for APEX. The platform ingests contact centre telephony and quality data from the Genesys A3S Snowflake database and Verint Workforce Management (WFM) feeds, and delivers it through a Bronze/Silver/Gold (Medallion) architecture on Databricks into a Kimball-style reporting layer.
You will own the architecture: ingestion strategy and cadence per source, incremental loading and batch control, data integrity standards, and the Gold-layer dimensional model. You will also partner with the existing CCARS pipeline team through parallel run and decommissioning of the legacy pipelines.
This role suits a hands-on Senior or Lead Data Architect who can turn complex reporting requirements into robust, auditable data solutions and explain architectural trade-offs to both engineers and business stakeholders.
Working Model: This is a hybrid role requiring 2 days per week in a downtown Toronto office.
Responsibilities:
- Design the Bronze/Silver/Gold data flow for the new Genesys A3S Snowflake and Verint sources, with clear standards for ingestion, transformation, validation, and consumption.
- Define ingestion cadence and watermarking strategy per source, based on each source's true data freshness.
- Design the Gold-layer Kimball star schema (facts, dimensions, grain, SCD Type I/II) to support sales, telephony, agent activity, and quality reporting.
- Choose between pure lakehouse and hybrid lakehouse-to-data-warehouse patterns based on source characteristics, volumes and performance needs.
- Build and optimize Databricks and Delta Lake pipelines, and set cost and performance practices for cluster sizing, job versus all-purpose compute, and Photon.
- Design ingestion from the external Genesys Snowflake source, including secure connectivity, incremental extraction and source-to-target reconciliation.
- Integrate Verint and other vendor feeds delivered as flat files, SFTP drops or REST APIs, normalizing inconsistent formats and tolerating schema drift.
- Design API ingestion that handles pagination, rate limits, asynchronous submit-poll-retrieve jobs and per-entity fan-out at scale.
- Define data integrity checks, batch control, idempotency and failure-recovery patterns for each pipeline, including row-count and checksum reconciliation and late-arriving data handling.
- Design orchestration with Azure Data Factory, Databricks Workflows or equivalent, with independent batch tracking and failure isolation per source feed.
- Write and review production-grade transformation logic in T-SQL, Spark SQL and Python/PySpark.
- Document architecture decisions and trade-offs for engineering and business audiences.
- Partner with the existing CCARS pipeline team on parallel-run validation, cutover planning and legacy pipeline decommissioning.