Data Engineer (Fabric)

Job Overview

The Microsoft Fabric Data Engineer designs, builds, and operates modern data platforms using Microsoft Fabric. This role focuses on ingesting, modeling, and serving data via OneLake, Lakehouse, Data Warehouse, Data Pipelines, and Power BI—delivering trusted, performant datasets and governed analytics at scale. The role collaborates closely with data architects, analytics engineers, BI developers, and business stakeholders.

Key Responsibilities

1. Data Platform Engineering (Fabric)

• Build and manage Lakehouses (Delta Lake) and Fabric Data Warehouses.
• Develop Data Pipelines and Dataflows Gen2 for batch and near-real-time ingestion.
• Create and optimize Notebook-based transformations (PySpark/SQL) and SQL stored procedures for DW workloads.
• Implement medallion architecture (bronze/silver/gold) for scalable curation.
• Publish certified semantic models and Power BI datasets aligned to business domains.

2. Performance & Reliability

• Optimize storage/compute in OneLake (file formats, partitioning, z-ordering).
• Tune Spark and SQL workloads (caching strategies, concurrency, workload isolation).
• Implement robust retry, alerting, and monitoring (Fabric Monitoring Hub, Metrics app).
• Conduct end-to-end pipeline performance testing and scalability assessments.

3. Governance, Security & Compliance

• Enforce data governance with sensitivity labels, row-level/column-level security, and workspace roles.
• Manage item-level permissions (Lakehouse tables, DW schemas, datasets) and Managed Identities for sources.
• Apply data quality rules, lineage, and documentation (Descriptions, Tags, Owner metadata; Purview if applicable).
• Ensure compliance with organizational standards (PII handling, audit, retention).

4. DevOps & Lifecycle Management

• Use Fabric Git integration and Deployment Pipelines for CI/CD across dev/test/prod.
• Parameterize pipelines and environments; externalize configuration and secrets (Key Vault).
• Implement automated testing for data transformations and schemas.
• Drive release management, change control, and rollback strategies.

5. Collaboration & Stakeholder Engagement

• Partner with analytics engineers and BI teams to design star schemas, semantic models, and DAX measures.
• Work with data source owners for SLAs, schema change management, and contracts.
• Translate business requirements into technical designs and document architecture decisions.
• Provide knowledge transfer, best practices, and support to data consumers.