The Lead BI Engineer will be responsible for architecting, developing, and optimizing enterprise-scale business intelligence solutions. This role requires deep expertise in Power BI, Azure, Databricks, and SQL, with the ability to design scalable data pipelines, deliver advanced analytics, and drive BI adoption across the organization. The ideal candidate combines strong technical skills with leadership capabilities, guiding teams and collaborating with stakeholders to enable data-driven decision-making.
Key Responsibilities
Data Engineering & Pipeline Development
- Design and implement data ingestion pipelines from multiple sources using Azure Databricks.
- Develop scalable and reusable frameworks for ingesting and transforming datasets.
- Integrate end-to-end data pipelines ensuring data quality, consistency, and governance.
- Work with event-based/streaming technologies for real-time data ingestion and processing.
- Optimize Databricks code and perform performance tuning for large-scale workloads.
SQL Development & Optimization
- Strong expertise in SQL, T-SQL, and PL/SQL with hands-on experience in SQL Server.
- Write, analyze, and optimize complex SQL queries including stored procedures, functions, triggers, joins, updates, and constraints.
- Deep knowledge of SQL architecture: Agent Jobs, Indexes, Partitions, Views, and Procedures.
- Proven ability to design high-performance queries and implement best practices for SQL optimization.
BI & Visualization
- Lead the design and development of Power BI dashboards, reports, and KPI scorecards.
- Proficient in DAX, advanced calculations, and data modeling aligned with warehouse standards.
- Collaborate with business stakeholders to translate requirements into actionable BI solutions.
- Champion BI adoption and foster a culture of data-driven decision-making.
Cloud & Infrastructure
- Design and deploy data applications on Azure cloud solutions (Synapse, Data Lake, SQL DB, Blob Storage).
- Hands-on experience with Azure Data Factory, Azure Functions, and Azure DevOps for automation and orchestration.
- Provision and manage Databricks clusters, including scheduling and job automation.
- Familiarity with AWS/GCP integration where applicable.
Advanced Analytics & Development
- Experience with PySpark and Python for data processing and advanced analytics.
- Hands-on experience with Spark SQL, Spark Streaming, Delta Lake, and Scala/Python notebooks.
- Develop proofs of concept (POCs) and prototypes with customer teams.
- Implement transformational big data projects, including real-time streaming using Databricks.