As a member of the Cyber Security Engineering (CSE) team within Information Security & Risk Management (ISRM), the Data Engineer focuses on expanding data capabilities. This role is responsible for delivering high-value data management solutions, including data pipelines, models, and SIEM platform optimization, to empower analysts and protect the business.
Responsibilities:
- Data Pipeline Development: Design, implement, and enhance robust streaming and batch data pipelines utilizing message brokers to efficiently feed the SIEM and other downstream analytics engines.
- Data Transformation & Normalization: Leverage observability pipelines to aggressively route, filter, and normalize/harmonize data, creating structured datasets from unstructured logs prior to SIEM ingestion.
- Data Modeling & Architecture: Build scalable data models and enhance standard schemas within data warehousing solutions to deliver reliable, cost-effective, query-optimized storage.
- Data Integrity & Lineage: Verify data integrity and translations across distributed systems and message topics while managing end-to-end data lineage.
- Development & Integration: Analyze requirements to determine the necessary coding, API integrations, and programming activities to connect disparate security telemetry sources into the SIEM, data warehouses, or other repositories.
- Testing & Quality Assurance: Execute testing plans, debug pipeline routing issues, and thoroughly document data flows, routing configurations, and integration protocols.
- Data Management Operations: Perform the compilation, cataloging, caching, and rapid retrieval of telemetry within the SIEM and associated data lakes.
- Analytics Toolsets: Create, manage, and support advanced analytics and reporting environments operating outside the primary SIEM for long-term security analytics and hunting.
- Requirements & Capacity Planning: Define precise data specifications and proactively plan for capacity changes across streaming, routing, indexing, and storage infrastructure.
- Governance & Standards: Assist in developing, documenting, and enforcing comprehensive data ingestion standards, parsing policies, and retention procedures across all supported platforms.
- Actionable Insights: Analyze diverse data sources across the data stack to uncover trends, improve data quality, and provide actionable recommendations to the security operations team.
- Metrics Automation: Develop standards and implement robust automations for metrics aggregation and dissemination, pulling key telemetry from the SIEM and data warehouses.