Data Engineer

Job Overview

The role entails managing and supporting technical configurations related to data systems, including addressing connectivity issues, API failures, and service disruptions. The candidate will handle documentation of changes, research and specify technical requirements, and manage security vulnerability fixes. Collaboration with infrastructure teams and support for data product transformations in the cloud is also key. Additionally, the role involves providing technical consultancy, developing monitoring reports, generating test data, implementing DevOps workflows, optimizing data pipelines, and ensuring adherence to data management policies.

Primary Responsibilities

• Conduct investigations and resolve incidents concerning technical configurations like connectivity errors, API timeouts, and service outages.
• Maintain detailed documentation for adjustments in configurations, including data sources, pipelines, and user accounts.
• Research and compile technical requirements for projects and processes.
• Review and execute fixes for reported security vulnerabilities promptly.
• Work with infrastructure teams to address data team requests.
• Support product managers during cloud migration and transformation initiatives for data products.
• Offer technical assistance and consulting to business stakeholders.
• Build reports that track usage statistics, system performance, and security incidents.
• Create simulated data for testing and develop robust unit tests.
• Establish and maintain DevOps pipelines to streamline development and deployment.
• Enhance data pipeline performance for greater efficiency.
• Audit data pipelines regularly to ensure conformity with data management standards and policies.

Required Qualifications and Skills

• Possess a minimum of a Bachelor's degree in Computer Science, Computer Engineering, or a closely related field.
• Have at least four years of professional experience in Data Engineering, Data Science, Software Engineering, or IT Infrastructure roles.
• Skilled in creating dashboards using tools such as QuickSight or Tableau.
• Experienced in designing user-friendly dashboard interfaces (UX design).
• Proficient in SQL scripting for data analysis, transformation, and integration across multiple sources.
• Competent in programming with Python.