Data Engineer

Role Overview

The Data Engineer will utilize their extensive knowledge of data technologies and architectures aligned with Nedbank's Data Architecture Roadmap. This role involves advancing technical leadership within the enterprise, delivering tailored data products, and backing data-driven projects. Additionally, the engineer will enhance the bank's data systems to facilitate advanced analytics, AI, and machine learning by ensuring provision of accurate and accessible data to stakeholders.

Primary Duties

• Maintain, upgrade, clean, and manipulate datasets within the bank's operational and analytics databases.
• Develop and manage robust, scalable, and secure data infrastructure using technologies such as DB2, PostgreSQL, MSSQL, HBase, NoSQL, Azure Data Lake Gen 2, SAS, Azure Databricks, Azure Data Factory, HDInsight, Denodo, Netezza, and Azure Cloud, ensuring data privacy in coordination with Information Security and Data Governance teams.
• Build and sustain data pipelines for ingestion, provisioning, streaming, and API delivery, integrating multiple data sources and formats from trusted origins to Nedbank's Data Warehouse and marts supporting business, regulatory, and compliance needs.
• Transform data into unified formats conducive for reporting and analysis, utilizing big data tools like Hadoop, Kafka, and data replication software such as IBM Infosphere Data Replication.
• Promote use of data integration tools including Ab Initio and cloud-based tools like Azure Data Factory and Databricks.
• Collaborate with Data Modelers to design data schemas for various warehouse layers and marts.
• Automate and monitor data pipeline performance for optimal operation.
• Work closely with analysts, software engineers, data scientists, scrum masters, and warehouse teams within agile squads, owning delivery of Epics end-to-end and ensuring business value realization.
• Implement rigorous data quality and governance practices to maintain accuracy, security, and consistency.
• Optimize performance of data warehouse components, integration flows, batch and real-time processing, streaming, and API endpoints.
• Develop APIs that support a data-driven organization in partnership with software engineering teams.

Qualifications and Experience

• Minimum education: Matric/Grade 12 with advanced diplomas or first degrees preferably in BCom, BSc, or BEng fields.
• Preferred certifications include Cloud (Azure, AWS), DevOps, Data Engineering, Data Science certifications from recognized platforms like Coursera, Udemy, SAS, or Microsoft.
• 3 to 6 years of relevant professional experience working independently within team environments, demonstrating ability to deliver without supervision.
• Proven expertise in designing, building, and maintaining data warehouses and lakes.
• Experience in big data technologies such as Hadoop, Spark, Hive, and programming skills in Python, Java, and SQL.
• Familiarity with relational (PostgreSQL, MS SQL, IBM DB2) and NoSQL databases (HBase, MongoDB).
• Hands-on experience with cloud computing services including AWS, Azure, and Google Cloud Platform.
• Knowledge of data visualization tools and an analytical, innovative problem-solving mindset.

Technical and Professional Skills

• Expertise in cloud data engineering across platforms like Azure, AWS, and Google Cloud.
• Advanced data warehousing and database management skills.
• Proficiency in programming languages: Python, Java, and SQL.
• Competence in data analysis, modeling, and pipeline development using ETL tools such as Ab Initio, Azure Data Bricks, Azure Data Factory, and SAS ETL.
• Experience working within Agile methodologies.

Behavioral Competencies

• Strong decision-making skills.
• Influencing and interpersonal communication abilities.
• Creativity and innovative thinking.
• Ability to build and maintain effective partnerships.
• Commitment to continuous learning and professional growth.