Software Engineer - Data & AI Technology

About the Role

Join Scotiabank's dedicated and inclusive Data & AI Technology team in Toronto as a Software Engineer, focusing on developing and enhancing the enterprise data catalog and management solutions leveraging the Databricks ecosystem. This role emphasizes creating robust, scalable, and secure data services to facilitate data discovery and governance across the bank.

Key Responsibilities

• Design and implement scalable components for the enterprise data catalog to automate and streamline data governance workflows.
• Integrate the data catalog system with Databricks and other data platforms within the organization.
• Develop solutions for data quality, profiling, and classification to maintain data integrity and usability.
• Lead architectural discussions and make technical decisions pertinent to data management solutions.
• Promote and uphold best engineering practices in collaboration with your development squad.
• Assess trade-offs between robustness, performance, and accuracy to deliver optimal solutions aligned with customer needs.
• Proactively identify risks and develop scalable long-term fixes.
• Exhibit strong ownership and attention to detail in all aspects of work.
• Mentor and support junior engineering team members.
• Engage with stakeholders and product managers to understand requirements and ensure the data infrastructure aligns with product objectives.

Qualifications and Skills

• Bachelor’s or Master’s degree in Computer Science, Engineering, or a related discipline.
• Deep understanding of design patterns and lakehouse architecture.
• Experience with backend and frontend development frameworks.
• Proficiency in Python and/or Java programming.
• Hands-on experience with Databricks, Unity Catalog, Delta Lake, and Apache Spark.
• Familiarity with enterprise data catalog tools such as Atlan, Alation, Collibra, DataHub, or OpenMetadata.
• Skill in building reusable frameworks for data quality, profiling, and classification.
• Experience with implementing data security and access control.
• Strong knowledge of data governance, metadata management, and data lineage concepts.
• Experience with cloud platforms like Azure or GCP and tools such as Terraform for infrastructure-as-code.
• Understanding of RESTful APIs, microservices, and distributed system design.
• Knowledge of streaming technologies like Kafka and structured streaming is a plus.
• Strong analytical and problem-solving abilities with creativity and persistence.
• Awareness of product development lifecycles and data infrastructure necessities.

Employee Experience and Benefits

• Commitment to Diversity, Equity, Inclusion & Allyship by fostering an inclusive culture with bias-free practices and supportive Employee Resource Groups.
• Focus on Accessibility and Workplace Accommodations to ensure an inclusive environment meeting accessibility standards.
• Opportunities for skill development through online courses, cross-functional projects, and tuition assistance.
• Comprehensive rewards package including bonuses, flexible vacation, personal and sick days, plus benefits starting from day one.
• A collaborative and dynamic workspace with amenities like free tea and coffee and universal washrooms.
• Programs promoting community engagement and belonging regardless of work location.

Additional Information

Scotiabank is a major banking institution in the Americas, dedicated to helping customers and communities achieve success through varied financial services. The organization is committed to providing accommodations throughout recruitment and selection processes for applicants with disabilities. Candidates must apply directly online. Only those shortlisted for interviews will be contacted.