Azure Data Engineer

About the Role

Join Orcapod Consulting Services as a Custom Software Engineer dedicated to creating tailored applications that align tightly with specific business workflows and requirements. You will collaborate with stakeholders to gather needs, develop innovative and efficient solutions, and optimize applications for both performance and user experience. Participation in thorough code reviews and ongoing enhancement of development practices to meet strategic organizational goals will be integral.

Key Responsibilities

• Design, build, and maintain sophisticated ETL pipelines on Azure cloud technologies to ingest and integrate data from diverse sources, ensuring readiness for analytics and data science applications.
• Evaluate machine learning algorithms for problem-solving suitability and prioritize them based on success likelihood.
• Ensure data delivered to users—including analysts, data scientists, and executives—is accurate, insightful, and trustworthy for decision-making.
• Create optimized dimensional database schemas capable of supporting hundreds of users querying millions of transactions with response times ranging from minutes to sub-second.
• Develop next-generation platforms utilizing the latest cloud-based technologies.
• Participate actively in the complete software development lifecycle.

Professional and Technical Qualifications

• Experience working with structured, semi-structured, and unstructured data, extracting insights and mapping relationships across disparate data sets.
• Proficiency in at least one programming language such as Scala or Python.
• Hands-on experience with traditional data warehousing technologies and Microsoft Azure Data Services, including Azure Data Factory, Azure Databricks, Azure Data Lake Storage, Azure DevOps, and Azure storage solutions.
• Advanced SQL skills along with relational database experience, including query authoring and familiarity with multiple database systems, such as PostgreSQL, MS SQL Server, Oracle, and capability with distributed processing platforms like Databricks or Apache Spark.
• Solid knowledge of machine learning libraries such as scikit-learn and pandas is preferred.
• Expertise in managing large complex codebases using Github, Gitflow, Fork/Pull Models, and continuous integration/deployment (CI/CD) pipelines.
• Exposure to developing web services using SOAP and RESTful APIs.
• Working familiarity with Agile development methodologies including SCRUM, XP, and Kanban.

Additional Valuable Skills

• Exposure to IoT Central or IoT Hub.
• Experience working with Azure Kubernetes Service (AKS).

Additional Information

• Minimum 6 years of experience in Microsoft Azure Data Services is mandatory.
• This role requires your presence at the Pune office.
• Candidates must have completed at least 15 years of formal education.