Data Engineer with Expertise in Hadoop, Spark, and Scala

About Tata Consultancy Services

Tata Consultancy Services is a leader in IT services, consulting, and business solutions, serving many of the world's top businesses for over five decades. The company embraces innovation and shared knowledge to shape a purposeful future.

Role Overview

We are seeking a Data Engineer to join our team with a strong background in big data technologies and programming to support large-scale data processing and management initiatives.

Key Responsibilities

• Develop, implement, and maintain data engineering solutions using technologies such as Hadoop, Spark, Kafka, and Hive.
• Create and optimize ETL pipelines using tools including Apache NiFi, Talend, or Informatica.
• Design and manage data warehousing systems using platforms like Amazon Redshift, Google BigQuery, and Snowflake.
• Perform data manipulation and processing using programming languages such as Python, Java, and Scala.
• Manage SQL and NoSQL database systems effectively to support data storage and retrieval.

Required Skills and Qualifications

• Proficient in programming languages for data engineering, specifically Python, Java, or Scala.
• Strong practical knowledge of big data ecosystems including Hadoop, Spark, Kafka, Hive, and HDFS.
• Expertise in SQL and NoSQL databases for efficient data handling.
• Experience with ETL development processes and related tools.
• Familiarity with cloud environments such as AWS, Azure, and Google Cloud for deployment and scaling.
• Knowledge of data modeling principles for creating scalable and efficient data architectures.
• Experience with version control tools like Git to collaborate and manage codebases.
• Understanding of data governance, data quality protocols, and security practices to safeguard sensitive data and comply with standards.

Additional Information

Candidates should have any graduate degree. The position is based in Hyderabad, India, with on-site work requirements.