Python + AWS Glue Data Engineer

About the company

HCLTech is a global technology services organization that supports enterprises in transforming for the digital era. With decades of experience, a strong innovation culture, worldwide delivery capabilities, and a large workforce spread across many countries, the company works with leading businesses across industries. It also highlights its focus on diversity, social responsibility, sustainability, education initiatives, and building an environment where people can grow and perform at their best.

Role overview

The organization is seeking Data Engineers with hands-on experience in AWS, AWS Glue, and Python for a Bangalore-only opening. The role is centered on designing, building, and running secure, dependable, and cost-conscious data pipelines on AWS. You will work across batch, streaming, and CDC ingestion flows through to curated datasets used for analytics, while embedding testing, governance, observability, and CI/CD throughout the data lifecycle.

Key responsibilities

• Develop ingestion and change-data-capture pipelines using AWS services such as Lambda and EMR, with support for batch and streaming use cases as the platform evolves.
• Create workflow orchestration using Step Functions and/or MWAA (Airflow), including retries, alerts, and SLA handling.
• Design lakehouse and warehouse datasets on S3 using Parquet/Iceberg, Glue Data Catalog, and Athena, with strong focus on partitioning, compaction, and performance improvement.
• Build automated testing and data quality checks using tools such as Great Expectations or Deequ, along with PySpark unit tests, data contracts, and CI validation gates.
• Apply security and governance controls including IAM, KMS, VPC endpoints, Lake Formation policies, PII controls, audit logging through CloudTrail, and RBAC/RLS where applicable.
• Set up monitoring and cost controls through CloudWatch metrics and alerts, lineage and usage dashboards, and storage/compute optimization practices such as S3 lifecycle management, compression, and proper job sizing.
• Implement infrastructure and deployment automation using Terraform, CloudFormation, or CDK, and create delivery pipelines through GitHub Actions with promotion and rollback support.
• Maintain technical documentation, architecture diagrams, SLAs/SLOs, and operational playbooks for incident handling.
• Reuse existing capabilities from other teams whenever appropriate and contribute improvements back to the wider engineering community rather than duplicating work.

Desired outcomes in the first 60-90 days

• Bring one complete data pipeline into production with testing, monitoring, and alerting in place.
• Put in place a governance foundation for two important data sources, including Lake Formation, tagging/classification, encryption, and data contracts.
• Establish CI/CD and infrastructure-as-code for data services and lower at least one cost driver through storage or compute optimization.

Experience and technical profile

• At least 5 years of experience building data platforms on AWS, with strong Python skills including PySpark and solid SQL capability.
• Practical experience with services and platforms such as DMS, Kinesis/MSK, Glue/EMR, Lambda, Step Functions/MWAA, S3/Parquet/Iceberg, Glue Catalog, Athena, and Redshift/Serverless.
• Experience with infrastructure as code using Terraform and continuous delivery using GitHub Actions.
• Working knowledge of data quality frameworks, schema evolution, and backfill handling.
• Background in security and compliance controls such as IAM/KMS, private networking, Lake Formation, audit, and lineage.
• Ability to tune performance and control costs across storage and compute layers.
• Nice-to-have exposure to Snowflake, Apache Flink, Redshift Spectrum, OpenLineage, and policy-as-code tools such as OPA.

Application details

This position is open only for candidates based in Bangalore. Interested candidates are asked to share their profile details, including total years of experience, experience in AWS Glue, experience in Python, current CTC, expected CTC, notice period, current location, and preferred location. The hiring contact provided is srikanth.domala@hcltech.com.

Eligibility

Any graduate may apply, provided they meet the experience and skill requirements for the role.

Additional notes

The role is for a core data engineering position focused on AWS-based data platforms. The source listing mentions a salary range of INR 15,00,000 to INR 30,00,000 per year.