Senior Data Engineer

About Datum Technologies Group

Datum Technologies Group encompasses three global brands dedicated to trust, transparency, and excellence. Their mission is to be the preferred partner for comprehensive technology solutions and staffing services. With expertise in managed services and government contracting, they are well-equipped to help clients achieve goals and drive growth.

Role Overview and Key Responsibilities

• Design, build, and maintain scalable and automated data engineering solutions emphasizing robust data quality and production readiness.
• Apply advanced AI knowledge to incorporate modern AI models and AI-assisted development tools into data workflows, understanding prompts, tokens, context windows, reasoning behavior, and integration points.
• Develop and support Python-based scripting solutions for automation, file processing, data validations, and reusable engineering utilities.
• Lead automation efforts to minimize manual tasks, increase repeatability, reinforce controls, and enhance data processing speeds.
• Execute Unix shell scripting, scheduling, file management, log analysis, and troubleshoot operational issues across data platforms.
• Contribute to the development and upkeep of specification-driven data processing modules (Spec Kit).
• Maintain high data quality standards through comprehensive validation, reconciliation, exception handling, root cause analysis, and continuous monitoring.

Mandatory Qualifications and Experience

• 7 to 8 years of experience as a Data Engineer or Senior Data Engineer in data-intensive technical environments.
• Strong technical proficiency with knowledge of complex data systems, batch jobs, pipelines, and integration techniques.
• In-depth understanding of AI models and AI-assisted coding, including their functions in large language models, code suggestion, reasoning modes, and constraints relevant to data engineering.
• Expertise in Python for coding and scripting to automate data validation, processing, file handling, and creating reusable tools.
• Experience using GitHub Copilot for AI-driven code generation, reviews, debugging, unit testing, documentation, and productivity enhancement.
• Hands-on Unix skills involving shell scripting, command-line tools, file transfers, scheduling, log inspections, and system troubleshooting.
• Solid grasp of data quality frameworks covering validation, reconciliation, exception management, and defect prevention.
• Familiarity with AI model prompt interpretation, context window influence, generated outputs, and AI-assisted automation impacts on data quality.
• An automation-centric engineering approach focused on script creation, reusable components, monitoring checks, and standardized operational procedures.
• Experience in Spec Kit creation including specification analysis, building, testing, validating, and maintaining data processing logic.
• Knowledge of DevOps practices such as version control, continuous integration/deployment, environment setup, monitoring, and supporting production environments.
• Strong communication skills for clear documentation and information sharing.

Preferred Skills

• Experience with advanced automation frameworks, reusable script repositories, or AI-enhanced data engineering accelerators.
• Healthcare domain knowledge is a plus.
• Advanced Python capabilities for integrating AI outputs with automation workflows, developing data profiling tools, and production-grade scripts.
• Awareness of AI development lifecycle (AI DLC) practices including prompt engineering, model selection, output validation, testing, monitoring, governance, documentation, and ethical AI usage.