AI Product Engineer - ClickStack

About ClickHouse

ClickHouse, recognized on the 2025 Forbes Cloud 100 list, is a leading and rapidly expanding private cloud company boasting over 3,000 customers and an annual recurring revenue growing by more than 250% year over year. The company specializes in real-time analytics, data warehousing, observability, and AI workloads. Recently, a $400 million Series D funding round underscored its accelerating growth. Key customers include Capital One, Lovable, Decagon, Polymarket, Airwallex, Meta, Cursor, Sony, and Tesla.

Role Overview

Join ClickHouse in the mission to revolutionize data usage by building the AI layer for observability. The role focuses on ClickStack, an open-source observability platform integrating logs, metrics, traces, and session replays to help engineers quickly identify root causes. Your work will center on developing agentic systems capable of incident investigation, anomaly detection, and summarizing insights for on-call engineers.

Key Responsibilities

• Develop intelligent agents that probe incidents, detect anomalies, and diagnose production issues utilizing ClickStack.
• Create a reusable skill library reflecting team debugging tactics, root cause analysis, query creation in ClickHouse, and incident response procedures to empower agents with predefined playbooks.
• Take full ownership of the agent technology stack, including context engineering, tool development, evaluation frameworks, and cost monitoring, ensuring reliable production performance.
• Enhance ClickStack’s AI workload capabilities by building MCP servers, SDKs, and integrations enabling agents to process telemetry data, perform actions, and maintain their own observability.
• Collaborate openly with open-source contributors and clients to resolve issues and integrate feedback into product improvements.
• Address technical challenges such as latency, operating costs, context window constraints, evaluation coverage, and hallucinations on real telemetry data.

Qualifications and Experience

• At least five years of software engineering experience, including 1-2 years working with large language model-powered systems or intelligent agents in a production environment.
• Proficiency in backend development using TypeScript/Node.js and/or Python, comfortable with both languages.
• Demonstrated experience designing, building, and operating multi-step tool using agents with capabilities like planning, memory management, and error recovery.
• Expertise in skill design, including markdown-based workflows or Anthropic-style frameworks, with clarity on when to utilize skills versus tools.
• Familiarity with MCP architecture: server construction, tool design, authentication, scoping, and observability for agentic systems.
• Strong evaluation methodologies experience, including use of golden datasets, LLM-based judgment, and regression detection techniques.
• Solid SQL skills, specifically querying ClickHouse databases directly.
• Working knowledge of Docker and Kubernetes containerization and orchestration technologies.
• Active participation in open source and developer communities.

Preferred Additional Skills

• Experience building or managing production agents within observability, incident response, or Site Reliability Engineering (SRE) contexts.
• Well-formed opinions on agent observability including tracing, cost allocation, evaluation pipelines, and application of OpenTelemetry for agents, with actionable improvement ideas.
• Experience with techniques like prompt caching and context compaction to optimize agents’ operations on high-volume telemetry data.
• Familiarity with columnar database systems and event ingestion data pipelines.
• Contributions to or maintenance of open-source artificial intelligence or agent projects.
• Knowledge of systems programming languages such as Go or Rust for integrations and high-throughput infrastructure.

Compensation Disclosure

The salary range for this role varies by location and reflects minimum and maximum expected pay at posting. Placement within the range considers factors including education, skills, and business needs. Inquiries regarding pay transparency are welcomed via the company’s dedicated contact.

Perks

• Flexible remote work environment supporting a globally distributed team across 20+ countries.
• Employer contributions toward healthcare benefits.
• Equity offerings with stock options for all new hires.
• Generous and flexible time-off policies tailored by region.
• $500 stipend for home office setup for remote staff.
• Opportunities for in-person connection through company-wide global gatherings and offsite events.

Culture and Values

As a rapidly growing startup, employees play a pivotal role in shaping company culture and values. The organization supports an inclusive workplace that prohibits discrimination or harassment based on protected characteristics.

Equal Opportunity and Privacy

ClickHouse upholds equal employment opportunities for all and adheres strictly to privacy standards ensuring nondiscriminatory hiring and employment practices.