Senior Lead Software Engineer

Be an integral part of an agile team that's constantly pushing the envelope to enhance, build, and deliver top-notch technology products. As a Senior Lead of Software Engineering – Payment Data Platform Infrastructure at JPMorgan Chase within the Commercial and Investment Banking – Data Analytics Payments Team, you are an integral part of an agile team that works to enhance, build, and deliver trusted market-leading technology products in a secure, stable, and scalable way. Promote significant business impact through your capabilities and contributions, and apply deep technical expertise and problem-solving methodologies to tackle a diverse array of challenges that span multiple technologies and applications. Job responsibilities - Executes software solutions, design, development, and technical troubleshooting with ability to think beyond routine or conventional approaches to build solutions or break down technical problems - Produces architecture and design artifacts for complex applications while being accountable for ensuring design constraints are met by software code development - Deploy and operate ClickHouse clusters across environments (on-prem and/or cloud), including upgrades, scaling, backups, and disaster recovery. - Designs and implements scalable, secure, and cost-effective AWS infrastructure solutions. - Develops and maintains infrastructure as code (IaC) using tools such as Terraform, CloudFormation, or AWS CDK. - Monitors and optimizes the performance, availability, and scalability of AWS infrastructure. - Implements and manages security best practices, including identity and access management (IAM), encryption, and network security. - Automates infrastructure provisioning, configuration, and deployment processes. - Identifies opportunities for cost optimization and implement strategies to reduce AWS spending. - Develops and maintain scripts and tools to enhance operational efficiency. - Stays up-to-date with the latest AWS services, features, and best practices. Required qualifications, capabilities, and skills - Formal training or certification on software engineering concepts and 8+ years applied experience - Hands-on experience deploying and supporting ClickHouse in production (multi-node clusters). - Strong knowledge of ClickHouse internals and performance levers: MergeTree family, parts/merges, partitions, primary key vs ordering key, compression, TTLs, sampling, distributed query execution. - Experience with ClickHouse Keeper / ZooKeeper coordination and cluster topology design. - Experience with Linux systems engineering and troubleshooting (CPU, memory, disk, networking). - Expertise in AWS cloud platform (S3, Lambda, EKS, DynamoDB, EventBridge, Kafka, Opensearch), big data technologies (Apache Spark, Glue, Redshift), and data lake/lakehouse architectures (Iceberg). - Experience with containerization/orchestration and automation (Kubernetes and Terraform) - Stays up-to-date with the latest AWS services, features, and best practices. - Contributes to software engineering communities of practice and events that explore new and emerging technologies - Experience leading adoption of agentic AI-enabled engineering practices (using enterprise-authorized tools within the work environment) across teams, including defining operating expectations (human-in-the-loop validation, quality gates), measuring outcomes, and ensuring secure handling of sensitive inputs/outputs.