Lead Software Engineer - Data Engineer
JP Morgan Services India Pvt LtdJob Description
Lead Software Engineer - Data Engineer
Job responsibilities
- Build automated, self-healing data operations using agentic frameworks and AI-driven workflows for intelligent monitoring, optimization, and remediation
- Design and deploy data solutions using AWS services including S3, Lambda, EKS (Elastic Kubernetes Service), and Step Functions
- Implement Infrastructure as Code (IaC) practices for reproducible and scalable deployments
- Drives team adoption of enterprise-authorized AI-assisted engineering practices within the work environment to improve code quality, delivery speed, and operational outcomes (e.g., AI-assisted code review/refactoring, test strategy acceleration, incident/root-cause analysis support), while establishing consistent validation standards (secure coding, peer review, automated testing) and promoting reuse of effective patterns across the team.
- Applies knowledge of tools within the Software Development Life Cycle toolchain, including enterprise-authorized AI-assisted development and automation capabilities, to improve the value realized by automation.
- Build and optimize scalable data ingestion, transformation, and analytics solutions using Apache Spark, AWS Glue, and Apache Iceberg
- Develop and manage both batch and real-time data pipelines leveraging Apache Kafka and Apache Flink for streaming data processing. Implement Data Lake architectures with efficient data organization, partitioning, and governance strategies
- Design and architect end-to-end data solutions from ingestion to analytics, ensuring scalability, reliability, and performance at enterprise scale
- Define data architecture standards and best practices for batch and real-time data processing pipelines
- Formal training or certification on software engineering concepts and 5+ years applied experience
- Demonstrated experience leading effective use of approved AI-assisted software development tools (e.g., for coding, code review, test acceleration, troubleshooting) with the ability to set team expectations for validating AI outputs for correctness, performance, and security.
- Strong understanding of responsible AI use in engineering workflows, including data sensitivity considerations, secure handling of inputs/outputs, and adherence to resiliency and security expectations; experience coaching engineers on safe, compliant adoption within delivery practices
- Experience with modern data engineering practices including DataOps, data observability, and data governance
- Deep expertise in data ingestion, transformation, and analytics using Apache Spark, AWS Glue, Apache Iceberg, and Data Lake architectures
- Proven experience designing and managing batch and real-time data pipelines with Apache Kafka and Apache Flink
- Strong hands-on experience with AWS cloud services including S3, Lambda, EKS, and Step Functions
- Experience building automated solutions using agentic frameworks and AI-driven workflows for intelligent data operations
- Advanced proficiency in programming languages such as Python, Scala, or Java with strong SQL skills
- Demonstrated ability to architect end-to-end data solutions from ingestion to analytics at scale
Job role
Job requirements
About company
Similar jobs you can apply for
Software / Web DeveloperMobile App Developer
MSKGRC Private Limited
Digital Marketing Head
PLC Projects Private Limited
Associate Software Engineer
Elevate Global Technologies Private Limited
FULL STACK ENGINEER – MERN STACK
The Hard Cash
Field Auditor
Freecharge Payment Technologies Private LimitedAssociate IOT Engineer
Krytos Consulting LlpYou can expect a minimum salary of 0 INR. The salary offered will depend on your skills, experience and performance in the interview.
The candidate should have completed the required education and people who have 5 to 31 years are eligible to apply for this job. You can apply for more jobs in Hyderabad to get hired quickly.
The candidate should have sound communication skills and sound communication skills for this job.
Both Male and Female candidates can apply for this job.
No, it's not a work from home job and can't be done online. You can explore and apply for other work from home jobs in Hyderabad at apna.
No work-related deposit needs to be made during your employment with the company.
Go to the apna app and apply for this job. Click on the apply button and call HR directly to schedule your interview.
The last date to apply for this job is . For more details, download apna app and find Full Time jobs in Hyderabad . Through apna, you can find jobs in 64 cities across India. Join NOW!