Spark Developer

About the Role We are looking for a Senior Software Engineer with strong expertise in Apache Spark and distributed systems. The role involves working on Spark internals, contributing to the Apache Spark open-source community, and building scalable data processing solutions aligned with the platform requirements. Key Responsibilities - Contribute fixes and enhancements to the Apache Spark open-source project. - Upgrade and maintain compatibility with newer Spark releases. - Optimize Spark internals for performance, scalability, and reliability. - Design and develop distributed data processing solutions. - Debug complex Spark-related performance and stability issues. - Collaborate with product and platform teams on architecture and technical initiatives. Required Skills - Strong experience with Apache Spark internals and distributed systems. - Hands-on experience contributing to open-source projects. - Strong Java and/or Scala programming skills. - Expertise in query execution, distributed processing, and performance optimization. - Strong debugging and problem-solving abilities. - Ability to work independently and drive technical initiatives. Good to Have - Experience with Kubernetes and cloud-native environments. - Knowledge of JVM tuning and system optimization. - Contributions to Apache or other open-source communities. - Experience with large-scale analytics or data platforms. Originally posted on Himalayas