| Spark, Scala, and SQL Developer (Saint Louis MO Hybrid) at Remote, Remote, USA |
| Email: [email protected] |
| From: Debasish Pattnaik, MRTECHNOSOFT [email protected] Reply to: [email protected] Job Title: Spark, Scala, and SQL Developer Location: (Saint Louis MO Hybrid) Job Type:Contract Job Description: We are looking for a talented and experienced Spark, Scala, and SQL Developer to join our dynamic team. The ideal candidate will have a strong background in big data technologies, proficient in Spark, Scala, and SQL, and will be responsible for designing, developing, and maintaining scalable data processing systems. This role requires a blend of technical expertise, analytical skills, and a passion for working with big data. Key Responsibilities: Design and Development: Develop and optimize data processing applications using Apache Spark and Scala. Write efficient and scalable SQL queries for data extraction, transformation, and loading (ETL) processes. Design and implement complex data processing pipelines, ensuring high performance and reliability. Data Management: Manage and manipulate large datasets, ensuring data quality and consistency. Implement data ingestion, transformation, and storage solutions. Collaborate with data scientists and analysts to understand data requirements and deliver relevant datasets. Optimization and Performance Tuning: Optimize Spark jobs for performance and scalability. Troubleshoot and resolve performance issues with Spark and SQL queries. Monitor and tune data processing jobs to ensure efficient resource usage. Collaboration and Communication: Work closely with cross-functional teams, including data engineers, analysts, and product managers. Communicate complex technical concepts to non-technical stakeholders. Provide technical guidance and mentorship to junior developers. Documentation and Reporting: Document data processing workflows, data models, and job configurations. Prepare technical reports and documentation for data processing projects. Ensure compliance with data governance and security policies. Qualifications: Education: Bachelor's or Master's degree in Computer Science, Information Technology, or a related field. Experience: Proven experience as a Spark, Scala, and SQL Developer or similar role. Strong background in big data technologies and distributed computing. Hands-on experience with Apache Spark, including writing and optimizing Spark jobs. Proficiency in Scala programming language. Advanced SQL skills with experience in writing complex queries and optimizing them. Technical Skills: Familiarity with data processing frameworks and tools such as Hadoop, Kafka, and Hive. Experience with cloud platforms like AWS, Azure, or Google Cloud is a plus. Knowledge of data warehousing and ETL processes. Understanding of data modeling and schema design. Soft Skills: Excellent problem-solving and analytical skills. Strong communication and collaboration abilities. Ability to work in a fast-paced and dynamic environment. Attention to detail and commitment to quality. Preferred Qualifications: Experience with machine learning frameworks and libraries. Knowledge of other programming languages like Python or Java. Experience with containerization and orchestration tools such as Docker and Kubernetes. Thanks Debasish Pattnaik [email protected] www.mrtechnosoft.com Keywords: Missouri Spark, Scala, and SQL Developer (Saint Louis MO Hybrid) [email protected] |
| [email protected] View all |
| Tue Jul 30 00:10:00 UTC 2024 |