| Big Data Engineer with extensive experience in PySpark Onsite at Dallas TX or Richmond VA at Richmond, Virginia, USA |
| Email: [email protected] |
| From: Debasish Pattnaik, MRTECHNOSOFT [email protected] Reply to: [email protected] Big Data Engineer with extensive experience in PySpark Onsite at Dallas TX or Richmond VA We are seeking a highly skilled Big Data Engineer with extensive experience in PySpark to join our dynamic team. The ideal candidate will be responsible for designing, developing, and maintaining big data solutions to support our data-driven initiatives. This role requires a deep understanding of big data technologies, distributed systems, and the ability to work collaboratively with cross-functional teams.Key Responsibilities: Design and implement scalable data pipelines using PySpark and other big data technologies. Develop and maintain data processing workflows to ingest, process, and analyze large datasets. Optimize and tune PySpark applications for performance and scalability. Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and deliver robust solutions. Ensure data quality, integrity, and security across all data processing activities. Monitor and troubleshoot data pipeline issues, providing timely resolutions. Stay up-to-date with the latest industry trends and advancements in big data technologies. Contribute to the development of best practices and standards for data engineering.Qualifications: Bachelor's or Master's degree in Computer Science, Information Technology, or a related field. 3+ years of experience in big data engineering, with a focus on PySpark. Strong proficiency in PySpark, including data frame operations, RDDs, and Spark SQL. Experience with big data technologies such as Hadoop, HDFS, Hive, and Kafka. Proficient in programming languages such as Python, Scala, or Java. Familiarity with cloud platforms (e.g., AWS, Azure, GCP) and their big data services. Strong understanding of data warehousing concepts and ETL processes. Experience with version control systems (e.g., Git) and CI/CD pipelines. Excellent problem-solving skills and attention to detail. Strong communication and collaboration skills.Preferred Qualifications: Experience with real-time data processing frameworks like Apache Flink or Apache Storm. Knowledge of machine learning frameworks and libraries. Experience with containerization and orchestration tools (e.g., Docker, Kubernetes). Certifications in big data technologies or cloud platforms. Thanks Debasish Pattnaik [email protected] www.mrtechnosoft.com Keywords: continuous integration continuous deployment Texas Virginia Big Data Engineer with extensive experience in PySpark Onsite at Dallas TX or Richmond VA [email protected] |
| [email protected] View all |
| Fri Jun 28 20:55:00 UTC 2024 |