| Big Data Engineer - Ex- Amex, MasterCard, Visa, Paypal Consultants only at Remote, Remote, USA |
| Email: [email protected] |
| From: ayush, Scalable Systems [email protected] Reply to: [email protected] Big Data Engineer Job Description We are seeking a highly skilled Big Data Engineer to join our team in Phoenix, AZ. The ideal candidate will have a strong foundation in data warehousing, SQL, and a deep understanding of big data technologies. This role requires hands-on experience with public cloud platforms, specifically GCP, and a proven track record of success in designing and developing big data applications. Responsibilities: Design, develop, and implement big data solutions using technologies such as MapReduce, Spark (PySpark), and Hive. Build and optimize data pipelines to efficiently extract, transform, and load data into data warehouses. Develop and maintain complex SQL queries and data models. Collaborate with cross-functional teams to understand business requirements and translate them into technical solutions. Leverage cloud-based data warehousing services (GCP, Azure, AWS) to build scalable and cost-effective data solutions. Implement CICD pipelines, automated testing, and DevOps practices to ensure efficient and reliable software delivery. Participate in Agile development methodologies (Scrum) and contribute to team success. Required Skills: Expert-level proficiency in SQL and data warehousing concepts. Hands-on experience with public cloud data warehousing platforms (GCP, Azure, AWS) with a strong preference for GCP. Mandatory experience with MapR. Strong programming skills in Python or Java. In-depth knowledge of big data technologies including Spark (PySpark) and Hive. Experience with CICD pipelines, automated testing frameworks, DevOps, and source code management tools (XLR, Jenkins, Git, Maven). Excellent communication and analytical skills with the ability to effectively present complex technical information. Familiarity with Agile and Scrum methodologies. Preferred Qualifications: GCP certification Experience with additional big data technologies (Hadoop, Kafka, etc.) Knowledge of data visualization tools (Tableau, PowerBI) If you are a passionate Big Data Engineer with a proven track record of success and a desire to work on challenging projects, we encourage you to apply. Keywords: Big Data Engineer, MapReduce, Spark, PySpark, Hive, SQL, data warehousing, GCP, Azure, AWS, cloud computing, data pipeline, Python, Java, CICD, DevOps, Agile, Scrum, data engineering, data analysis. Sources and related content Keywords: Arizona Big Data Engineer - Ex- Amex, MasterCard, Visa, Paypal Consultants only [email protected] |
| [email protected] View all |
| Thu Aug 15 00:31:00 UTC 2024 |