| GCP Data Engineer with Spark Scala and LARGE SCALE E-COMMERCE EXPERIENCE : Remote: 70 per hr at Commerce, Georgia, USA |
| Email: [email protected] |
| From: Jay, Brillius [email protected] Reply to: [email protected] GCP Data Engineer with Spark Scala and L ARGE SCALE E-COMMERCE EXPERIENCE Duration: 6+ Month(s) Location: Remote VISA restrictions: N/A Pay: $70/hr C2C Please only submit candidates that have an active Linkedin account About The Role The Data Engineer contractor role will be a project based role focused on migrating data pipelines from legacy infrastructure and frameworks such as Scalding to more modern infrastructure we support such as Spark Scala. This role will be responsible for: Analyzing existing data pipelines to understand their architecture, dependencies, and functionality. Working closely with data engineers to develop a migration strategy for converting pipelines from the current framework to the target framework. Designing, building, and testing new data pipelines in the target framework, ensuring they meet or exceed existing performance and reliability standards. Debugging and troubleshooting issues that arise during the migration process, working collaboratively with cross-functional teams to resolve them quickly. Communicating progress, challenges, and timelines to stakeholders on a regular basis. Must-Haves: MUST HAVE LARGE SCALE E-COMMERCE EXPERIENCE (examples: Paypal, eBay, Amazon, TEMU, Walmart etc.) Scala programming language expertise Spark framework expertise Experience working with BigQuery Familiarity scheduling jobs in Airflow Fluency with Google Cloud Platform, in particular GCS and Dataproc Scalding framework expertise . Java background is also a big bonus. Nice-to-Haves Python programming language fluency Scalding framework fluency Pyspark framework fluency Dataflow(Apache Beam) framework fluency Keywords: |
| [email protected] View all |
| Tue Mar 05 00:14:00 UTC 2024 |