| Remote Job Title : Data Engineer USC AND GC at Remote, Remote, USA |
| Email: [email protected] |
| Job Title : Data Engineer Visa: USC, GC only MOI: Skype Rate: $65/hr c2c Client : TRP Location: Can be fully remote but EST Job Details : Must Have: Python PySpark Database technologies (AWS or similar) AWS Glue Data Engineer This position will focus on developing various database and data loading/transformation efforts. Initiatives could include multiple source files and/or files sourced both internally and externally. Candidate is required to interact with databases using AWS Lambda, AWS Glue (for transformations) Python and Spark. Possible work leveraging SnapLogic for ETL, depending upon use case. Work will be applicable to multiple applications. Knowledge of automated testing of data a plus. Responsibilities: Design and built systems for analytical reporting Analyze data and design ETL frameworks using Python and Spark technologies Collaborate with business partners to gather requirements and translate that to technical specifications use Adhere to lean agile methodologies Production Support including trouble shooting and problem-solving capabilities Implement data pipelines to meet business requirements and adhere to best practices. Required Skills: Proficient in Python, PySpark and database technologies Experience with various AWS services like AWS Glue, IAM, RDS, S3, SQS, SNS, Lambdas, Cloud Formation, Cloud Watch, Event Bridge, CLI Experience with Unix/Linux Experience with SQL Experience with source code control, preferably Git. Experience in CI (Continuous Integration)/ CD (Continuous Delivery) software development pipeline stages like Commit, Build, Automated Tests, and Deploy Desired Skills: Experience in creating AWS Glue components - Jobs, Triggers, Workflows Experience with AWS CLI Experience working with databases such as Snowflake and/or Aurora Experience with Hashicorp tool suite - Terraform, Vault Experience with soap and rest services. Experience working with XML, JSON, YAML Experience with logging tools such as Cloudwatch and Splunk -- Keywords: continuous integration continuous deployment sthree information technology green card Remote Job Title : Data Engineer USC AND GC [email protected] |
| [email protected] View all |
| Thu Mar 28 21:42:00 UTC 2024 |