| Data Engineer - Day 1 Onsite in Holmdel NJ or Bethlehem, PA at Bethlehem, Connecticut, USA |
| Email: [email protected] |
| From: Laxmi Kumari, Smart IT Frame [email protected] Reply to: [email protected] Dear Vendor, Please help me with some appropriate profiles for the below closable job position. Role: Data Engineer Location: Day 1 Onsite in Holmdel NJ/Bethlehem, PA Duration: 12+ months Job Description: 5+ years of experience with Bachelors / masters degree in computer science, Engineering, Applied mathematics or related field. Extensive hands-on development experience in writing complex SQL queries and good to have Python/Bash scripting knowledge. Extensive Experience in performance optimization of data pipelines. Experience working with cloud data warehouse and data lake platforms like Databricks, Redshift or Snowflake. Familiarity with building and deploying scalable data pipelines to develop and deploy Data Solutions using Python, SQL, PySpark. Extensive experience in all stages of software development and expertise in applying software engineering best practices. Experience in developing end-to-end orchestration layer for data pipelines using frameworks like Apache Airflow, Prefect, Databricks Workflow. Experience in creating and configuring continuous integration/continuous deployment using pipelines to build and deploy applications in various environments and use best practices for DevOps to migrate code to Production environment. Ability to investigate and repair application defects regardless of component: front-end, business logic, middleware, or database to improve code quality, consistency, delays and identify any bottlenecks or gaps in the implementation. Ability to write unit tests in SQL. Additional Qualifications (nice to have): Experience in using and implementing data observability platforms like Monte Carlo Data, Metaplane, Soda, bigeye or any other similar products.Expertise in debugging issues in Cloud environment by monitoring logs on the VM or using AWS features like Cloudwatch. Experience with DevOps tech stack like Jenkins and Terraform. Experience working with concept of Observability in software world and experience with tools like Splunk, Zenoss, Datadog or similar.Experience in developing and implementing Data Quality framework either home grown or using any open-source frameworks like Great Expectations, Soda, Deequ. Ability to learn and adapt to new concepts and frameworks and create proof of concept using newer technologies.Ability to use agile methodology throughout the development lifecycle and provide updates on regular basis, escalating issues, or delays in a timely manner. Familiar with RESTful Webservices (REST APIs) to be able to integrate with other services. API Gateways like APIGEE to secure webservice endpoints. Data pipelines, Concurrency and parallelism. Best Regards, Laxmi Kumari Smart IT Frame LLC laxmi @smartitframe.com www.smartitframe.com -----WBENC------ Keywords: information technology New Jersey Pennsylvania |
| [email protected] View all |
| Tue Feb 06 01:27:00 UTC 2024 |