Home

Data Engineer - Day 1 Onsite in Holmdel NJ at Day, New York, USA
Email: [email protected]
From:

Laxmi Kumari,

Smart IT Frame

[email protected]

Reply to:   [email protected]

Dear Vendor,

Please help me with some appropriate profiles for the below closable job position.

Role: Data Engineer 

Location: Day 1 Onsite in Holmdel NJ

Duration: 12+ months

Job Description:

5+ years of experience with Bachelors / masters degree in computer science, Engineering, Applied mathematics or related field.

Extensive hands-on development experience in writing complex SQL queries and good to have Python/Bash scripting knowledge.

Extensive Experience in performance optimization of data pipelines. Experience working with cloud data warehouse and data lake platforms like Databricks, Redshift or Snowflake.

Familiarity with building and deploying scalable data pipelines to develop and deploy Data Solutions using Python, SQL, PySpark. Extensive experience in all stages of software development and expertise in applying software engineering best practices.

Experience in developing end-to-end orchestration layer for data pipelines using frameworks like Apache Airflow, Prefect, Databricks Workflow.

Experience in creating and configuring continuous integration/continuous deployment using pipelines to build and deploy applications in various environments and use best practices for DevOps to migrate code to Production environment. Ability to investigate and repair application defects regardless of component: front-end, business logic, middleware, or database to improve code quality, consistency, delays and identify any bottlenecks or gaps in the implementation. Ability to write unit tests in SQL.

Additional Qualifications (nice to have):

Experience in using and implementing data observability platforms like Monte Carlo Data, Metaplane, Soda, bigeye or any other similar products.Expertise in debugging issues in Cloud environment by monitoring logs on the VM or using AWS features like Cloudwatch. Experience with DevOps tech stack like Jenkins and Terraform. Experience working with concept of Observability in software world and experience with tools like Splunk, Zenoss, Datadog or similar.Experience in developing and implementing Data Quality framework either home grown or using any open-source frameworks like Great Expectations, Soda, Deequ. Ability to learn and adapt to new concepts and frameworks and create proof of concept using newer technologies.Ability to use agile methodology throughout the development lifecycle and provide updates on regular basis, escalating issues, or delays in a timely manner.

Familiar with

RESTful Webservices (REST APIs) to be able to integrate with other services.

API Gateways like APIGEE to secure webservice endpoints.

Data pipelines, Concurrency and parallelism.

Best Regards,

Laxmi Kumari

Smart IT Frame LLC

laxmi
@smartitframe.com

www.smartitframe.com

-----WBENC------

Keywords: information technology New Jersey
[email protected]
View all
Tue Feb 27 23:20:00 UTC 2024

To remove this job post send "job_kill 1161778" as subject from [email protected] to [email protected]. Do not write anything extra in the subject line as this is a automatic system which will not work otherwise.


Your reply to [email protected] -
To       

Subject   
Message -

Your email id:

Captcha Image:
Captcha Code:


Pages not loading, taking too much time to load, server timeout or unavailable, or any other issues please contact admin at [email protected]
Time Taken: 49

Location: , Indiana