| DataBricks Data Engineer| Locals to Chicago, IL area candidates at Chicago, Illinois, USA |
| Email: [email protected] |
| Role- DataBricks Data Engineer Location- Locals to Chicago, IL area candidates Duration- 3+ months Contract to Hire Hybrid contract Visa- GC and USC only Skills/Experience needed: BS Degree with 7+ years of experience in batch and streaming ETL using Databricks, Spark, Python, Scala, and Snowflake. MUST have 5+ years of recent experience with Snowflake and Databricks 5+ years of experience with orchestrating and implementing pipelines using Databricks Workflows. 3+ years of experience prepping structured and unstructured data for data science models. 3+ years of experience with containerization and orchestration technologies (Docker, Kubernetes) Experience with shell scripting in Bash, Unix, Windows shell. Experience working in vector databases like Milvus, Pinecone, Weaviate. Experience using machine learning in data pipelines to discover, classify, and clean data. Implemented CI/CD with automated testing in Jenkins, GitHub Actions, Gitlab, CI/CD Experience with AWS Services such as Glue, Athena, Lambda, S3, and DynamoDB Experience implementing data management life cycle, using data quality functions like standardization, transformation, rationalization, linking, and matching. Design and implement technical solutions and processes to ensure data reliability and accuracy. Develop data models and mappings and build new data assets required by data science teams. Perform exploratory data analysis on existing products and datasets. Collaborate with stakeholders, including team and product managers, to create secure and efficient data products. Ensure that data processing pipelines and ETL jobs are designed for scalability, using distributed technologies to handle increasing data volumes efficiently. Work within an Agile delivery / DevOps methodology to deliver product increments in iterative sprints. Thanks & Regards, Gaurav Jangid Senior Technical Recruiter Email- [email protected] - https://www.linkedin.com/in/gaurav-jangid-97a5241ab/ 806, New Castle , Wilmington, DE, US, 19801 -- Keywords: continuous integration continuous deployment sthree information technology green card Delaware Illinois DataBricks Data Engineer| Locals to Chicago, IL area candidates [email protected] |
| [email protected] View all |
| Tue Sep 24 19:47:00 UTC 2024 |