| DataBricks Data Engineer| Locals to Chicago, IL area candidates or who will relocate from Day 1 for sure on own expenses. at Chicago, Illinois, USA |
| Email: [email protected] |
| Role- DataBricks Data Engineer Location- Locals to Chicago, IL area candidates or who will relocate from Day 1 for sure on own expenses. Duration- 3+ months Contract to Hire Hybrid contract Visa- GC and USC only Skills/Experience needed: BS Degree with 7+ years of experience in batch and streaming ETL using Databricks, Spark, Python, Scala, and Snowflake. MUST have 5+ years of recent experience with Snowflake and Databricks 5+ years of experience with orchestrating and implementing pipelines using Databricks Workflows. 3+ years of experience prepping structured and unstructured data for data science models. 3+ years of experience with containerization and orchestration technologies (Docker, Kubernetes) Experience with shell scripting in Bash, Unix, Windows shell. Experience working in vector databases like Milvus, Pinecone, Weaviate. Experience using machine learning in data pipelines to discover, classify, and clean data. Implemented CI/CD with automated testing in Jenkins, GitHub Actions, Gitlab, CI/CD Experience with AWS Services such as Glue, Athena, Lambda, S3, and DynamoDB Experience implementing data management life cycle, using data quality functions like standardization, transformation, rationalization, linking, and matching. Design and implement technical solutions and processes to ensure data reliability and accuracy. Develop data models and mappings and build new data assets required by data science teams. Perform exploratory data analysis on existing products and datasets. Collaborate with stakeholders, including team and product managers, to create secure and efficient data products. Ensure that data processing pipelines and ETL jobs are designed for scalability, using distributed technologies to handle increasing data volumes efficiently. Work within an Agile delivery / DevOps methodology to deliver product increments in iterative sprints. Thanks & Regards, Vishal Kashyap Technical Recruiter Email- [email protected] | 806, New Castle , Wilmington, DE, US, 19801 -- Keywords: continuous integration continuous deployment sthree information technology green card Delaware Illinois DataBricks Data Engineer| Locals to Chicago, IL area candidates or who will relocate from Day 1 for sure on own expenses. [email protected] |
| [email protected] View all |
| Tue Sep 24 19:44:00 UTC 2024 |