| Lead Data Engineer at Remote, Remote, USA |
| Email: [email protected] |
| From: Chandan Rai, Vaaridatech [email protected] Reply to: [email protected] Role:-Lead Data Engineer Location:-Onsite VA Capital one formers 1 step interview Skills' Details 2 - 4 years PYTHON 2-4 years in Scala 2-4 years in Spark 2-4 years of AWS - Ec2, EMR, ECS, S3, SNS, Lambda, RDS, VPC Knowledgeable w/ SDLC best practices - CICD, unit testing, functional testing **Need experience working w/ legacy to serverless Secondary Skills - Nice to Haves Sql Data snowflake Job Description The Enterprise Detection group at Capital One is looking for strong Data Engineers with Python, Scala and Spark, that also have robust experience with AWS and CICD, as well as unit and functional testing. Must be able to quickly adapt, contribute to other initiatives. Additional Skills & Qualifications 2-4 years data warehousing and relational database experience (Snowflake, Oracle, or PostgresSQL) 2-4 years of exp. w/ Customer Services (Amazon EC2, Amazon FallGate) Employee Value Proposition (EVP) Work Environment All consultants/teams will sit out of the McLean or NYC offices at Capital One. Business Drivers/Customer Impact ENTERPRISE DETECTION Will be working on the MetaBot platform, a key piece in C1's journey to get the most value out of data. It will make data easy to govern, easy to find and easy to use. MetaBot communicates with Catalog, where all datasets need to be registered. MetaBot will solve the problem of "How do we get all different data sets in the company into the catalog, and get them registered" MetaBot's main goal is to go out and find all unregistered data sets and bring them to the data owners for them to take action, meaning: do I want to register fully in catalog, or just clean up/create exclusion or deletion rules There are still millions of unregistered data sets, and they're not sure who to present to. Right now, users are presented with UI with millions of results. It's so manual, error-prone, and tedious. Can't keep up with creation of new data sets. Goal for metabot is to autoregister - take out all manual intervention and make it so that whatever Metabot detects goes into Autoregistration services, fills in data, and registers it in the catalog. Why is the position open(provide details) External Communities Job Description One of our financial services customers is looking for strong Java backend engineers with experience in following areas: -2 - 4 years Python Development -2 - 4 years in Scala and Spark -2 - 4 years of AWS - Ec2, EMR, ECS, S3, SNS, Lambda, RDS, VPC -Knowledgeable w/ SDLC best practices - CICD, unit testing, functional testing Keywords: user interface sthree information technology golang Virginia Lead Data Engineer [email protected] |
| [email protected] View all |
| Fri Oct 11 21:07:00 UTC 2024 |