| Senior Data Engineer (Databricks, Python, PySpark) at Remote, Remote, USA |
| Email: [email protected] |
| From: vivek, vyzeinc [email protected] Reply to: [email protected] Job Description -Senior Data Engineer (Databricks, Python, PySpark)Fully Remote - Might be an F2F interview so candidate should be ready as per the client need (Near to candidate location of client) Video/Phone 6-12 MonthMust have Working proficiency in using Python/Scala, Spark (tuning jobs), SQL, Hadoop platforms to build Big Data products & platforms. Good understanding of Databricks security, clusters, user management, deployment and performance tuning. Strong PySpark skills Good programming skills in Java and spring boot and Junit. Knowledge in software development test approaches & frameworks Familiarity with RESTful APIs and micro-services architectures Experience in working with CI/CD Experience in working with SQL database like Postgres, Oracle Preferably with hands-on experience with Hadoop big data tools (Hive, Impala, Spark) Experience with data pipeline and workflow management tools: NIFI, Airflow. Comfortable in developing shell scripts for automation. Good troubleshooting and debugging skills. Proficient in standard software development, such as version control, testing, and deployment Demonstrated basic knowledge of statistical analytical techniques, coding, and data engineering Ability to quickly learn and implement new technologies Ability to Solve complex problems with multi-layered data sets Ability to innovate and determine new approaches & technologies to solve business problems and generate business insights & recommendations. Ability to multi-task and strong attention to detail Flexibility to work as a member of a matrix based diverse and geographically distributed project teams Good communication skills - both verbal and written - and strong relationship, collaboration skills, and organizational skillsNice to have Experience with performance Tuning of Database Schemas, Databases, SQL, ETL Jobs, and related scripts Experience in working with Cloud APIs (e.g., Azure, AWS) Experience participating in complex engineering projects in an Agile setting e.g. Scrum Keywords: continuous integration continuous deployment |
| [email protected] View all |
| Mon Nov 20 23:59:00 UTC 2023 |