| Data Engineer with spoark 10+ years at Remote, Remote, USA |
| Email: [email protected] |
| From: Hari krishna, 3K TECHNOLOGIES [email protected] Reply to: [email protected] Jobtitle: Data Engineer with spoark 10+ years Location : Remote Duration: 12+ Months JD Minimum Requirements: 10 years experience in data engineering. 2+ years working with Spark. Proficiency in SQL, Java, Scala, and Spark. Experience writing unit tests for Spark/ETL code and working with pipeline (CI/CD) deployments and JAR development. Expertise in writing efficient performant ETL code and providing meaningful code and architecture reviews. Experience with star schema and medallion architecture. Experience working with large, distributed teams on data engineering projects. Passion for mentorship, documentation, and raising the technical bar for the team. Experience with cluster configuration and management. Must be able to work effectively in a remote distributed team using meetings, calls, emails, chats, wikis and documentation. Must be comfortable managing timelines, making decisions independently, and building consensus where needed. Must be comfortable navigating and cutting through ambiguity and uncertainty to make iterative progress. Must be comfortable working with orchestration tools like Azure Data Factory and Jenkins as needed. Experience with Databricks a plus. Experience with Azure a plus. Experience with healthcare data and specs (ex: FHIR) a plus. Primary Responsibilities: Design, develop, test, deploy and monitor ETL pipelines and orchestration. Improve code and architecture. Work on ETL for analytic dashboards as well as platform components with an aim for modularity and reusability. Provide code review and architecture feedback. Mentor teammates and create documentation and patterns. Work closely with architects, client SMEs, and Technical Product Managers to understand requirements and underlying client data. Keywords: continuous integration continuous deployment |
| [email protected] View all |
| Thu Dec 14 23:29:00 UTC 2023 |