| Data Engineer at Los Angeles, California, USA |
| Email: [email protected] |
|
http://bit.ly/4ey8w48 https://jobs.nvoids.com/job_details.jsp?id=3278020&uid=3823abd5e710424bb72e23a83fca3575 From: Dildaar, alltech [email protected] Reply to: [email protected] Job Title: Data Engineer Location : Los Angeles CA or NYC (Hybrid) Hire type : CTH Need Number Job Summary We are looking for an experienced Databricks Data Engineer with strong DevOps expertise to join our data engineering team. The ideal candidate will design, build, and optimize large-scale pipelines on the Databricks Lakehouse Platform on AWS, while driving automated CI/CD and deployment practices. This role requires strong skills in PySpark, SQL, AWS cloud services, and modern DevOps tooling. You will collaborate closely with cross-functional teams to deliver scalable, secure, and high-performance data solutions. Must Demonstrate (Critical Skills & Architectural Competencies) Designing and implementing Databricks-based Lakehouse architectures on AWS Clear separation of compute vs. serving layers Ability to design low-latency data/API access strategies (beyond Spark-only patterns) Strong understanding of caching strategies for performance and cost optimization Data partitioning, storage optimization, and file layout strategy Ability to handle multi-terabyte structured or time-series datasets Skill in requirement probing, identifying what matters architecturally A player-coach mindset: hands-on engineering + technical leadership Key Responsibilities 1. Data Pipeline Development Design, build, and maintain scalable ETL/ELT pipelines using Databricks on AWS. Develop high-performance data processing workflows using PySpark/Spark and SQL. Integrate data from Amazon S3, relational databases, and semi/nonstructured sources. Implement Delta Lake best practices including schema evolution, ACID, OPTIMIZE, ZORDER, partitioning, and file-size tuning. Ensure architectures support high-volume, multi-terabyte workloads. 2. DevOps & CI/CD Implement CI/CD pipelines for Databricks using Git, GitLab, GitHub Actions, or AWS-native tools. Build and manage automated deployments using Databricks Asset Bundles. Manage version control for notebooks, workflows, libraries, and environment configuration. Automate cluster policies, job creation, environment provisioning, and configuration management. Support infrastructure-as-code via Terraform (preferred) or CloudFormation. Keywords: continuous integration continuous deployment sthree California Data Engineer [email protected] http://bit.ly/4ey8w48 https://jobs.nvoids.com/job_details.jsp?id=3278020&uid=3823abd5e710424bb72e23a83fca3575 |
| [email protected] View All |
| 06:06 PM 08-Apr-26 |