| Onsite - ETL Developer - Santa Clara, CA - Locals at Santa Clara, California, USA |
| Email: [email protected] |
| From: Usha Geethari, StellarIT Solutions [email protected] Reply to: [email protected] Hi, This is Usha form Stellar IT Solutions! You intrigued us in your profile; hence I have a Great Opportunity for you with our direct client so kindly view the below job description and let me know your interest to proceed further. Title: ETL Developer Location: Santa Clara, CA (Only Locals) Duration: Long Term Job Description Must Skills: Scala, Streamsets, Upgrade, ETL, Spark, Research, kafka, PySpark, Python, AWS, Lambda, Performance Tuning. Domain: Banking, financial services and insurance (BFSI) Job Description: Migrate to the new version from 4.7 to 5.6 on prem. Create, maintain, and manage ETL/ELT pipelines. Create, maintain, and manage data transformations. Maintain/update documentation. Create, maintain, and manage data pipeline schedules. Monitor data pipelines. Create, maintain, and manage data quality gates (Great Expectations) to ensure high data quality. Create, maintain, and manage Spark Structured Steaming jobs, including using the newer Delta Live Tables. Research existing data in the data lake to determine best sources for data. Create, manage, and maintain Kafka Streams queries/code and enable pipelines using pyspark stream sets. Maintain and update Python-based data processing scripts executed on AWS Lambdas Unit tests for all the Spark, Python data processing and Lambda codes Maintain PCIS Reporting Database data lake with optimizations and maintenance (performance tuning, etc) Understanding of concepts and implementation strategies for different incremental data loads such as tumbling window, sliding window, high watermark, etc. Understanding of streaming data pipelines and batch systems Familiarity with CI/CD pipelines, containerization, and pipeline orchestration tools such as Airflow, Prefect, etc a bonus but not required Revett the existing solutions and extend it as per the need of the organization Thanks & Regards, Usha R Senior Talent Acquisition https://www.linkedin.com/in/usha-g-3a2ba2101 Work: 240-774-0203| Email: [email protected] Rockville, MD | McLean, VA | Palo Alto, CA StellarIT.com | StellariDeaLabs.com Keywords: continuous integration continuous deployment rlang information technology California Maryland Virginia |
| [email protected] View all |
| Tue Jan 16 23:55:00 UTC 2024 |