| ONLY USC AND GC *exp: 15+ must* IMMEDIATE HIRING FOR SENIOR DATA ENGINEER @ (VA or TX or wi OR GA or NJ or PA or OH or NC or MO or ME or AL or LA or CT) at Remote, Remote, USA |
| Email: [email protected] |
| Notes from the Manager : (ONLY USC AND GC) We are looking for seasoned candidates. Please do not submit mid-level resources. Need 15+ years of experience only. Must have skills: - Designed and built framework-based API ingestion architecture. - Multi-Cloud Expertise: AWS cloud environments & cloud service models - PaaS, SaaS, IaaS, etc. across AWS, Azure, and GCP. - Use tools like Airflow for workflow automation and scheduling. - Proficiency in Python and Spark for data processing and manipulation. - Hands-on experience with Databricks for data analytics and pipeline creation. - Experience with Kafka or similar messaging queues for data ingestion and integration. - Strong SQL skills for database querying and manipulation. Job Title: Senior Data Engineer Location R/O This role will be on-site - Miramar, FL or Dallas, TX preferred, or near any CGI office in the US. (Please see the document attached for CGI locations.) Job Description We are Seeking a Sr. Data engineer who has designed and built framework-based API ingestion architecture for data extract and exchange and can successfully manage and enhance the data framework, ensuring secure and efficient data handling across various platforms and environments. Your future duties and responsibilities: Data Integration Platform Design and Development: Design, build and implement a framework-based API data ingestion architecture with a low-code/no-code interface for data ingestion and processing using reusable plug-and-play components. Develop frameworks to consume API components such as JSON payloads, CSVs, etc. Abstract data to build canonical models and data pipelines to maintain them. Integrate and process private data securely across multiple AWS accounts with compliance to all security protocols, including PgP encryption and encryption at rest. Implement data transformations and curate ingested data to ensure high-quality and usable datasets. Integration and Ingestion: Utilize Databricks and Informatica as unified data integration platforms. Leverage Kafka (or similar messaging queue) as an enterprise integration tool to ingest and integrate near real time and large volumes of diverse data from multiple sources. Build frameworks around APIs to bring in data from various sources efficiently. Security and Compliance: Ensure secure handling of Personally Identifiable Information (PII) data in AWS cloud environments. Understand and implement secure zones and mechanisms for PII data handling. Manage data securely across different storage solutions like RedShift and other data warehouses. Multi-Cloud Expertise: Demonstrate an understanding of different cloud service models - PaaS, SaaS, IaaS, etc. across AWS, Azure, and GCP. Develop solutions that are compatible with multi-cloud environments, ensuring seamless integration and operation. Continuous Development and Deployment: Implement continuous development, deployment, testing, integration, and monitoring practices. Use tools like Airflow for workflow automation and scheduling. Required qualifications to be successful in this role Technical Skills: Proficiency in Python and Spark for data processing and manipulation. Hands-on experience with Databricks for data analytics and pipeline creation. Experience with Kafka or similar messaging queues for data ingestion and integration. Strong SQL skills for database querying and manipulation. Familiarity with Airflow for workflow management. Cloud Expertise: Advanced level understanding of the different cloud service models - PAAS, SAAS, IAAS etc. across AWS, Azure and GCP In-depth knowledge of AWS, including security, data handling, and storage solutions like RedShift and S3 Understanding of multi-cloud architectures and best practices. Security and Compliance: Experience handling PII data with a strong understanding of encryption standards (PgP, encryption at rest). Knowledge of secure zones and mechanisms for data protection in cloud environments. Analytical and Problem-Solving Skills: Ability to abstract and transform data to meet business requirements. Strong problem-solving skills to troubleshoot and optimize data pipelines. -- Keywords: sthree rlang information technology green card Florida Texas ONLY USC AND GC *exp: 15+ must* IMMEDIATE HIRING FOR SENIOR DATA ENGINEER @ (VA or TX or wi OR GA or NJ or PA or OH or NC or MO or ME or AL or LA or CT) [email protected] |
| [email protected] View all |
| Mon Jun 17 22:15:00 UTC 2024 |