| Senior Data Engineer (AI|ML) || Charlotte, NC at Charlotte, North Carolina, USA |
| Email: [email protected] |
| From: Ishavdeep Singh, Cloud Think Technologies [email protected] Reply to: [email protected] Responsibilities: As a Senior Data Engineer, you will be responsible for designing, implementing, and maintaining scalable data processing solutions and machine learning pipelines. Your primary focus will be on integrating AI services within the Google Cloud Platform (GCP) ecosystem, developing monitoring, and logging solutions, and ensuring data quality and security. Key Responsibilities: AI Integration with GCP: Integrate Vertex AI with BigQuery for training machine learning models. Deploy machine learning models using Vertex AI, including exporting, versioning, and monitoring. Automate ML workflows using Vertex AI Pipelines Implement data versioning and tracking using Vertex AI Dataset service and metadata stores. Set up monitoring and logging for AI models deployed on Vertex AI using Google Cloud Monitoring and Logging. Vertex AI and Gemini Integration: Integrate Gemini with Vertex AI for enhanced model training and predictions. Use Gemini for feature engineering in Vertex AI workflows. Manage model lifecycle and version control with Vertex AI and Gemini integration. Datalake with BigQuery: Design scalable data lake architectures using BigQuery. Optimize query performance in a BigQuery-based data lake through partitioning, clustering, and materialized views. Ingest data from various sources (e.g., Kafka, Pub/Sub) into BigQuery. Handle data security and access control in a BigQuery data lake. Implement data partitioning and clustering for efficient data retrieval in BigQuery. Required Skills and Experience: Extensive experience with GCP services, including Vertex AI, BigQuery ML, and AI Platform Pipelines. Strong understanding of data lake architectures and scalable data processing solutions. Familiarity with data versioning, metadata management, and data quality assurance practices. Excellent problem-solving skills and ability to handle complex data engineering challenges. Preferred Qualifications: Advanced degree in Computer Science, Data Engineering, or a related field. Certifications in Google Cloud Platform Knowledge of Kubeflow Pipelines and CI/CD practices for ML workflows. Keywords: continuous integration continuous deployment artificial intelligence machine learning Senior Data Engineer (AI|ML) || Charlotte, NC [email protected] |
| [email protected] View all |
| Thu Jul 25 01:02:00 UTC 2024 |