| Urgent Requirement- Big Data Architect with Devops -New York City, NY (Hybrid) - Experience 13 Plus Year at York, New York, USA |
| Email: [email protected] |
| Hello , Greetings, We have an opening Big Data Architect with DevOps . Below are the requirement details, just go through it & if you feel interested, revert back to me with your updated resume Your earliest reply is highly appreciated. Role: Big Data Architect with DevOps Location: New York City, NY (Hybrid) Duration: Long Term Job description: Roles and responsibilities Design and implement various development, testing, automation, CI/CD tools, and cloud infrastructure. Set up tools and required infrastructure across various services such Snowflake, AWS, SnapLogic and Alation. Monitor product roadmaps for different providers and components and recommend enhancements according to product updates. Ability to architect and engineer timebound sandboxes for PoC and innovation. Define test, release, update, and support processes for operation of data platform infrastructure. Monitor development processes for adherence to standards. Deploy and maintain cybersecurity measures, vulnerability assessment and risk management, i.e. Snowflake dashboards for risk and audit, automated code vulnerability scanning tools like Snyk, etc. Perform root cause analysis of production errors and resolve technical issues. Mentor and guide other developers in best practices, deployment processes. Perform coaching, peer reviews and brown bags. Preferred Skills Strong troubleshooting and debugging skills. Thorough understanding of tradeoffs of leveraging various components. Strong experience with various open source tools and frameworks. Experience setting up complex build pipelines with approvals, version tagging, readable deployment plans and artifacts. 10+ years of experience in automating infrastructure setup and application deployment At least 5+ years of setting up and using AWS components, EMR, EC2, Lambda, ECR, ELB, Route53, S3. Strong Experience with python, Terraform, YAML, JSON, TOML, Snowsql, Flyway, preferred. Experience with containers and orchestration utilizing Docker or ECS. Experience with Databricks and Databricks terraform preferred. Education/Certifications: One of the following certifications preferred AWS Certified DevOps Engineer, AWS Certified SysOps Administrator, AWS Certified Solutions Architect, Snowflake SnowPro Bachelors degree in computer science or software engineering DevOps / Application Architect Supporting Projects DevOps Foundation & Framework Develop infrastructure as code templates for server builds. Set up vulnerability, static analysis tools in build pipelines. Integration Jira, ServiceNow and Azure DevOps to unify change management and approvals to make release process more efficient and streamlined. Extend use of data quality / integrated regression testing features using Great Expectations. Creating DevOps templates for the deployment of various flavors of RDS to migrate away from on premise manual Microsoft SQL Server deployments. Creating DevOps templates for the deployment of scripts to legacy Microsoft SQL platforms. Creating DevOps templates for the deployment of various AWS services that delivery teams can deploy on demand. Services to be prioritized based on stakeholder feedback. Examples include: EMR / Spark ELB / ALB Lambda SQS Event Bridge SNS Create DevOps templates for extending the deployment of various Snowflake services: Tasks Streams Streaming architecture Notifications Extend existing sandbox templates Provide training and knowledge sharing on use of GIT and DevOps frameworks. Create methodology for KPI reporting and measurement: How much time do developers save by removing dependencies on central teams to automate provisioning How much more quickly can new workloads be deployed by shifting left security Develop DevOps templates for deploying message bridge pattern for enabling eventual decommission of ESB technology. A message bridge allows for adapters leveraging different message bus and queuing technologies to publish and subscribe to updates without a big bang cutover enabling smoother transition for individual development teams. Perform analysis on whats required to achieve future state vision around source control (migration away from on premise Azure DevOps) Coinvest / Corporate Service Reporting Design and optimization of Aurora DevOps templates for deployment and scaling ACUs on demand. Build core infrastructure components based on future state reporting vision. Posit Connect Workbench GPE Design and deployment of automation to support Snowflake Container Services and Posit Workbench Supports Global Research dashboarding. Increases performance, security and speed of development using R. Removes need for installing locally and whitelisting lots of tools. Data stays in Snowflake improving performance and security. Alation Deploy infrastructure and API components that support Data Governance / Management: Data quality and health integration Snowflake tags and policies Metadata extraction Virtual data sources (API) DevOps Platform Management Update versions of different components as needed to maintain and enhance functionality and mitigate vulnerabilities: Terraform providers for Snowflake and AWS. Great Expectations (data quality). Flyway (database deployments). Coach developers on the use of GIT and provided templates. Create new repositories as needed. Manage repository permissions and branch control policies. Operational, Audit and Security Dashboards Extend development of security and risk dashboards Service accounts without network policies. Accounts using local username and password. Data exfiltration. Cross segment grants and data access. Skill Importance (1-10) Snowflake 8 SnapLogic 4 AWS (various services) 10 Terraform 10 Databricks 5 Data Quality / Great Expectations 6 Source control / GIT 10 Message bus and queuing architecture 9 Vulnerability / static analysis / DevSecOps 9 Bash / scripting 9 Flyway 6 Python 8 Best Regards Raja Technical Recruiter ______________________________________________________________ Tanisha Systems Inc. Email: Raja @tanishasystems.com Web: www.tanishasystems.com Address: 99 Wood Ave South Suite # 308,Iselin, NJ 08830 E-Verify, Certified Minority Business Enterprise MBE, Dun & Bradstreet CREDIBILITY CORP Certified -- Keywords: continuous integration continuous deployment sthree rlang information technology golang New Jersey New York Urgent Requirement- Big Data Architect with Devops -New York City, NY (Hybrid) - Experience 13 Plus Year [email protected] |
| [email protected] View all |
| Thu Jun 13 19:18:00 UTC 2024 |