| Data Engineer at Remote, Remote, USA |
| Email: [email protected] |
| From: Ammy, Resource Logistics [email protected] Reply to: [email protected] 8+ years experience in Apache Sprk, Structured Streaming, Azure, data modeling or data engineering work Architecting, building, and maintaining end-to-end, high-throughput data systems and their supporting services. Designing efficient data structures and database schemas and working with distributed systems architecture Incorporating data processing and workflow management tools into pipeline design. Utilizing a variety of data stores, including data warehouses, relational database systems, in-memory caches, and searchable document databases. Experience working with large data sets using SQL/Azure Data Lake/Spark, etc., to derive actionable insights is highly desirable. Experience in working with KUSTO ( Azure Data Explorer) Developing for continuous integration and automated deployments. Using profiling tools, debugging logs, performance metrics, and other data sources to make code- and application-level improvements Responsibilities Ship high-quality, well-tested, secure, and maintainable code. Design, develop, and maintain data pipelines and back-end services for real-time decisioning, reporting, optimization, data collection, and related functions. Manage automated unit and integration test suites. Work collaboratively and communicate effectively with a small, motivated team of engineers and product managers. Experiment with and recommend new technologies that simplify or improve the tech stack. Participate in an on-call rotation and work occasional off-hours. Keywords: |
| [email protected] View all |
| Thu Nov 16 19:23:00 UTC 2023 |