Data Engineer Manager

The Data Engineer is responsible for building and maintaining scalable data pipelines, data models, and data platforms to support analytics, business intelligence, and machine learning workflows across the organization. Key Responsibilities • Design and develop data pipelines for batch and real time processing (ETL/ELT). • Build and maintain data warehouse/data lake solutions (Snowflake, BigQuery, Redshift, Hive). • Develop processing jobs using Airflow, Spark, Flink, Kafka. • Perform data cleaning, standardization, and data quality management. • Collaborate with analysts, data scientists, and product teams to deliver reliable datasets. • Optimize data models, query performance, and storage efficiency. • Support data governance, metadata management, and security compliance. • Troubleshoot pipeline issues and ensure high availability of data services. Requirements • 8 years of experience as a Data Engineer or similar role. • Strong SQL skills and proficiency in Python/Java/Scala. • Experience with distributed processing frameworks (Spark/Flink/Kafka). • Familiarity with cloud platforms (AWS/GCP/Azure) and their data services. • Understanding of data modeling, warehouse/lake architecture, and API integrations. • Strong communication and cross team collaboration skills. Nice to Have • Experience with BI tools (Tableau, Power BI). • Experience with CI/CD, containerization (Docker/Kubernetes). • Exposure to ML pipelines or MLOps. • Experience in large scale enterprise data environments.

职位描述

Daisy Liu