Note: The job is a remote job and is open to candidates in USA. YipitData is the leading market research and analytics firm for the disruptive economy, seeking a Data Engineering Manager to join their team. The role involves designing and maintaining large-scale data pipelines, establishing best practices for ETL, and collaborating with cross-functional teams to deliver scalable data solutions.
Responsibilities
- Own the design, build, and optimization of end-to-end data pipelines that power our vendor universe
- Establish and enforce best practices in data modeling, orchestration, and system reliability
- Collaborate with product, engineering, and business stakeholders to translate requirements into robust, scalable data solutions
- Work extensively with Databricks and Airflow for large-scale data processing and orchestration
- Troubleshoot and resolve complex pipeline issues to ensure reliability and performance
- Contribute to the team’s technical strategy, helping drive improvements in scalability, performance, and efficiency
- Lead, mentor, and support engineers through challenges, code reviews, and project execution
Skills
- 6+ years of professional experience in Data Engineering or equivalent technical roles (e.g., data architecture, big data development, or ETL engineering)
- 2+ years of managerial experience, including mentoring, team leadership, and supporting delivery
- Strong expertise in SQL and distributed data systems
- Proficiency with PySpark and Databricks for processing and scaling large datasets
- Hands-on experience with Airflow for pipeline orchestration (Dagster/dbt a plus)
- Proven track record of delivering in fast-paced, deadline-driven environments with minimal oversight
- Strong problem-solving skills and ability to translate business needs into scalable technical solutions
- Excellent communication and collaboration skills with both technical and non-technical stakeholders
- Experience leveraging AI/ML models, vector search, or Elasticsearch to enhance data pipelines
- Familiarity with Django or similar web frameworks to support data-driven applications
Benefits
- Flexible work hours
- Flexible vacation
- A generous 401K match
- Parental leave
- Team events
- Wellness budget
- Learning reimbursement
- Equity
Company Overview
YipitData is a market research firm that provides detailed insights and reports based on alternative data. It was founded in 2007, and is headquartered in New York, New York, USA, with a workforce of 501-1000 employees. Its website is https://yipitdata.com.