Dir Data Engineering (Databrick/ADO/Heathcare) NUMÉRO DE POSTE: 484414
Director, Data Engineering & Platform
Reports to: Vice President, Enterprise Data Platforms
Direct Reports: Approximately 10 Data Engineers
Peer Leaders: Director, Data Products; Director, Data Architecture
Core Platform: Databricks
Delivery Environment: Azure DevOps (ADO), Agile/Scrum
Role Summary
The Company is seeking a Director of Data Engineering & Platform to serve as the engineering and platform leader for the enterprise data organization. Reporting to the Vice President of Enterprise Data Platforms, this leader will own the teams, engineering practices, platform operations, and delivery processes responsible for turning the Company’s enterprise data strategy and architecture into scalable, production-ready solutions.
The Director will lead a team of approximately 10 data engineers operating across multiple Databricks environments and will be accountable for the reliability, scalability, performance, and disciplined delivery of the Company’s enterprise data platform.
This is a highly technical leadership role requiring deep Databricks experience combined with strong people leadership and delivery management capabilities. The Director will remain close enough to the technology to guide engineering decisions, challenge designs, establish standards, and troubleshoot complex platform issues while developing and leading a high-performing engineering organization.
The role will operate as a key member of the VP’s data leadership team alongside the Director of Data Products and Director of Data Architecture. Together, these leaders will connect business priorities, enterprise architecture, and engineering execution.
Key Responsibilities
Data Engineering Leadership
- Lead, develop, and manage a team of approximately 10 data engineers responsible for building and operating the Company’s enterprise data platform.
- Establish a strong engineering culture focused on quality, accountability, scalability, automation, and predictable delivery.
- Set engineering standards for data pipelines, transformations, testing, deployment, monitoring, documentation, and production support.
- Provide technical guidance and mentorship to engineers while developing future technical and team leaders.
- Maintain sufficient technical depth to participate in design reviews, challenge engineering approaches, and help resolve complex technical issues.
- Establish clear performance expectations and continually improve the productivity and capabilities of the engineering organization.
Databricks Platform Leadership
- Serve as the primary engineering leader for the Company’s Databricks ecosystem.
- Lead engineering and platform execution across multiple Databricks environments and workspaces.
- Establish consistent standards for development, testing, promotion, security, monitoring, and production operations across environments.
- Drive platform performance, scalability, reliability, observability, and cost optimization.
- Partner with Data Architecture on the implementation and evolution of the Company’s lakehouse architecture.
- Establish reusable frameworks and engineering patterns leveraging Databricks, Spark, Delta Lake, Python, and SQL.
- Support adoption and optimization of capabilities including Unity Catalog, Delta Lake, Databricks Workflows, MLflow, and broader Databricks tooling.
Deep, production-level Databricks experience is a must-have for this position.
Engineering Delivery & Agile Leadership
A major component of this role is creating a disciplined and highly effective engineering delivery organization.
- Own the engineering team's Agile/Scrum delivery process.
- Serve as a strong Scrum leader, driving sprint planning, backlog refinement, daily stand-ups, retrospectives, capacity planning, and delivery commitments.
- Establish clear sprint goals, engineering priorities, and measurable delivery expectations.
- Partner with Data Products to translate product priorities and business requirements into executable engineering work.
- Manage engineering capacity across platform initiatives, new development, technical debt, production support, and strategic projects.
- Identify and remove blockers that impact engineering velocity.
- Create visibility into engineering progress, dependencies, risks, and delivery commitments.
- Drive continuous improvement through sprint metrics, retrospectives, and engineering performance data.
Azure DevOps (ADO)
- Own and mature the engineering team's use of Azure DevOps (ADO) for planning, execution, and delivery management.
- Establish standards for Epics, Features, User Stories, Tasks, Bugs, and engineering backlogs.
- Ensure engineering work is properly defined, estimated, prioritized, assigned, and tracked.
- Build visibility into sprint velocity, capacity, backlog health, delivery risks, and dependencies.
- Leverage ADO to improve transparency and communication across Engineering, Data Products, Data Architecture, and executive leadership.
- Partner with engineering teams on CI/CD and automated deployment practices.
Strong hands-on Azure DevOps and Agile/Scrum leadership experience is required.
Data Pipeline Engineering
- Lead the design, development, and operation of scalable pipelines ingesting and transforming clinical, financial, operational, and enterprise data.
- Establish modern engineering standards for ETL/ELT, batch and near-real-time processing, orchestration, testing, observability, and code promotion.
- Ensure pipelines meet established standards for performance, reliability, data quality, and availability.
- Build reusable engineering frameworks that accelerate the integration of new data sources.
- Ensure engineering capabilities can scale to support the Company’s continued organizational growth and acquisition activity.
- Establish production support and incident-management practices for critical data pipelines and platform services.
Partnership Across the Data Organization
Director of Data Products
Partner closely with the Director of Data Products to translate business and clinical priorities into an executable engineering roadmap.
Data Products defines what needs to be delivered and why; Data Engineering determines how it gets built and delivered reliably at scale.
Director of Data Architecture
Partner closely with the Director of Data Architecture to implement the Company’s enterprise data architecture, governance, modeling, security, and data quality standards.
Data Architecture establishes the architectural direction and standards; Data Engineering turns those standards into production technology and engineering practices.
VP, Enterprise Data Platforms
Provide the VP with clear visibility into engineering capacity, platform health, delivery performance, technical risks, resource needs, and major dependencies.
The Director will be the VP’s primary leader for engineering execution and Databricks platform delivery.
Platform Reliability & Engineering Excellence
- Establish clear SLAs and operational standards for critical data pipelines and platform services.
- Implement monitoring, alerting, observability, and incident-response practices.
- Drive automated testing and data validation throughout the engineering lifecycle.
- Establish CI/CD and disciplined code-promotion processes across Databricks environments.
- Ensure appropriate source control, peer review, documentation, and release-management practices.
- Continuously identify opportunities to improve platform performance and engineering productivity.
- Balance new feature delivery with technical debt, platform maintenance, and reliability.
Security, Governance & Compliance
- Partner with Data Architecture, Security, and Governance teams to ensure engineering solutions meet enterprise security and data governance standards.
- Ensure appropriate access controls, auditability, encryption, and secure development practices across Databricks environments.
- Ensure data engineering practices support HIPAA and applicable healthcare regulatory requirements.
- Implement data lineage, quality, classification, and governance standards within engineering workflows.
Required Qualifications
- 8 years of progressive data engineering experience, including significant experience building enterprise-scale data platforms.
- 3 years of engineering leadership experience, ideally managing teams of 8–15 engineers.
- Deep hands-on Databricks experience is required, including experience operating complex production environments.
- Demonstrated experience working across multiple Databricks environments/workspaces.
- Strong expertise with Python, SQL, PySpark/Spark, and Delta Lake.
- Strong experience building production-grade ETL/ELT pipelines and modern lakehouse architectures.
- Experience with Databricks Workflows and/or orchestration technologies such as Azure Data Factory or Airflow.
- Strong understanding of CI/CD, automated testing, observability, source control, and modern engineering practices.
- Heavy Azure DevOps (ADO) experience, including backlog management, sprint planning, work-item management, capacity planning, and delivery reporting.
- Demonstrated experience leading Agile/Scrum engineering teams and driving disciplined sprint execution.
- Strong experience managing technical teams while remaining meaningfully engaged with architecture and engineering decisions.
- Experience operating in Azure or another major cloud environment.
Preferred Qualifications
- Healthcare or other highly regulated industry experience.
- Experience working with clinical, EHR, claims, financial, or healthcare operational data.
- Experience with Unity Catalog and enterprise Databricks governance.
- Experience with Azure Data Factory, Azure Data Lake, Azure Functions, dbt, and related Azure data technologies.
- Experience supporting analytics, data science, machine learning, or AI workloads.
- Experience supporting enterprise growth, acquisitions, or integration of new business units and data sources.
Leadership Profile
The ideal candidate is an engineering leader first.
This person should be equally comfortable leading a team of 10 engineers, reviewing a complex Databricks design, managing competing priorities in ADO, running a disciplined Scrum process, and explaining platform risks or delivery timelines to senior leadership.
This is not primarily a BI, analytics, data governance, or data strategy position. Those capabilities will sit alongside this role within the broader data organization.
The Director of Data Engineering & Platform owns the critical middle layer between strategy and execution: the people, platform, processes, and engineering discipline required to reliably turn the Company’s data strategy into production capabilities.
Critical Must-Haves
- Databricks: Deep, recent, production-level Databricks experience.
- Engineering Leadership: Proven success leading and developing a data engineering team of comparable scale.
- Platform Leadership: Experience owning engineering execution across multiple environments and complex enterprise data ecosystems.
- Azure DevOps: Heavy hands-on ADO experience managing engineering backlogs, sprints, capacity, dependencies, and delivery.
- Scrum: Strong Scrum leadership with the ability to create structure, accountability, and predictable engineering delivery.
- Technical Depth: Strong Python, SQL, Spark/PySpark, pipeline, orchestration, CI/CD, and modern data engineering knowledge.
- Execution: Ability to translate architecture and product priorities into scalable, production-ready engineering solutions.
Avis sur l’égalité en matière d’emploi
Vaco by Highspring est un employeur qui souscrit au principe de l’égalité en matière d’emploi et qui ne pratique aucune discrimination à l’égard des employés ou candidats à un emploi sur le plan de la race (notamment en ce qui a trait aux caractéristiques historiquement associées à la race, comme la texture des cheveux et la coiffure), couleur de la peau, le sexe (ce qui inclut l’état de grossesse ou les conditions connexes), la religion ou les croyances, l’origine nationale, la citoyenneté, l’âge, le handicap, le statut d’ancien combattant, l’adhésion à un syndicat, l’appartenance ethnique, le genre, l’identité de genre, l’expression de genre, l’orientation sexuelle, l’état civil, l’affiliation politique ou autre caractéristique protégée par les lois fédérales, d’État ou locales.
Vaco by Highspring et sa société mère, ses sociétés affiliées et ses filiales s’engagent à la pleine inclusion de toutes les personnes qualifiées. Dans le cadre de cet engagement, Vaco by Highspring et sa société mère, ses sociétés affiliées et ses filiales veilleront à ce que les personnes handicapées bénéficient de mesures d’adaptation raisonnables. Si des mesures d’adaptation raisonnables sont nécessaires pour vous permettre de participer au processus de candidature ou d’entretien d’embauche, d’exercer les fonctions essentielles du poste et/ou de bénéficier d’autres avantages et privilèges liés à l’emploi, veuillez contacter HR@vaco.com.
Vaco by Highspring souhaite également que tous les candidats sachent que la loi interdit la discrimination dans le milieu de travail.
Avis de représentation
En soumettant votre candidature à ce poste, vous convenez que vous accordez à Vaco by Highspring le droit exclusif de vous présenter en tant que candidat eu égard à l’offre d’emploi susmentionnée. Vous reconnaissez en outre que vous avez fourni des renseignements exacts sur vous-même et que vous n’avez pas volontairement dénaturé vos qualifications. Dans toute la mesure permise par la loi, vous vous engagez également à garder confidentiels tous les renseignements que vous obtiendrez de Vaco by Highspring concernant le poste et à ne divulguer les renseignements relatifs au poste que si cela est absolument nécessaire pour respecter une obligation afférente à votre candidature. En contrepartie, Vaco by Highspring s’engage à déployer des efforts raisonnables pour vous représenter dans le cadre de toute sollicitation, présélection en matière d’emploi et diffusion de CV.
Concernant les résidents de l’Ontario, Canada : d’après ce que Highspring retient de ses discussions avec son Client ce poste est actuellement vacant (soit par le truchement de Highspring en tant que sous-traitant, soit auprès du Client).
Avis de confidentialité
Vaco by Highspring, ainsi que sa société mère, ses sociétés affiliées et ses filiales (« nous », « notre/nos » ou « Vaco by Highspring ») respectent votre vie privée et s’engagent à vous informer de ses politiques en toute transparence.
- Les résidents de la Californie peuvent consulter l’Avis de collecte par les RH de Vaco by Highspring concernant les candidats et les employés de la Californie ici.
- Les résidents de la Virginie peuvent consulter nos politiques propres à l’État ici.
- Les résidents de tous les autres États peuvent accéder à nos politiques ici.
- Les résidents canadiens peuvent consulter nos politiques en anglais ici et en français ici.
- Les résidents des pays régis par le RGPD peuvent consulter nos politiques ici.
De plus, les candidatures à ce poste seront soumises à une présélection des candidats effectuée par l’IA visant à vérifier que les exigences minimales indiquées au titre du poste sont respectées. Vous trouverez plus de détails sur l’utilisation de l’IA par Vaco by Highspring ici (https://www.highspring.com/ai-use-notices/). L’évaluation des candidatures après cette phase initiale sera menée par des recruteurs et gestionnaires d’embauche. Vaco by Highspring n’a pas d’information ni d’opinion sur l’utilisation de solutions d’IA par son client dans le cadre de l’embauche.
Avis sur la transparence salariale
La rémunération fixée pour ce poste, et d’autres, auprès de Vaco by Highspring dépend de nombreux facteurs,notamment :
- les compétences, l’expérience et la formation de la personne;
- les exigences en matière de permis d’exercice et de certification;
- l’emplacement du bureau et autres considérations géographiques;
- d’autres besoins commerciaux et organisationnels.
Cela étant, conformément à la loi locale, Vaco by Highspring estime qu’en fonction des critères susmentionnés, la fourchette salariale suivante constitue une estimation raisonnable de la rémunération de base pour une personne embauchée à ce poste dans les régions géographiques exigeant la divulgation de la fourchette salariale. Le ou la titulaire du poste peut également être admissible à des primes discrétionnaires.