Principal Data Engineer POST NUMBER: 470480
Core Qualifications:
- 15 years of software engineering experience, with a minimum of 10 years specializing in backend distributed systems or data infrastructure at scale.
- Expert-level Python and SQL proficiency (10 years) rooted in a career of building production-grade software.
- Hands-on mastery of Snowflake (internals and clustering), dbt (macro design and Jinja), Airflow (scheduler internals), and Power BI (Import vs. Live connection).
- Comprehensive knowledge of AWS services, including IAM, VPC, Glue, S3, SFTP, Lambda, CloudWatch, and SNS.
- Proven track record of designing large-scale data platforms with a deep understanding of CAP theorem, eventual consistency, and the trade-offs between batch and streaming architectures.
- Experience implementing Gitlab CI/CD and DataDog for robust system monitoring and alerting.
- Ability to design RAG architectures, manage vector databases, and integrate LLMs into complex data pipelines.
- Skilled in writing persuasive RFCs and ADRs that drive consensus among architects and engineering leadership.
- Proven ability to influence technical strategy and facilitate cross-functional alignment across organizational levels without direct managerial authority.
- Bachelor’s or Master’s degree in Computer Science, Software Engineering, or a related field.
Technical Strategy & Platform Architecture
- Define the long-term technical architecture for the Enterprise Data Platform, translating business strategy into scalable Data Mesh and domain-oriented specifications.
- Implement automated CI/CD pipelines and Infrastructure as Code (IaC) to foster a unified engineering culture across disciplines.
- Design robust APIs that enable seamless data consumption across operations, finance, and product teams.
- Build resilient ELT/ETL pipelines utilizing S3, Lambda, Glue, dbt, and Airflow (MWAA).
- Establish data quality, observability, lineage, and SLAs as core, first-class features of the data platform.
- Standardize enterprise-wide schema design, modeling patterns, and deployment workflows.
- Lead the integration of AI-assisted development (Cursor, MCP, Copilot) to accelerate developer velocity and reduce cognitive load.
- Scale LLM-driven code generation for AWS data pipelines, including automated test creation, documentation, and semantic schema generation.
- Leverage Amazon Bedrock, Snowflake Cortex, and AI-enabled IDEs to optimize the data lifecycle and reduce delivery lead times.
- Enforce high standards for code quality through rigorous PR reviews, unit testing, and automated schema validation within CI/CD pipelines.
- Architect SRE-focused resiliency frameworks and self-healing systems to ensure 99.9% availability for critical data pipelines.
- Partner with Data Science, Product, and Platform teams to align on performance, compliance, and feature engineering.
- Mentor Senior and Lead engineers through design reviews, RFCs, and pair programming to elevate the organization's technical bar.
- Design and productionize end-to-end ML infrastructure, including feature stores, model experimentation frameworks, and deployment monitoring.
- Build optimized ETL/ELT workflows for training data and model deployment leveraging Snowflake ML (Snowpark) and Amazon SageMaker.
- Lead the design and POC of Graph RAG (Retrieval-Augmented Generation) architectures, enabling LLM agents to query structured Snowflake data via Knowledge Graphs.
- Apply knowledge of open-source Knowledge Graph solutions to create advanced patterns for semantic data retrieval.
- Coordinate project testing, deployment, and post-implementation activities, ensuring successful project delivery and that lessons learned are applied to future projects.
Vaco by Highspring values a diverse workplace and strongly encourages women, people of color, LGBTQ+ individuals, people with disabilities, members of ethnic minorities, foreign-born residents, and veterans to apply.
EEO Notice
Vaco by Highspring is an Equal Opportunity Employer and does not discriminate against any employee or applicant for employment because of race (including but not limited to traits historically associated with race such as hair texture and hair style), color, sex (includes pregnancy or related conditions), religion or creed, national origin, citizenship, age, disability, status as a veteran, union membership, ethnicity, gender, gender identity, gender expression, sexual orientation, marital status, political affiliation, or any other protected characteristics as required by federal, state or local law.
Vaco by Highspring and its parents, affiliates, and subsidiaries are committed to the full inclusion of all qualified individuals. As part of this commitment, Vaco by Highspring and its parents, affiliates, and subsidiaries will ensure that persons with disabilities are provided reasonable accommodations. If reasonable accommodation is needed to participate in the job application or interview process, to perform essential job functions, and/or to receive other benefits and privileges of employment, please contact HR@vaco.com .
Vaco by Highspring also wants all applicants to know their rights that workplace discrimination is illegal.
By submitting to this position, you agree that you will be giving Vaco by Highspring the exclusive right to present your as a candidate for the foregoing employment opportunity. You further agree that you have represented information about yourself accurately and have not affirmatively misrepresented your qualifications. You also agree to maintain as confidential, to the fullest extent permitted by law, any information you learn from Vaco by Highspring about the position and you will limit disclosure of information about the position only to the extent necessary to perform any obligations in furtherance of your application. In exchange, Vaco by Highspring agrees to exercise reasonable efforts to represent you through all solicitation, job screening and resume dispersal.
Privacy Notice
Vaco by Highspring and its parents, affiliates, and subsidiaries (“we,” “our,” or “Vaco by Highspring”) respects your privacy and are committed to providing transparent notice of our policies.
- California residents may access Vaco by Highspring HR Notice at Collection for California Applicants and Employees here.
- Virginia residents may access our state specific policies here.
- Residents of all other states may access our policies here.
- Canadian residents may access our policies in English here and in French here.
- Residents of countries governed by GDPR may access our policies here.
Pay Transparency Notice
Determining compensation for this role (and others) at Vaco by Highspring depends upon a wide array of factors including but not limited to:
- the individual’s skill sets, experience and training;
- licensure and certification requirements;
- office location and other geographic considerations;
- other business and organizational needs.
With that said, as required by local law, Vaco by Highspring believes that the following salary range referenced above reasonably estimates the base compensation for an individual hired into this position in geographies that require salary range disclosure. The individual may also be eligible for discretionary bonuses.