Data Engineer POST NUMBER: 478000
This is a full-time, direct-hire Data Engineer role focused on Microsoft Fabric and Azure data engineering, based about 20 minutes north of Downtown Pittsburgh with a hybrid schedule and local residency required.
Position overview
We are seeking a Data Engineer to design, build, and maintain modern enterprise data platforms using Microsoft Fabric and Azure services in support of advanced analytics, real-time reporting, and AI/ML use cases. This is a direct-hire opportunity located approximately 20 minutes north of Downtown Pittsburgh, offering a hybrid work schedule (on-site several days per week). Candidates must currently reside in the greater Pittsburgh area or be willing to relocate prior to starting; this role is not open to C2C, third parties, or visa sponsorship.
Key responsibilities
-
Design, develop, and maintain scalable, production-grade data pipelines and integrations using Microsoft Fabric, Azure Data Factory, Fabric Data Factory, Azure Databricks, Azure Event Hubs, OneLake, Fabric Lakehouse, and Fabric Data Warehouse.
-
Build analytics-ready datasets to support pricing, supply chain, POS sales, customer behavior analytics, executive dashboards, and AI/ML workloads.
-
Implement dual-engine data pipelines leveraging Azure Data Factory for structured batch workloads and Azure Event Hubs / Kafka for real-time event ingestion.
-
Support multiple ingestion patterns including batch ETL/ELT, CDC/database mirroring, streaming ingestion, API-based integrations, and SaaS connectors.
-
Develop near real-time analytics solutions using Eventstream and Real-Time Intelligence capabilities in Microsoft Fabric.
-
Design and optimize PySpark workloads in Azure Databricks and Fabric Spark to process high-volume historical datasets, XML/JSON log files, streaming transactional events, and operational telemetry data.
-
Build scalable transformation logic that supports both streaming and batch architectures.
-
Model and transform enterprise data using ANSI SQL, T-SQL, dbt, and Lakehouse design principles.
-
Design star and snowflake schemas, semantic models, and curated analytical datasets to enable governed self-service analytics across the organization.
-
Maintain and optimize Azure Data Lake Storage Gen2 environments, including Delta Lake formats, ACID-compliant patterns, schema evolution, partitioning, and performance tuning.
-
Support enterprise Lakehouse architecture leveraging Microsoft Fabric OneLake.
-
Partner closely with Analytics and Business stakeholders to deliver Power BI dashboards, executive scorecards, KPI reporting, and self-service analytics solutions, including semantic models, Direct Lake datasets, row-level security, and data governance standards.
-
Enable Copilot-driven analytics and AI-assisted reporting capabilities on top of governed datasets.
-
Deploy and manage cloud infrastructure using Terraform, Azure Resource Manager (ARM), and Infrastructure-as-Code practices.
-
Automate CI/CD workflows for data pipelines and analytics assets using Azure DevOps, Git, and Docker.
-
Orchestrate and schedule enterprise workflows with Azure Data Factory, Fabric Pipelines, Managed Apache Airflow, and Control-M (where applicable).
-
Implement robust data observability, including automated monitoring and alerting for batch failures, streaming interruptions, data quality issues, schema drift, and pipeline latency.
-
Build checksum and reconciliation frameworks between source systems and analytics platforms to support enterprise data governance and operational resiliency initiatives.
Required qualifications
-
Local to the Pittsburgh region and able to work in a hybrid on-site schedule approximately 20 minutes north of Downtown Pittsburgh; no relocation assistance, C2C, third parties, or visa sponsorship is available for this role.
-
3–5 years of hands-on experience in data engineering, cloud analytics, or enterprise data platforms with a strong focus on Azure services.
-
Proven experience with:
-
Microsoft Azure and Microsoft Fabric
-
Azure Data Lake Storage Gen2 (ADLS Gen2)
-
Azure Databricks and Fabric Spark (including Spark Structured Streaming)
-
Azure Data Factory and Fabric Data Factory
-
Azure Event Hubs and Kafka for real-time ingestion
-
Azure Synapse Analytics and/or Fabric Data Warehouse
-
-
Strong proficiency in:
-
Python and PySpark
-
ANSI SQL and T-SQL
-
Batch and streaming data processing (Spark Structured Streaming, Azure Stream Analytics, event-driven architectures)
-
-
Hands-on experience with:
-
dbt (Data Build Tool)
-
Delta Lake and Lakehouse architectures
-
Data warehousing concepts and dimensional modeling (star and snowflake schemas)
-
Semantic layer design and enterprise data modeling
-
-
Demonstrated ability to build and support production data pipelines, troubleshoot performance issues, and optimize large-scale data processing workloads.
Preferred qualifications
-
Bachelor’s or Master’s degree in Data Science, Computer Science, Information Systems, Engineering, Statistics, Mathematics, or a related technical field.
-
Experience delivering and supporting Power BI analytics, including semantic models, Direct Lake datasets, and row-level security.
-
Hands-on work with Fabric Real-Time Intelligence, OneLake, REST APIs, XML/JSON processing, and event-driven architectures.
-
Exposure to AI/ML workloads and tools such as Azure OpenAI, Copilot integrations, and predictive analytics solutions.
-
Experience supporting large-scale enterprise analytics environments with complex operational datasets and strict SLAs.
Work authorization and engagement terms
-
Full-time, salaried, direct-hire position with the client organization (not contract or contract-to-hire).
-
Candidates must be currently authorized to work in the United States on a permanent basis; we are unable to provide visa sponsorship or work with C2C arrangements or third-party agencies for this role
Vaco by Highspring values a diverse workplace and strongly encourages women, people of color, LGBTQ+ individuals, people with disabilities, members of ethnic minorities, foreign-born residents, and veterans to apply.
EEO Notice
Vaco by Highspring is an Equal Opportunity Employer and does not discriminate against any employee or applicant for employment because of race (including but not limited to traits historically associated with race such as hair texture and hair style), color, sex (includes pregnancy or related conditions), religion or creed, national origin, citizenship, age, disability, status as a veteran, union membership, ethnicity, gender, gender identity, gender expression, sexual orientation, marital status, political affiliation, or any other protected characteristics as required by federal, state or local law.
Vaco by Highspring and its parents, affiliates, and subsidiaries are committed to the full inclusion of all qualified individuals. As part of this commitment, Vaco by Highspring and its parents, affiliates, and subsidiaries will ensure that persons with disabilities are provided reasonable accommodations. If reasonable accommodation is needed to participate in the job application or interview process, to perform essential job functions, and/or to receive other benefits and privileges of employment, please contact HR@vaco.com .
Vaco by Highspring also wants all applicants to know their rights that workplace discrimination is illegal.
By submitting to this position, you agree that you will be giving Vaco by Highspring the exclusive right to present your as a candidate for the foregoing employment opportunity. You further agree that you have represented information about yourself accurately and have not affirmatively misrepresented your qualifications. You also agree to maintain as confidential, to the fullest extent permitted by law, any information you learn from Vaco by Highspring about the position and you will limit disclosure of information about the position only to the extent necessary to perform any obligations in furtherance of your application. In exchange, Vaco by Highspring agrees to exercise reasonable efforts to represent you through all solicitation, job screening and resume dispersal.
Privacy Notice
Vaco by Highspring and its parents, affiliates, and subsidiaries (“we,” “our,” or “Vaco by Highspring”) respects your privacy and are committed to providing transparent notice of our policies.
- California residents may access Vaco by Highspring HR Notice at Collection for California Applicants and Employees here.
- Virginia residents may access our state specific policies here.
- Residents of all other states may access our policies here.
- Canadian residents may access our policies in English here and in French here.
- Residents of countries governed by GDPR may access our policies here.
Pay Transparency Notice
Determining compensation for this role (and others) at Vaco by Highspring depends upon a wide array of factors including but not limited to:
- the individual’s skill sets, experience and training;
- licensure and certification requirements;
- office location and other geographic considerations;
- other business and organizational needs.
With that said, as required by local law, Vaco by Highspring believes that the following salary range referenced above reasonably estimates the base compensation for an individual hired into this position in geographies that require salary range disclosure. The individual may also be eligible for discretionary bonuses.