About HPDL
A small practice, deliberately
HPDL is a New Zealand data engineering practice. We work on data logistics: the movement, shaping, and verification of data between the systems a business already owns. It is narrow on purpose. Depth in one area is more useful to a client than shallow coverage of ten.
What we care about
Re-runnable over clever
The most valuable property of a pipeline is that it can be run again safely. We optimise for that before anything else, because it is what makes recovery possible at two in the morning.
Visible over assumed
If a number cannot be traced to a source, it is a rumour. Lineage, freshness, and quality checks are part of the build, not a later phase that never arrives.
Handover over dependency
Every engagement ends with your team able to operate what was delivered. Documentation and walkthroughs are deliverables, not extras.
Platforms
We work with Databricks, Snowflake, Microsoft Azure Fabric, dbt, and orchestration built on Airflow or native scheduler tooling. The platform matters less than the discipline applied to it: contracts, idempotency, reconciliation, and monitoring look the same wherever they run.
Who we work with
Mid-market and larger New Zealand organisations that have already invested in a modern data platform and need it to perform under real operational pressure. Typically that means an existing team, a platform in production, and a specific set of problems that generalist delivery has not resolved.
Based here
Operating from New Zealand, in the same time zone as the teams we work with. See services for what we do, or contact us to discuss a specific problem.
How we work in practice
Small teams with direct access to senior engineers, rather than a layered structure where the person who understands your platform is not the person you speak to. Documents are written to be read by whoever inherits the system, not to look thorough at the point of delivery.
We prefer reversible decisions. Where a choice cannot be reversed cheaply, we spend longer on it and record why it was made, including the alternatives that were rejected and the conditions that would change the answer.
Monitoring is treated as part of the build. A pipeline without freshness checks, row-count expectations, and an alert that reaches a human is not finished, because the first indication of failure would otherwise be a person noticing an odd number in a report.
Where we fit
Some organisations need a platform built from nothing. Many more already have one, along with a specific set of problems that make it unreliable. That second situation is where this practice is useful: existing investments, real operational pressure, and a need for focused engineering rather than another strategy document.