ravidata.com
Get in touch

Open to data engineer roles across the UK

Production-grade data pipelines. Tested like software.

I'm Ravi Teja, a data engineer in London with 5+ years across data engineering and data testing. I build batch and streaming pipelines on Azure, Databricks and Snowflake using PySpark, Delta Lake and Kafka, with the quality checks built in, so bad data fails loudly before it reaches a report.

  • Azure
  • Databricks
  • Snowflake
  • PySpark
  • Delta Lake
  • Kafka
  • Airflow
  • dbt
See the pipelines Live demo Download CV
uk_weather_pipeline queued
  1. extract_rawqueued
  2. transform_and_loadqueued
  3. check_qualityqueued
  4. build_summaryqueued

Simulated in your browser. The real DAG, with these four tasks and five checks, is on GitHub.

5+ yrs
Data engineering and data testing4+ on production Azure pipelines
10.8M
Taxi trips through bronze, silver and goldnyc-taxi-spark, PySpark
76%
Smaller Kafka messages with zstd batching828 to 198 bytes, measured
8
Pipelines built front to backone of them running live
Toolkit

The manifest

What I reach for, grouped by where it sits in the flow. Pick a tool with a build count to see the pipelines that use it.

day jobused in production work 3 buildspipelines on this page that use it

Architecture

How the data moves

Two paths I build: a batch medallion and an event stream. Pick a stage to see the pattern, the reasoning and the code that does it.

Case studies

Eight builds, front to back

Streaming and batch. Most run locally end to end, with the source, the processing and the checks all in the repo. The newest one is running live.

Data quality

The checks that ship with the pipeline

A testing background is the part of my CV that changes how the code looks. These are the guardrails I write in by default.

schema_contract
Column names and types are asserted on read. A source that changes shape fails at the boundary instead of silently corrupting everything downstream of it.
null_ratio
Business-critical columns carry a null threshold. Crossing it stops the run rather than publishing a table that is technically fine and practically useless.
row_count_delta
Every load is compared against the previous one. A ninety percent drop is a broken extract, not a quiet Tuesday.
freshness_sla
The newest partition is checked against the window it is expected to land in, before anything reads from it.
referential_integrity
Fact rows arriving without a matching dimension key get quarantined for inspection. Never dropped, never joined away.
dead_letter_topic
Streaming events that fail validation go to a dead-letter topic unchanged, with the reason, source partition and offset in the headers, ready to inspect or replay.
at_least_once
A consumer only commits its offset after the batch is processed and every bad message is safely stored. A crash means re-reading, never losing.
unit_tests + ci
Transformation logic is covered by pytest and run in GitHub Actions on every push, so a broken transform is caught before it reaches a branch anyone trusts.
Career

I came into data engineering through testing

Five years in, that is still the thing that shapes how I build.

Ravi Teja

I have spent enough time finding broken data downstream to want the schema contracts, the row counts and the null thresholds sitting inside the pipeline, where a bad load fails loudly instead of quietly.

Computer science and mathematics underneath; Azure, PySpark and Delta Lake on top. Most of my work sits in the Microsoft ecosystem, but the patterns travel between clouds: land it raw, model it deliberately, and test it before anyone builds a report on it.

Download CV GitHub
Based in
London, UK
Available
Roles anywhere in the UK
Open to
Permanent, contract or temporary
Background
Computer science and mathematics

  1. Data Engineer

    London, UK
    • Design, build and maintain production ETL and ELT pipelines in Azure Data Factory.
    • Write Python ingestion workflows and Azure Databricks pipelines in PySpark, with validation built into the run.
    • Develop and tune complex SQL and T-SQL across Azure SQL Database.
    • Put data quality and reconciliation controls in place, then monitor and troubleshoot the pipelines in production.
    • Work with analysts and stakeholders so the datasets they report on can be trusted.

  2. Data Tester

    London, UK
    • Source-to-target data validation in SQL.
    • Reconciliation across record counts and field mappings.
    • Designed functional and regression test scenarios.
    • Raised and tracked defects in Azure DevOps, working in Agile sprints.

  3. BSc Computer Science and Mathematics

Contact

Let's build data people can trust.

Hiring, contract work, or a pipeline that needs a second pair of eyes. Available UK-wide, permanent, contract or temporary.

Phone +44 7405 484827 GitHub github.com/RaviTejaData LinkedIn in/ravi-teja-2991233b2
Goes straight to my inbox.