Data Engineer · Dallas, TX

Narendranath Edara

Six years in data. Currently on a multi-tenant payroll system.

  • CDC ETL from 30 minutes to under 8, compute down 67%, freshness SLA unchanged.
  • Database provisioning made idempotent, returning about 5,000 engineering hours a year.
  • repo-context-hooks on PyPI: signed releases, 330+ tests, no runtime dependencies.

SQL Server · CDC · Azure DevOps · Microsoft Fabric · SSIS · Python · Kafka · Spark · AKS · FastAPI

Open to senior data engineering roles on SQL Server, Azure and Kafka.

Email me

Production Data Systems

Changes I shipped and kept running at work, tagged by where they ran.

CDC ETL runtimeEXPONENTHR

30 min→under 8 min

Idempotent incremental merge-upserts replaced full-table reloads. Compute down 67%, freshness SLA unchanged.

Database provisioningEXPONENTHR

1 hr, 20+/day→one idempotent run

Safely rerunnable; four latent failure modes closed. Roughly 5,000 engineering hours a year returned.

Release cycleEXPONENTHR

3 months→14 days

Azure DevOps CI/CD owned outright, with SSRS and Power BI release steps automated.

Report query responseEXPONENTHR

12 s→under 4 s

Reporting moved onto Fabric semantic models, with index and stored-procedure tuning on the OLAP paths.

CDC capture after failoverEXPONENTHR

specialist restore→under 1 hr

Availability-group failover written up as runbooks, including the post-failover capture failure that gives no signal.

Alert signal-to-noiseMISSOURI S&T

1:5→1:1.2

Per-service thresholds filtered about 250 non-actionable P3 alerts a week.

Anomaly-detection computeMISSOURI S&T

20 static nodes→4–8 autoscaled

AKS with horizontal pod autoscaling. $3,200 a month saved, CPU utilization 12% to 64%.

Experience

Jul 2024 - Present

Data Engineer

ExponentHR · Dallas, TX

Multi-tenant payroll and HR platform

Payroll accrual and SECURE 2.0

PTO balances convert to cash at termination, so a defect in accrual logic is a payout rather than a display bug. I traced it to a payroll-period boundary calculation in SQL and put the fix through change-request review. SECURE 2.0 compliance went out in the same release train.

payout-class defect · traced in SQL · shipped under change control

Failure detection and failover recovery

CDC capture jobs fail three ways, and retention windows close while they go unnoticed — the change history inside them does not come back.

Capture failure modes run never starts run hangs exits clean — captured nothing monitoring covers all three

Availability-group failover is documented as recovery procedures, including the post-failover capture failure that produces no error at all. One live incremental-load failure was diagnosed and restored in place rather than falling back to a full reload.

3/3 modes monitored · failover recovery < 1 hr · no full reload

CDC incrementals and idempotent provisioning

The CDC ETL reloaded whole tables on every run, and standing up a database was an hour of manual orchestration repeated twenty-plus times a day. Both failed in ways that needed a person to clean up before the next attempt. Moving CDC onto idempotent incremental merge-upserts held the freshness SLA through the change; provisioning was rewritten to be safely rerunnable, closing four latent failure modes on the way.

DETAIL
Latent failure modes closed orphaned CDC cleanup jobs terminated properly guard kills active replication sessions before disabling CDC restore parameters can no longer silently default stuck-restore detection with pre-flight race checks

30 min → <8 min · compute −67% · freshness SLA held

1 hr × 20+/day → idempotent rerun · ~5,000 eng-hr/yr returned

Azure Platform Engineering topology: Azure Repos triggers Azure Pipelines CI/CD, output flows to SQL Server CDC, then to AAG replica and Microsoft Fabric Enterprise data platform at ExponentHR. Azure Repos triggers Azure Pipelines on every push - build, test and deploy stages run on every push, compressing the release cycle from three months to fourteen days. SQL Server uses CDC incremental merge-upserts, cutting ETL compute by sixty-seven percent. An idempotent pipeline automates Contained Always-On Availability Group copy-downs, saving about an hour per request. Microsoft Fabric semantic models consume the CDC stream for reporting. Azure Repos PR / push trigger protected main Azure Pipelines CI/CD owned build → test → deploy 3mo → 14-day cycle idempotent stages SQL Server CDC incremental ETL 30min → <8min merge-upsert AAG Replica ~1hr saved / copy-down Microsoft Fabric semantic models
FIG. 1 — EXPONENTHR PLATFORM TOPOLOGY

Release ownership and the reporting layer

Releases went out roughly quarterly. Taking CI/CD through Azure DevOps compressed the cycle, with SSRS and Power BI release steps automated. Reporting moved onto Fabric semantic models with index and stored-procedure tuning underneath.

3 mo → 14 days · query response 12 s → <4 s

SQL Server · CDC · SSIS · Azure DevOps · Microsoft Fabric · AAG · Power BI / SSRS

Aug 2023 - Jul 2024

Data Engineer

Missouri University of Science and Technology · Rolla, MO

ML infrastructure and anomaly detection

Anomaly detection and autoscaling

The detection pipelines select an algorithm per time-series profile, and thresholds are set per service, so a noisy service can be tuned without raising the floor on the rest. One memory leak surfaced four hours before it would have taken the service down. Moving the workloads off static D-series VMs onto AKS raised utilization while cutting compute spend.

signal-to-noise 1:5 → 1:1.2 · ~250 P3 alerts/wk filtered

20 static nodes → 4–8 autoscaled · −$3,200/mo · CPU 12% → 64%

NLP over visitor feedback

Pipelines in spaCy, NLTK, RoBERTa and VADER feeding dimensional models and Power BI. Published in the Journal of Nonprofit & Public Sector Marketing, 2025, as DOI 10.1080/10495142.2025.2525123.

10,000+ reviews · peer-reviewed, 2025

Python · spaCy · NLTK · RoBERTa · AKS · Power BI

Jun 2023 - Aug 2023

Product Intern

C2FO · Kansas City, KS

Transaction and behavioural SQL on a working-capital platform

The SQL work went across transaction and user behavior on a B2B working-capital platform.

SQL

Sep 2020 - Mar 2021

Business Intelligence Analyst

udaan.com · Nashik, India

Demand forecasting and fulfillment analytics

Demand and inventory models in Power BI for a new-city Staples launch on a B2B marketplace, plus a continuity plan that kept deliveries moving through application downtime.

allocation ROI +7% · fulfillment 99.3% under volume growth

Power BI

Mar 2018 - Sep 2020

Business Analyst

Zomato · Hyderabad, India

Competitor pricing data, search ranking signals, unit economics

Built the competitor analytics platform that replaced week-old pricing data with a current refresh, and indexed 100K+ internal documents in Elasticsearch so the support desk stopped answering the same questions daily. On search I supplied and evaluated ranking signals from the analyst side and did not train the models.

contribution −₹18 → +₹2 · 300 restaurants

Elasticsearch

Projects

AutoApply AI

SOLO BUILD

Job-application automation across three clients — Chrome extension, FastAPI backend, React dashboard — sharing one state pipeline.

40+ endpoints · 11 ATS adapters · 6 LLM providers behind one router · 355 backend tests

FastAPI · Chrome MV3 · React · PostgreSQL · Redis · pgvector · Clerk

DETAIL

ARCHITECTUREChrome MV3 extension, FastAPI backend and React dashboard on one state pipeline.

DECISIONShadow DOM isolation on the client, so page styles cannot reach the injected UI.

VERIFICATION355 backend tests. The repository is public and the deployment is not, so the counts are my own — the code is the part you can read.

Source Notes

repo-context-hooks

v1.0 · SIGNED

Pre-commit and post-commit hooks that snapshot repo context for an LLM agent between sessions.

330+ tests @ v1.0 · Hypothesis property tests · 0 runtime deps

Python · GitHub Actions · Sigstore · CodeQL · PyPI

DETAIL

DECISIONStandard library only at runtime — nothing external to break inside a hook context.

RELEASESigstore-signed publishing through an OIDC Trusted Publisher, CodeQL on every pull request, Dependabot on the Actions dependencies.

VERIFICATION330+ tests at v1.0, including Hypothesis property tests over sampling boundaries and hook dedup.

repo_context_hooks/bundle/hooks.json · excerptSOURCE ↗

"SessionStart": [
  {
    "matcher": "",
    "hooks": [
      {
        "type": "command",
        "command": "python \"$CLAUDE_PROJECT_DIR\"/.claude/scripts/repo_specs_memory.py session-start",
        "timeout": 20
      },
      {
        "type": "command",
        "command": "python \"$CLAUDE_PROJECT_DIR\"/.claude/scripts/session_context.py session-start",
        "timeout": 20
      }
    ]
  }
],

PreCompact, PostCompact and SessionEnd are wired the same way.

PyPI Source Write-up

Fraud detection platform

100K SYNTHETIC TX

Kafka-fed transaction scoring: LightGBM behind FastAPI, predictions and latencies persisted to PostgreSQL.

P50 0.45ms · P95 0.89ms · P99 1.12ms @ 100 TPS

Kafka · LightGBM · FastAPI · MLflow · Airflow · Prometheus · Grafana

DETAIL

FLOWproducer → partitioned topic → consumer → lightgbm · fastapi → postgresql

FINDINGMost of the 1.12ms P99 is FastAPI overhead rather than the model: LightGBM itself takes microseconds, so the per-call DataFrame is what would go first if latency ever mattered.

CONSTRAINTSynthetic data: 100,000 generated transactions with 2,034 labeled fraud, running on one laptop.

Source Notes

Portfolio risk analytics

STRUCTURED STREAMING

Streaming value-at-risk for an equity portfolio, with per-position attribution of what is driving it.

VaR 95/99 · historical simulation · 30d window · component VaR per position

Kafka · Spark Structured Streaming · Delta · FastAPI · Streamlit

DETAIL
Portfolio risk streaming topology Market ticks enter through Kafka. Spark Structured Streaming aggregates them on five-second tumbling windows behind a ten-second watermark and writes the bronze layer to Delta. FastAPI computes value at risk at 95 and 99 percent by historical simulation over a thirty-day window with component VaR per position, and a Streamlit dashboard consumes the API. Kafka market ticks in Spark Structured Streaming 5s tumbling windows 10s watermark Delta bronze layer FastAPI VaR 95/99 — historical simulation 30d window · component VaR Streamlit dashboard
FIG. 2 — STREAM TOPOLOGY

src/streaming/spark_kafka_consumer.py · excerptSOURCE ↗

# 5-second windowed aggregations
stats_df = enriched_df \
    .withWatermark('processing_time', '10 seconds') \
    .groupBy(
        window('processing_time', '5 seconds'),
        'symbol'
    ) \
    .agg(
        count('*').alias('ticks'),
        round(avg('price'), 2).alias('avg_price'),
        round(min('price'), 2).alias('min_price'),
        round(max('price'), 2).alias('max_price'),
        sum('volume').alias('volume')
    )

DECISIONVaR by historical simulation over a 30-day window rather than a parametric model: equity returns are fat-tailed and a normal assumption understates the tail.

RESULTComponent VaR is computed per position, so the dashboard can name which holding is driving portfolio risk.

CONSTRAINTThe API serves the Delta output; the seeded sample set is only what a fresh checkout falls back to before the stream has run.

Source Notes

Also public: FinTune, a Mistral-7B QLoRA service with PII redaction and drift monitoring, JobScout, which tracks 153 company career pages across six ATS APIs, and tailor-resume, a document pipeline on PyPI with 190 tests across four distribution surfaces.

Education, Research and Certifications

MS in Information Science and Technology, Missouri S&T, GPA 4.0, Jan 2022 to Dec 2023.

An Examination of Sentiment Analysis as a Tool for Gathering Visitor Feedback. Bojanic, Edara, Zhang. Journal of Nonprofit & Public Sector Marketing, Taylor & Francis, 2025.

Five Claude agents audited my plugin release in parallel. Sigstore signing, OIDC Trusted Publisher, CodeQL, and the 41 findings the review produced.

DP-700 Microsoft Certified Fabric Data Engineer Associate · Certified Scrum Product Owner · Tableau Analyst · AI-900 Azure AI Fundamentals

Contact

Naren Edara

Open to senior data engineering roles. Email is the fastest way to reach me.

edara.narendranath@gmail.com

LinkedIn GitHub Resume