Data Engineer at ExponentHR / Dallas, TX

Narendranath Edara

Payroll data ready sooner.
Less work done by hand.

I build SQL Server and Azure pipelines for payroll teams. My work makes data processing faster, removes repetitive database setup, and helps teams ship changes sooner.

Open to U.S. relocation · H-1B transfer required

SQL · Python · Azure · Data quality

Explore three production outcomes ↓

01 / Production impact

Three changes that made work easier.

Production work at ExponentHR. Each result starts with an operational problem and a measurable change.

01 / Faster payroll data

30 minutes to under 8

Processing the same data repeatedly made payroll ingestion slow. I changed the pipeline to process only what had changed.

CDC runtime fell from 30 minutes to under 8, with compute cost down approximately 67%.

How it works

Incremental merge-upserts replaced full-table reloads in SQL Server. Rerunnable ingestion and validation help keep the data correct.

02 / Less manual setup

5,000 engineering hours a year

Database provisioning needed repeated manual coordination. I automated the copy, restore, security, and CDC setup into a process teams can rerun.

Automated provisioning saves 5,000 engineering hours annually.

How it works

Idempotent steps make it possible to recover and rerun setup without rebuilding everything by hand.

03 / Changes delivered sooner

Three months to 14 days

Long release cycles kept changes waiting. I owned Azure DevOps release automation and coordinated delivery across development, testing, IT, and business teams.

Release cycles shortened from approximately three months to 14 days.

How it works

CI/CD and automated reporting release steps made delivery more repeatable and reduced manual handoffs.

More production results: reporting, recovery, and cloud costs

Report query responseEXPONENTHR

12 s→under 4 s

Reporting moved onto Fabric semantic models, with index and stored-procedure tuning on the OLAP paths.

CDC capture after failoverEXPONENTHR

specialist restore→under 1 hr

Availability-group failover written up as runbooks, including the post-failover capture failure that gives no signal.

Alert signal-to-noiseMISSOURI S&T

1:5→1:1.2

Per-service thresholds filtered about 250 non-actionable P3 alerts a week.

Anomaly-detection computeMISSOURI S&T

20 static nodes→4-8 autoscaled

AKS with horizontal pod autoscaling. $3,200 a month saved, CPU utilization 12% to 64%.

02 / Experience

The path to this work.

Jul 2024 - Present

Data Engineer

ExponentHR · Dallas, TX

Legacy reporting platform migration · multi-tenant payroll and HR

Move a legacy reporting platform onto a modern data foundation

The broader ExponentHR program is a reporting and analytics migration: replacing a legacy platform with a Microsoft SQL Server and SSIS-based architecture. My work sits inside that transition, rebuilding and supporting the data movement, transformation, validation, and operational workflows the new reporting stack depends on.

legacy platform → SQL Server + SSIS · reporting modernization · migration support

Payroll pipelines, data validation, database provisioning, and release automation.

Production engineering details

Catch data defects before they become recurring work

Built a control-table-driven validation framework so data checks are defined as configuration. Reconstructed SSIS execution history in SQL to make pipeline behavior easier to investigate. The focus is correctness at the model boundary: grain, missing rows, unexpected values, and differences between full and incremental loads.

metadata-driven checks · reusable validation · diagnosable failures

Get payroll rules right as requirements change

PTO balances convert to cash at termination, so a defect in accrual logic is a payout rather than a display bug. I traced it to a payroll-period boundary calculation in SQL and put the fix through change-request review. SECURE 2.0 compliance went out in the same release train.

payout-class defect · traced in SQL · shipped under change control

Find the failures that do not raise an error

Incremental ETL jobs can miss their schedule, become stuck, or fail without recording a completion time. I built monitoring around these distinct states so failures can be detected before the CDC retention window closes.

Capture failure modes run never starts run hangs run ends without a completion record monitoring covers all three

Availability-group failover is documented as recovery procedures, including the post-failover capture failure that produces no error at all. One live incremental-load failure was diagnosed and restored in place rather than falling back to a full reload.

3/3 modes monitored · failover recovery < 1 hr · no full reload

Stop reloading everything. Stop rebuilding by hand.

The CDC ETL reloaded whole tables on every run, and standing up a database was an hour of manual orchestration repeated twenty-plus times a day. Both failed in ways that needed a person to clean up before the next attempt. Moving CDC onto idempotent incremental merge-upserts held the freshness SLA through the change; provisioning was rewritten to be safely rerunnable, closing four latent failure modes on the way.

DETAIL
Latent failure modes closed orphaned CDC cleanup jobs terminated properly guard kills active replication sessions before disabling CDC restore parameters can no longer silently default stuck-restore detection with pre-flight race checks

30 min → <8 min · compute −67% · freshness SLA held

1 hr × 20+/day → idempotent rerun · ~1 hr of manual work removed per request

Azure Platform Engineering topology: Azure Repos triggers Azure Pipelines CI/CD, output flows to SQL Server CDC, then to AAG replica and Microsoft Fabric Enterprise data platform at ExponentHR. Azure Repos triggers Azure Pipelines on every push - build, test and deploy stages run on every push, compressing the release cycle from three months to fourteen days. SQL Server uses CDC incremental merge-upserts, cutting ETL compute by sixty-seven percent. An idempotent pipeline automates Contained Always-On Availability Group copy-downs, saving about an hour per request. Microsoft Fabric semantic models consume the CDC stream for reporting. Azure Repos PR / push trigger protected main Azure Pipelines CI/CD owned build → test → deploy 3mo → 14-day cycle idempotent stages SQL Server CDC incremental ETL 30min → <8min merge-upsert AAG Replica ~1hr saved / copy-down Microsoft Fabric semantic models
FIG. 1 - EXPONENTHR PLATFORM TOPOLOGY

Modernize reporting while making releases easier to ship

As reporting capabilities move off the legacy platform, the replacement stack uses SQL Server and SSIS for the data layer and Microsoft Fabric semantic models for analytics. I also took CI/CD through Azure DevOps, automating SSRS and Power BI release steps while tuning indexes and stored procedures underneath the reporting paths.

legacy reporting migration · 3 mo → 14-day releases · query response 12 s → <4 s

SQL Server · CDC · SSIS · Azure DevOps · Microsoft Fabric · AAG · Power BI / SSRS

Aug 2023 - Jul 2024

Data Engineer

Missouri University of Science and Technology · Rolla, MO

ML infrastructure and anomaly detection

Azure research workloads, anomaly detection, and lower cloud costs.

Research engineering details

Match compute to demand, not a fixed fleet

The detection pipelines select an algorithm per time-series profile, and thresholds are set per service, so a noisy service can be tuned without raising the floor on the rest. One memory leak surfaced four hours before it would have taken the service down. Moving the workloads off static D-series VMs onto AKS raised utilization while cutting compute spend.

signal-to-noise 1:5 → 1:1.2 · ~250 P3 alerts/wk filtered

20 static nodes → 4-8 autoscaled · −$3,200/mo · CPU 12% → 64%

Turn visitor feedback into something teams can use

Pipelines in spaCy, NLTK, RoBERTa and VADER feeding dimensional models and Power BI. Published in the Journal of Nonprofit & Public Sector Marketing, 2025, as DOI 10.1080/10495142.2025.2525123.

10,000+ reviews · peer-reviewed, 2025

Python · spaCy · NLTK · RoBERTa · AKS · Power BI

Jun 2023 - Aug 2023

Product Intern

C2FO · Kansas City, KS

Transaction and behavioural SQL on a working-capital platform

SQL analysis of financial transactions and product usage.

Product analytics details

The SQL work went across transaction and user behavior on a B2B working-capital platform.

SQL

Sep 2020 - Mar 2021

Business Intelligence Analyst

udaan.com · Nashik, India

Demand forecasting and fulfillment analytics

Demand forecasting and fulfillment analytics for a new-city launch.

Business intelligence details

Demand and inventory models in Power BI for a new-city Staples launch on a B2B marketplace, plus a continuity plan that kept deliveries moving through application downtime.

allocation ROI +7% · fulfillment 99.3% under volume growth

Power BI

Mar 2018 - Sep 2020

Business Analyst

Zomato · Hyderabad, India

Competitor pricing data, search ranking signals, unit economics

Competitor analytics and internal search, from the business analyst side.

Analytics details

Built the competitor analytics platform that replaced week-old pricing data with a current refresh, and indexed 100K+ internal documents in Elasticsearch so the support desk stopped answering the same questions daily. On search I supplied and evaluated ranking signals from the analyst side and did not train the models.

contribution −₹18 → +₹2 · 300 restaurants

Elasticsearch

03 / Independent projects

Independent projects you can inspect.

Built outside employment. These projects demonstrate ingestion, document workflows, and developer tooling; they are separate from my production SQL Server and Azure work.

JobScout

DATA INGESTION

A direct-source ingestion system that turns inconsistent company career feeds into normalized, searchable job records.

Per-source retries · isolated failures · normalized dates and salary fields

Python · Flask · SQLite WAL · React · GitHub Actions

Engineering decisions

PROBLEMThird-party schemas change independently. One failed feed must not stop the rest of the ingestion run.

DESIGNATS-specific adapters, bounded retries, a shared progress broker, and concurrency guards around maintenance operations.

TRADEOFFSQLite keeps operations small, but the process-level coordinator assumes one worker. Multiple workers would require shared coordination.

BOUNDARYConfigured coverage is not the same as successful live coverage. The project README also records ranking controls whose stored weights are not yet applied.

AutoApply AI

SOLO BUILD

Document intelligence for job-application workflows: capture unstructured pages, retrieve supporting career evidence, and generate drafts for user review across a Chrome extension, API, and dashboard.

Chrome extension + API + dashboard · retrieval from work history · provider fallback

FastAPI · Chrome MV3 · React · PostgreSQL · Redis · pgvector · Clerk

DETAIL

ARCHITECTUREChrome MV3 extension, FastAPI backend and React dashboard on one state pipeline.

DECISIONShadow DOM isolation on the client, so page styles cannot reach the injected UI.

VERIFICATIONPublic source with backend and browser tests. The README tracks security issues that gate public launch; this is a personal project, not an enterprise production deployment.

repo-context-hooks

v1.0 · SIGNED

Pre-commit and post-commit hooks that snapshot repo context for an LLM agent between sessions.

330+ tests @ v1.0 · Hypothesis property tests · 0 runtime deps

Python · GitHub Actions · Sigstore · CodeQL · PyPI

DETAIL

DECISIONStandard library only at runtime - nothing external to break inside a hook context.

RELEASESigstore-signed publishing through an OIDC Trusted Publisher, CodeQL on every pull request, Dependabot on the Actions dependencies.

VERIFICATION330+ tests at v1.0, including Hypothesis property tests over sampling boundaries and hook dedup.

repo_context_hooks/bundle/hooks.json · excerptSOURCE ↗

"SessionStart": [
  {
    "matcher": "",
    "hooks": [
      {
        "type": "command",
        "command": "python \"$CLAUDE_PROJECT_DIR\"/.claude/scripts/repo_specs_memory.py session-start",
        "timeout": 20
      },
      {
        "type": "command",
        "command": "python \"$CLAUDE_PROJECT_DIR\"/.claude/scripts/session_context.py session-start",
        "timeout": 20
      }
    ]
  }
],

PreCompact, PostCompact and SessionEnd are wired the same way.

Fraud detection platform

100K SYNTHETIC TX

Kafka-fed transaction scoring: LightGBM behind FastAPI, predictions and latencies persisted to PostgreSQL.

Synthetic transaction dataset · model serving · observable pipeline

Kafka · LightGBM · FastAPI · MLflow · Airflow · Prometheus · Grafana

DETAIL

FLOWproducer → partitioned topic → consumer → lightgbm · fastapi → postgresql

FINDINGLatency claims need a reproducible workload, hardware description, and end-to-end measurement boundary. This project is presented as an engineering implementation; local benchmark figures are not production guarantees.

CONSTRAINTSynthetic data: 100,000 generated transactions with 2,034 labeled fraud, running on one laptop.

Portfolio risk analytics

STRUCTURED STREAMING

Streaming value-at-risk for an equity portfolio, with per-position attribution of what is driving it.

VaR 95/99 · historical simulation · 30d window · component VaR per position

Kafka · Spark Structured Streaming · Delta · FastAPI · Streamlit

DETAIL
Portfolio risk streaming topology Market ticks enter through Kafka. Spark Structured Streaming aggregates them on five-second tumbling windows behind a ten-second watermark and writes the bronze layer to Delta. FastAPI computes value at risk at 95 and 99 percent by historical simulation over a thirty-day window with component VaR per position, and a Streamlit dashboard consumes the API. Kafka market ticks in Spark Structured Streaming 5s tumbling windows 10s watermark Delta bronze layer FastAPI VaR 95/99 - historical simulation 30d window · component VaR Streamlit dashboard
FIG. 2 - STREAM TOPOLOGY

src/streaming/spark_kafka_consumer.py · excerptSOURCE ↗

# 5-second windowed aggregations
stats_df = enriched_df \
    .withWatermark('processing_time', '10 seconds') \
    .groupBy(
        window('processing_time', '5 seconds'),
        'symbol'
    ) \
    .agg(
        count('*').alias('ticks'),
        round(avg('price'), 2).alias('avg_price'),
        round(min('price'), 2).alias('min_price'),
        round(max('price'), 2).alias('max_price'),
        sum('volume').alias('volume')
    )

DECISIONVaR by historical simulation over a 30-day window rather than a parametric model: equity returns are fat-tailed and a normal assumption understates the tail.

RESULTComponent VaR is computed per position, so the dashboard can name which holding is driving portfolio risk.

CONSTRAINTThe API serves the Delta output; the seeded sample set is only what a fresh checkout falls back to before the stream has run.

Also public: FinTune, a Mistral-7B QLoRA service with PII redaction and drift monitoring, JobScout, with adapters for six ATS APIs, and tailor-resume, a document pipeline distributed through PyPI, a CLI, Streamlit, and MCP.

Education & credentials

MS in Information Science and Technology, Missouri S&T, GPA 4.0, Jan 2022 to Dec 2023.

An Examination of Sentiment Analysis as a Tool for Gathering Visitor Feedback. Bojanic, Edara, Zhang. Journal of Nonprofit & Public Sector Marketing, Taylor & Francis, 2025.

Five Claude agents audited my plugin release in parallel. Sigstore signing, OIDC Trusted Publisher, CodeQL, and the 41 findings the review produced.

DP-700 Microsoft Certified Fabric Data Engineer Associate · Certified Scrum Product Owner · AI-900 Azure AI Fundamentals

04 / Let's talk

Let's talk about your data engineering role.

Naren Edara

If your team is spending too much time on manual data work, slow releases, or pipeline recovery, I would like to help change that. I am looking for a data engineering role where I can own the problem and build the tools to solve it.

narendranath.ed@gmail.com