LangGraph agent for catalog data
The data team loaded third-party catalog data by hand, and it was slow. I built an agent workflow that takes a short request, fetches the matching parts data, cleans it, and writes it into the catalog. Jobs run on a schedule, resume after a failure, and end with a report anyone can check.
- My role
- I built it alone: the research, the architecture, the agent, the scheduling, and the demo UI.
- Status
- A working prototype with a demo UI for the data team. It was not in production when I left in July 2026.
- Stack
- Python, LangGraph, OpenAI API, APScheduler, PostgreSQL
- Scope
- Brand, model, year, and region requests for the 7 vehicle brands the company handles
The problem
Loading third-party vehicle and parts data into the company catalog was repetitive manual work. Someone had to pick many parameters, download the data, clean it, and map each record into the right catalog tables.
The goal was to automate this, so the data team could stop doing it by hand.
How it works
- Request Brand, model, year, and region, chosen from drop-down lists.
- Resolve variants Find every matching vehicle variant and expand it into the catalog scope to fetch.
- Fetch Call the third-party catalog API, with caching and retries.
- Normalize and map Clean the records and map them to the right catalog tables.
- Write Save to PostgreSQL in batches, with checkpoints so a job can resume or be cancelled safely.
- Report Show how each batch was normalized and mapped, so the data team can check it.
- SchedulerAPScheduler with a small UI for scheduled and manual runs
- CacheCatalog data cached down to the group level
- ControlsScope limits, job status, and visible API usage
What I built
- Started simple, then made it predictable. The first version took a natural-language request, had a clarification step, and traced every agent step for debugging. Once the workflow was stable, I replaced free text with drop-down inputs, so runs are repeatable.
- Why LangGraph. The LLM does only a small part of the work. LangGraph still pays off: the workflow is structured, easy to inspect, and able to resume after a failure.
- Built for very large jobs. Saves are chunked and resumable, so a vehicle model with 100K+ variations does not start over after a failure.
- Scheduling. APScheduler runs jobs on a schedule or on demand, from a small UI.
- Output people can audit. Automatic reports show how each batch was normalized and mapped, so non-engineers can review the result.
- Documentation. I wrote reference docs for the catalog schema that say which tables the agent reads and which it writes.
Engineering judgment
Before building, I compared building with buying. I looked at existing AI data-collection tools and at workflow engines such as Airflow, Prefect, and Celery, and chose a lightweight approach that fit the project's stage and could grow later.
I also kept job scope and billable API use visible, instead of hiding them behind autonomous behavior. The design cuts manual work while keeping people in control of cost and data.
Status
The prototype works, with scheduling and a demo UI for the data team, and it was being merged into a single internal admin tool. It was not in production when I left in July 2026.
This page leaves out employer code, internal data, and private URLs.