At Hendrickson, my internship is building an LLM-powered agent that sits on top of the tools program managers already use — automating task tracking, surfacing blockers before they become escalations, and cutting down the manual coordination overhead that eats a PM's week. A few months in, here is what I have actually learned, as opposed to what I expected going in.
Expectation: the hard part is the LLM. Reality: the hard part is the data model.
I expected most of the engineering effort to go into prompting and agent orchestration with LangChain. In practice, the highest-leverage work was upstream of that — building a clean, consistent representation of "what is a task, what is a blocker, what is a status" out of project data that had been entered inconsistently by different people for months. An agent reasoning over messy, ambiguous source data will produce confident, articulate, wrong summaries. Cleaning and normalizing the input mattered more than any prompt engineering.
Blockers are not always where they claim to be
Early versions of the agent flagged "blockers" based on explicit status fields — anything marked blocked, was a blocker. That missed most of the real ones. The tasks that were actually stuck were usually still marked "in progress," just with no update in three weeks. The useful signal turned out to be staleness combined with dependency position, not a self-reported status field that people forget to update. Once the agent started reasoning over silence instead of only explicit signals, the blocker surfacing became genuinely useful instead of just restating what people already knew.
Guardrails that made it trustworthy enough to actually use
A PM will stop trusting a tool the first time it confidently states something false. A few guardrails made the difference between a demo and something people would rely on:
- Cite the source, always. Every summary the agent generates links back to the specific task or update it is drawing from. No unsourced claims, ever.
- Separate "fact" from "inference." The agent is explicit about what is directly stated in the data versus what it is inferring from patterns (like the staleness signal above). Conflating the two is where trust breaks.
- FastAPI backend, PostgreSQL for state. Keeping the agent's reasoning stateless and re-derivable from the database on each run, rather than accumulating hidden state, made debugging incorrect outputs tractable. If a summary looked wrong, I could always trace it back to the exact rows it was generated from.
What surprised me most
How much of "AI program management" is really just disciplined data engineering with a conversational layer on top. The interface is a chat-style summary. The actual product is a normalized, trustworthy model of project state that a PM would want even without the LLM attached. That reframing changed how I prioritized the rest of the internship — data quality first, agent behavior second.