MotionDog Cloud Services
Blog, White Papers, and Articles

Agentforce in Production: Six Months of Lessons Learned

The Agentforce demo is impressive. Production is messier. After six months of running agents in real environments — a service org, a sales motion, and an internal IT use case — here’s what I’ve learned that I wish I’d known on day one.

Agentforce: What Goes Wrong vs. What Goes RightWhat goes wrong• Vague topic boundaries• Stale grounding data• No fallback to human• Missing eval loop• Permission leakage• No observability• Treating it as one launchWhat goes right• Narrow, specific topics• Curated knowledge base• Clear escalation rules• Eval set on every change• Permission-aware actions• Logged conversations• Iterative releases

Lesson 1: Narrow beats general

Every agent we tried to launch as “a general assistant for the team” underperformed. Every agent scoped to one specific job — “help customer service triage password reset tickets,” “help sellers find similar closed-won deals” — worked on the first iteration.

Define the agent’s job in a sentence. If you can’t, the topics are too broad. Split it into multiple agents, each with a tight scope.

Lesson 2: Grounding data is the product

Agents are only as good as the data they’re grounded in. The temptation is to point them at the entire knowledge base and hope. The reality: agents quote outdated articles, conflict-resolve poorly, and produce wrong-but-confident answers.

Curate the corpus. Tag content with freshness dates. Build a content review cadence — monthly is the minimum — so an agent never quotes a doc that hasn’t been opened in 18 months.

Lesson 3: Build the escalation path before you build the agent

The agent is going to fail. That’s fine. What matters is whether the failure is graceful. “I’m not sure — here’s a link to start a chat with a person” is a good answer. “I think the policy says X, here’s the article” when there’s no such article is a customer trust problem.

Define escalation triggers explicitly: confidence threshold, sensitive keywords, repeated user frustration signals. Test those handoffs as carefully as the happy paths.

Lesson 4: Treat changes like code, not like content

Every prompt change, every grounding update, every new action is a deploy. Have an eval set — a couple dozen representative test conversations with expected good outcomes — and run it before every change. The Agent Testing Framework that’s coming in Summer ’26 makes this much easier; until then, do it yourself with a spreadsheet if you have to.

Without evals, you’re guessing. With evals, you have a regression net.

Lesson 5: The agent inherits the user’s permissions

This trips up nearly every team on first deploy. The agent acts on behalf of the logged-in user, so it sees what they see and can do what they can do. If the user has access to a record, the agent can quote it. If they don’t, the agent can’t even though it might be holding the answer in working memory.

This is good for security. It’s confusing for users until you explain it. Build it into your training and your error messaging.

Lesson 6: Conversation logs are gold

Every conversation tells you something: where the agent confused itself, what users actually ask for, which topics surface most. Sample 20 conversations a week. Review them with the team. Half your roadmap will come from this.

Lesson 7: It’s a product, not a launch

The orgs that succeed assigned a real owner with a real backlog. The ones that struggled treated Agentforce as an IT project that gets delivered and then maintained on the side. There’s no maintaining-on-the-side here. Either you’re iterating, or it’s drifting.

If you can’t fund a part-time owner, you probably shouldn’t deploy the agent yet.


Comments

Leave a comment

This site uses Akismet to reduce spam. Learn how your comment data is processed.