Stop hallucinating sources: how we keep Marvin from making URLs up

An investor we know (let's call him D.) forwarded us a research brief another AI tool had written for him. Tight thesis on a small-cap industrial, three supporting bullets, and a citation to a Bloomberg article from the prior week: exactly the kind of corroborating signal you want before you bring an idea to your IC.

The article didn't exist.

He clicked through to grab the headline for his memo. 404. Tried the Wayback Machine. Nothing. He searched Bloomberg's site for the supposed author. No such person. The model had emitted a Bloomberg-shaped URL because Bloomberg-shaped URLs were statistically likely given the surrounding context, not because it had read anything. By the time he caught it, the brief had been on a partner's desk for two days.

He told us, politely, that he was done with AI research tools.

We hear some version of this every week. It's the most common objection prosumers have to letting an AI into their workflow: I tried one of these, it made stuff up, why would yours be any different?

Fair question. Here's the answer.

The guarantee

Every URL Marvin cites in a scheduled newsletter is a URL that a real web search, run for you, this morning, actually returned.

Not "probably." Not "almost always." Every one. If the model tries to slip in a citation that didn't come from a search this run, the newsletter doesn't send until the model fixes it. That is enforced at the door, not requested in a prompt.

You can verify this yourself the way D. should have been able to: click any link in any Marvin newsletter. It will resolve. If one ever doesn't, that's a bug we want to know about: reply to the email and tell us.

How it works, briefly

The mechanism is simpler than you'd think.

When Marvin builds a newsletter, the first thing he does is run real searches against real search backends and pull back actual results. Every URL those searches return goes onto a list of URLs Marvin is allowed to cite in this newsletter. Nothing else qualifies.

When the model finishes the draft and tries to send it, we scan every link in the body. Any link not on the allowed list gets rejected, and the model has to swap in a real citation or drop the bullet. It cannot quietly ship a draft with a made-up Bloomberg URL pasted in.

So the newsletter that lands in your inbox only contains links that were actually found.

You can put that to the test on any claim. Get a brief that cites something you want to confirm, and reply to the same thread:

To: marvin@ccmarvin.com

Hi Marvin,

The second bullet says the plant restart got pushed to Q3. Where did that come from? Send me the link you're citing.

You'll get the actual URL back, the one a search returned this morning, not a plausible-looking reconstruction:

From: Marvin marvin@ccmarvin.com

Hi James,

That came from the company's May 6 operations update. Here's the page I pulled it from:

https://www.sec.gov/cgi-bin/browse-edgar?action=getcompany

The Q3 date is in the second paragraph under "Facility Status." If it doesn't say what I claimed, that's on me and I want to know.

Sincerely,
Marvin

We didn't flip the switch on faith

Hard enforcement is easy to ship and easy to regret. Reject too aggressively and your Monday brief stops landing.

So before we made the rejection real, we ran enforcement in logging-only mode for a few weeks. Every fabricated citation that would have been blocked got recorded, but we let the newsletter through and watched the counts. We wanted to know the rejection rate was low enough that turning enforcement on wouldn't break deliveries.

Fewer than 2% of runs needed any correction at all. We turned hard enforcement on, and it's been on ever since.

What this doesn't cover yet

A few honest limits, because we'd rather you hear them from us.

The guarantee above applies to scheduled newsletters: your recurring topic briefs, portfolio digests, and research updates. If you ask Marvin for a one-shot deep research report ("dig into the lithium supply chain, get me something by tonight"), the same enforcement isn't running there yet. Deep research has a different shape: longer-running, more search passes, a post-generation review step. We're building the equivalent guarantee into that pipeline now. Until then, on deep research, click through to verify any source that's going to drive a decision.

Second: a real URL is not the same as an accurate summary of what's at it. We verify the link is real and was found by an actual search. We don't yet verify that Marvin's two-sentence summary perfectly matches the article. We read the article at search time when we can, which catches most drift, but the gap exists.

Finally: your citation history is yours. A URL cited in your newsletter doesn't end up on anyone else's allowed list. Nothing about what Marvin reads for you leaks sideways.

Try it

If you already get Marvin newsletters: pick the last one, click three links at random, and watch them resolve. If one of them ever 404s, hit reply and tell us. It shouldn't be possible, and if it happens we want to find out why before you do.

If you don't get Marvin newsletters yet and want to, tell him what you'd want a recurring brief on:

To: marvin@ccmarvin.com

Hi Marvin,

Send me a morning brief on small-cap industrials: anything material on capacity, restarts, or guidance changes. Include the source link for every item.

He'll ask a few questions, then set one up. The next morning you'll get the first one, with citations that actually go somewhere.