Avoiding Hallucinated APIs
Stop the model inventing plausible-but-fake functions, packages, and options by constraining the toolbox and verifying every call.
TL;DR
- Models invent plausible functions, options, and even package names that do not exist.
- Constrain the toolbox, pin versions, allow only named libraries, and show real signatures.
- Catch hallucinations with the type-checker, the docs, and by refusing to run unknown calls.
Know The Failure
Fake MethodsCalls to methods that sound right but do not exist on the object.
arr.removeDuplicates() // not a
real Array methodFake OptionsParameters or config keys the real API never accepts.
fetch(url, { retry: 3 }) // fetch
has no retry optionFake PackagesEntirely invented package names, a real supply-chain risk.
import x from 'super-utils-pro'
// does this even exist?Shrink The Space
Pin VersionsA version anchors the model to one real API surface.
"React 18, Next 15. Use only their
real APIs."Allow-List LibrariesRestrict to the dependencies you actually have.
"Use only: zod, date-fns. No other
packages."Show Real SignaturesPaste the actual exports so the model calls what exists.
"Available: parse(s: string):
Result. Use this."Catch With Tools
Type-CheckThe compiler flags unknown members and wrong options instantly.
tsc --noEmit # red on fake callsLint & ResolveLinters and module resolution surface unknown imports.
Unresolved import -> likely
hallucinated package.Check The DocsConfirm any unfamiliar call against the official reference.
Does the doc list this method?
No -> do not use it.Safe Habits
Verify Before InstallNever add a package you have not confirmed exists and is correct.
Check the registry + repo before
`npm i`.Ask For SourcesHave the model name where an API is documented.
"Which package/version documents
this? Link it."Distrust ConfidenceIdiomatic-looking code is not proof the call is real.
Looks right != is real. Verify.Tips
- Paste the real module's exports or signature so the model calls what actually exists.
- Treat any unfamiliar method or package as guilty until verified against the official docs.
Warnings
- Hallucinated package names are a supply-chain risk: never install a package you have not verified exists.
- Fake calls often look perfectly idiomatic, so they pass a casual read; let the compiler check, not your eyes.
In Practice
A prompt that pins versions, allow-lists packages, supplies the real signature, and demands verifiable calls, then you let the type-checker confirm nothing invented slipped through.
- Pinning versions and allow-listing packages removes most room to invent.
- Pasting the real signature tells the model exactly what it may call.
- Requiring doc-backed calls makes the model flag anything uncertain.
- The type-check is the final gate that catches any fake member.
Implement `retryFetch(url, opts)` with retry + backoff.
Constraints to prevent invented APIs:
- TypeScript 5.x, Node 20. Use the built-in fetch only.
- Allowed packages: NONE (standard library only).
- fetch has no built-in retry; implement it yourself.
- Use this real signature; do not invent others:
type Opts = { retries?: number; signal?: AbortSignal }
- If you are unsure an API exists, say so instead of
guessing, and tell me what to verify.
Return the function only.
# Then I verify:
# tsc --noEmit (flags any fake member/option)
# grep imports (no surprise packages)FAQ
They generate the most plausible next token, not a verified fact. If a function with a given name would be reasonable, the model may produce it whether or not it exists. The output is confident and idiomatic, which is exactly what makes hallucinated calls easy to miss.
Non-existent methods on a real object, options or parameters a function does not accept, APIs from a different version than yours, and, most dangerously, entire package names that do not exist. The last is a supply-chain risk, since attackers register such names.
Shrink the space to invent in: pin the library and version, restrict to named dependencies or the standard library, and paste the real signatures or exports the code should use. The more concrete the toolbox, the less room to hallucinate.
Let tools check, not your eyes. Run the type-checker and linter, which flag unknown members immediately. Verify any unfamiliar call against the official docs, and never install a package without confirming it exists and is the one you intend.