AI Search Research Methodology.
An AI-generated answer is not a ranking list. It is a time-bound synthesis produced under a particular prompt, interface state, retrieval context and generation run. Research must capture the complete observation chain—not only the final text.
RESPONSECLAIMS
07 OBSERVEDCITATIONS
04 RESOLVEDSOURCES
03 UNIQUEVARIANCE
02 MATERIAL
Study the response as an event. Study retrieval as a system.
AI search research observes how a controlled query family produces claims, references and source selections across repeated captures. It does not infer an invisible internal process from one visible answer.
AI search research is the controlled observation of generated answers, retrieved evidence and run-to-run variation under explicitly recorded query and environment conditions.
Capture the returned state exactly as observed.
Resolve identity, page state and the passage relevant to each claim.
Change one dimension at a time and preserve every capture.
Do not convert output patterns into unsupported internal explanations.
Eight objects. None can substitute for another.
A domain mention is not a citation. A citation is not proof of claim support. A supported claim in one response is not stable visibility.
Query family
A versioned group of prompts expressing one information need through controlled formulations.
Response
The complete visible answer returned for one prompt, environment state and execution.
Atomic claim
One independently testable proposition extracted without changing its original meaning.
Mention
A named entity, domain, brand, author or concept appearing in the visible response.
Citation link
A visible response element that points to an externally resolvable source object.
Source document
The identified page or document state reached from a citation or retrieval reference.
Grounding relation
The tested relationship between a response claim and evidence present in its cited source.
Variation event
A material difference in claims, citations, sources or conclusions across comparable runs.
Twelve stages from question to bounded finding.
The pipeline preserves what was asked, what was returned, which evidence was visible and which conclusions survived repeated observation.
Define the information need
Name population, concept, outcome and observation window.
Construct prompt variants
Separate equivalent wording from changed intent or constraints.
Freeze environment state
Record language, region, date, surface and observable settings.
Execute repeated runs
Preserve complete outputs and timestamps without selective retention.
Extract atomic claims
Split compound statements into independently testable units.
Map mentions and citations
Distinguish names, links, domains, documents and duplicate sources.
Preserve source state
Record the source version inspected for the grounding decision.
Test claim support
Code direct, partial, contradictory, absent or inaccessible support.
Measure run variation
Compare inclusion, wording, citations, source identity and decision.
Map source concentration
Locate recurring domains, document types and topical subtrees.
Test uncertainty
Separate observable findings from hidden mechanism assumptions.
Publish bounded results
State scope, run count, measures, exceptions and raw evidence route.
One answer contains multiple evidence states.
Response-level scoring hides whether individual claims are supported, partially supported, unsupported or uncited.
Break synthesis into testable propositions.
The answer is stored intact, then copied into an atomic claim ledger. Every claim receives a stable identity and a separate relation to each visible citation.
No citation is credited merely because it appears near a paragraph.
Control one dimension. Observe what changes.
Every panel holds the information need constant while testing a different source of response instability.
Equivalent intent can still shift source selection.
Broad explanatory formulation without requested format.
Adds an evidence-quality constraint while preserving the core need.
Requests a methodological frame and changes response organization.
One prompt can produce multiple valid observations.
Entity coverage and internal architecture appear in the synthesis.
One architecture claim disappears; two source identities persist.
A new measurement claim appears without direct support.
A citation may resolve while its evidence has changed.
The response claims that the document defines four coverage layers.
The accessible page was revised after the response capture.
Current absence cannot establish what the response-time document contained.
Citation presence does not establish evidential support.
Atomic proposition is causal and unconditional.
The cited passage does not support automatic causality.
Code according to the exact relationship defined in the protocol.
Prompts are experimental conditions. Version every difference.
The lattice separates stable semantics from deliberate changes in audience, task, format and constraint.
| VARIANT | INFORMATION NEED | AUDIENCE | TASK | CONSTRAINT | FORMAT | REPEATS | COMPARISON ROLE |
|---|---|---|---|---|---|---|---|
| P-01 / BASE | Build topical authority | Unspecified | Explain | None | Open | 3 | BASELINE |
| P-02 / EVIDENCE | Build topical authority | Researcher | Explain | Use defensible evidence | Open | 3 | CONSTRAINT |
| P-03 / PROCESS | Build topical authority | Site operator | Design steps | Name dependencies | Ordered system | 3 | TASK |
| P-04 / CRITIQUE | Build topical authority | Analyst | Evaluate | Include failure modes | Argument | 3 | FRAME |
| P-05 / DRIFT | Build topic visibility | Unspecified | Explain | None | Open | 3 | SEMANTIC DRIFT |
Trace every claim from response to source passage.
The trace prevents a nearby citation, repeated domain or reputable source from receiving automatic credit for a claim it does not support.
Six diagnostics. No universal AI visibility score.
Each measure answers a specific question about appearance, stability, sourcing or evidential support.
Response inclusion rate
responses containing target object ÷ eligible responsesMeasures whether a defined entity, source or concept appears in the captured run set.
Claim persistence
runs containing equivalent claim ÷ comparable runsTests whether a proposition survives repeated execution under the same condition.
Citation persistence
runs citing resolved source ÷ comparable runsSeparates stable source selection from one-off citation appearance.
Direct support rate
directly supported cited claims ÷ tested cited claimsMeasures claim–source grounding after citation identity and source state are resolved.
Source concentration
citations from leading source set ÷ resolved citationsShows whether observed sourcing is distributed or repeatedly dependent on a narrow set.
Material variance rate
decision-relevant deltas ÷ compared response elementsCounts only changes capable of altering the reported finding or claim boundary.
Five prompts. Three repeats. One bounded interpretation.
The example demonstrates how topical concepts, citations and support states are compared without presenting synthetic values as live findings.
AI answers and ranked results require different units.
The methods can share query controls and source validation, but their observable structures are not interchangeable.
Eight shortcuts that create false AI search findings.
These errors collapse distinct objects, select convenient runs or claim access to mechanisms that were never observed.
Single-run certainty
One response is presented as stable visibility across time and prompt conditions.
Prompt cherry-picking
Only the wording that produces the desired inclusion is retained.
Mention–citation collapse
A named entity is counted as cited even when no source link attributes it.
Citation–support collapse
A visible citation receives credit without testing the cited passage.
Answer-level grounding
Mixed support states are hidden inside one score for the whole response.
Live-source substitution
A current source page silently replaces the state available at capture time.
Mechanism invention
Output patterns are described as proof of hidden retrieval or ranking logic.
Score compression
Visibility, persistence, citation and support are collapsed into one opaque number.
Every finding must retain its observation coordinates.
The study record makes each response, claim, citation, source and comparison independently traceable.
Twelve methods. One complete research chain.
MTH/12 closes the Methods branch by applying question design, scope, sampling, validation, reproducibility and uncertainty controls to AI-mediated search.