Authority is not a universal property.
Source quality is the claim-specific capacity of a source to provide dependable support. It depends on competence, access, transparency, incentives, consistency and accountability—evaluated against the exact proposition, scope and time being tested.
Quality is relational. The claim sets the test.
A source does not carry one permanent quality label. Its usefulness changes with the claim: subject, granularity, jurisdiction, time window, required directness and acceptable uncertainty.
Source quality is a structured judgment of fitness between a resolved source and a bounded claim—not a reputation score attached to a domain.
Provenance establishes which source and record are under review. Quality then asks whether that source was positioned to know, observed the relevant state, exposes enough process to be inspected, manages its incentives, behaves consistently and remains accountable for correction.
Change the claim. Watch the source change.
Select a claim context, then inspect or adjust each dimension. The output remains a profile rather than a universal score because strengths and limitations cannot safely be averaged away.
Strong for declaration. Bounded for reality.
The vendor is competent and directly positioned to state its own specification. The same source is not independent evidence that the product achieves the claimed effect in external conditions.
Inspect the dimensions. Do not hide them in an average.
Each dimension answers a different question. A source can be highly competent yet commercially constrained, transparent yet poorly positioned to observe, or accountable while still temporally stale.
Competence
Relevant knowledge, technical capability and domain-specific experience for the exact claim.
Access
Direct or otherwise adequate access to the event, system, records, population or process being described.
Transparency
Visible authorship, definitions, methods, inputs, limitations, dates and correction history.
Incentive alignment
Commercial, political, institutional or reputational pressures that may shape selection and presentation.
Consistency
Internal coherence and stability across versions, comparable claims and repeated observations.
Accountability
Named responsibility, review mechanisms, correction pathways and consequences for persistent error.
Expertise without access is interpretation. Access without competence is observation.
The strongest source for a particular claim often combines relevant competence with direct access. Sources outside that quadrant can remain valuable, but their role must be named accurately.
FIT
Still inspect incentives, transparency and version fit.
Do not relabel inference as direct observation.
Restrict conclusions beyond the observer’s competence.
Trace upstream before assigning evidential weight.
Bias is not a verdict. It is a pressure to model.
Every source operates under incentives. The task is not to imagine a pressure-free source, but to expose material pressures and test whether transparency, independent checks and accountability constrain them.
CLAIM
Pressure becomes risk when it can shape the claim invisibly.
A commercial source is not automatically weak, and a non-commercial source is not automatically neutral. Quality improves when the incentive is declared, the evidence basis is inspectable and unsupported claims create correction costs.
The same source can move from preferred to insufficient.
Quality is evaluated at the claim-source edge. These examples preserve the distinction between what a source can authoritatively declare and what requires independent observation or corroboration.
Direct declaration of current identity or policy.
Useful when version and product scope are explicit.
Self-report requires external performance evidence.
Cannot independently establish market superiority.
Identity should still resolve to first-party records.
Can test selected specification under stated conditions.
Strong when design reflects actual use conditions.
Requires representative products and common metrics.
May summarize but not own organizational identity.
Check whether specifications were independently tested.
Depends on data access and disclosed methodology.
Potentially strong with complete and comparable sample.
Does not establish official identity or policy.
Experience reports do not define formal specification.
Useful for recurring experience patterns after controls.
Selection, duplication and platform bias must be modeled.
Strong sources remain bounded sources.
Each example identifies the source’s strongest permissible use and the point where another evidence class must enter.
Vendor API specification
High competence and direct access for the declared interface, parameters and response fields. Commercial interest does not erase that access, but it limits claims about independent performance.
Structured API response
Strong access to the API provider’s returned dataset under a declared request context. Quality evaluation separates response integrity from whether the provider’s measurement model fits the research question.
Named expert analysis
Competence may be high for mechanism and interpretation while event access remains indirect. Credentials must be relevant to the exact subject rather than borrowed from adjacent prestige.
Large but undocumented corpus
Volume cannot repair missing authorship, sampling, definitions or collection conditions. It may generate hypotheses, but cannot carry a precise claim until its construction becomes inspectable.
Weakness appears as a specific failed control.
Red flags do not automatically reject a source. They identify the exact dimension that must be limited, repaired or independently checked.
Prestige in one field is used to support a claim outside demonstrated competence.
The source asserts firsthand knowledge without showing how it reached the relevant event or records.
Conclusions are presented without definitions, sample, inputs, calculation or limitations.
A material commercial or institutional connection is absent from the disclosure boundary.
Standards change when the same test would produce an inconvenient result.
Errors cannot be reported, reviewed or connected to a responsible author or publisher.
Exact figures disguise uncertain definitions, weak coverage or an undocumented measurement process.
Domain familiarity replaces inspection of the exact document, author, version and claim relation.
Evaluate, qualify, then route the evidence forward.
The protocol produces a bounded use decision. It does not convert judgment into false mathematical certainty.
Resolve the source
Confirm record identity and provenance before quality evaluation begins.
Bind the claim
Declare proposition, subject, scope, time and required directness.
Profile six dimensions
Preserve strengths, weaknesses and unknown states independently.
Set permitted use
Name what the source supports and what it cannot establish alone.
Route the gap
Send unresolved limits to collection, corroboration or conflict analysis.
Do not ask whether a source is good. Ask what it can establish.
A defensible evaluation converts a vague credibility judgment into six explicit decisions. Each decision limits the next: unresolved identity weakens every later judgment; an unbounded claim makes competence impossible to test; poor access prevents direct support; opaque methods restrict verification; incentives change the controls required; and missing correction mechanisms lower the confidence that errors will be repaired.
What exact record is being judged?
Evaluate the named author, publisher, document, dataset, interface, edition and timestamp—not a familiar brand in the abstract. A reputable organization can publish a weak page; an unfamiliar specialist can publish an exceptionally transparent primary record. Record-level identity prevents reputation from substituting for inspection.
What proposition needs support?
Write the claim so its subject, attribute, population, location, period and precision are visible. “Demand increased” is not yet testable. “Monthly branded searches in Croatia increased between January and June under the same collection method” creates a source-quality test that competence and access can answer.
How could this source know?
Separate expertise from observation. A platform operator may directly know its documented rules but not users’ private motives. A customer can report personal experience but not population prevalence. A researcher may estimate prevalence from a sample, provided sampling and measurement are disclosed.
Can the support be audited?
Look for definitions, inputs, selection rules, transformations, exclusions, dates and uncertainty. Transparency does not guarantee correctness, but it makes error discoverable. When a result cannot be reconstructed, quality must be limited to what is directly observable in the published record.
What can shape the account?
Map commercial, political, institutional, reputational and audience pressures to the parts of the claim they could alter. Then look for controls: preregistered rules, disclosed funding, independent replication, stable definitions, raw records and meaningful consequences for misstatement.
What may this source carry?
End with a bounded statement: preferred for direct declaration, strong for controlled observation, useful for interpretation, contextual for experience, discovery-only, or insufficient. State the limitation and the evidence needed to cross it. This is more informative than a single credibility score.
One source class. Six independent tests.
This matrix demonstrates why source labels are insufficient. The cells are not universal grades; they describe the questions that must be answered for a bounded claim. On smaller screens, the ledger scrolls horizontally so every comparison remains readable instead of collapsing into compressed text.
QUALITY TEST →
Authors should understand declared interfaces, parameters and supported behavior.
Strong access to intended design and current product state.
Require dates, changelog, examples, definitions and retirement notices.
Marketing incentives matter for superiority and performance claims.
Check whether wording and behavior remain stable across versions.
Issue tracking and revision history improve challengeability.
Expertise is strong when authorship and methods match the claim.
Access is limited to observed participants, records and conditions.
Definitions, analysis and limitations should be inspectable.
Model sponsor influence and incentives toward novel findings.
A single result may not remain stable across settings.
Corrections and retractions matter, but may occur slowly.
Specialist teams maintain classifications and collection procedures.
Often broad access to mandated records or representative samples.
Revisions, seasonal adjustment and suppressed values change meaning.
Governance can affect timing, framing and category definitions.
Strong when methods remain comparable and breaks are documented.
Named agencies and release calendars support correction tracking.
Competence rests on instrument use and category judgment.
Strong for the time, place and variables actually observed.
Preserve setup, time, device, exclusions and raw output.
Selection and interpretation incentives still require controls.
Stability must be tested across observers or repeated windows.
Accountability rises when raw evidence and protocol are retained.
Relevant experience and identity cannot be confirmed.
The account may be firsthand, but access is unverified.
Methods, context and omitted details are usually unavailable.
Affiliations, strategic motives and coordination may be invisible.
Deletion, editing and identity shifts reduce continuity.
No reliable correction or responsibility mechanism may exist.
Reviewers know their own experience, not the full market.
Each reviewer observes one bounded interaction.
Verification, moderation and collection rules require inspection.
Extremes, solicitation and manipulation can distort the corpus.
Recurring themes matter only after duplication and cohort controls.
Correction rights and moderation procedures differ by platform.
Concrete cases reveal where quality changes.
These cases use different claim structures so the same framework does not become repetitive. Each one names the valid support, the critical limit, the control that matters most and the next evidence class required for a stronger conclusion.
“The platform prohibits automated scraping.”
The current terms or policy page is the preferred source for what the platform declares. It has direct institutional competence and access to the rule. It does not, by itself, establish how consistently the rule is enforced, how many exceptions exist or how users behave in practice. Preserve the effective date and exact version because policies change without the underlying URL changing.
“System A returns results faster than System B.”
A reproducible benchmark can be strong evidence when hardware, network, workload, warm-up, sample size, failure handling and aggregation are equivalent. A vendor benchmark may still be informative, but the incentive pressure requires closer inspection of selected workloads and omitted competitors. The claim must specify which latency statistic matters; average, median and tail latency answer different questions.
“The service was unavailable for 43 minutes.”
An operator status record may directly declare incident start and resolution, while external monitoring can observe reachability from selected locations and users can document functional impact. These sources do not necessarily conflict when their clocks or definitions differ. Quality depends on access to the specific layer: operator telemetry, network endpoint, transaction path or user-facing function.
“Brand X leads the market with 31% share.”
A market estimate is only as strong as its market definition, denominator, geography, period and data acquisition model. Two reports showing the same number may share an upstream panel and therefore provide one evidence lineage, not two independent confirmations. Inspect whether “share” means revenue, shipments, active users, search visibility or surveyed preference before comparing values.
“The intervention caused the observed improvement.”
Temporal sequence and correlation are insufficient for causation. Source quality turns on design: assignment, control group, baseline equivalence, attrition, measurement validity, confounding and analysis choices. A qualified expert can explain a plausible mechanism but cannot replace the observed comparison. Strong causal language requires a design capable of excluding credible alternatives at the claimed scope.
“This change indicates a ranking-system update.”
An experienced search analyst may recognize a pattern, but expertise alone does not grant direct access to a private system. The source can offer a reasoned hypothesis if observations, alternative explanations and uncertainty are visible. It should not convert a plausible interpretation into an observed internal cause. Later official confirmation may strengthen the attribution without retroactively improving every earlier statement.
“I received this message after submitting the form.”
A preserved screenshot and message header can strongly support what one person received at one time. The witness has direct access to the experience, but the evidence does not establish that all users received the same message or why the system produced it. Remove unnecessary personal data while retaining the fields needed to verify time, sender, context and sequence.
“Several models gave the same answer.”
Repeated model outputs are observations of those runs, not automatically independent evidence for the answer. Models may share training sources, retrieval indexes or common misconceptions. Preserve prompt, model version, settings, date and cited sources. Treat the output as a discovery or synthesis layer, then evaluate the underlying evidence lineages that actually support the claim.
A conclusion is defensible when another reviewer can inspect it.
Source-quality evaluation should leave a compact record of what was judged, why, against which claim and with what limitations. The audit record preserves reasoning without pretending that qualitative judgments are perfectly objective.
Record the judgment, not just the verdict.
A label such as “credible” cannot be reviewed. A useful record identifies the source version, the exact claim, each dimension’s observed basis, unknowns, conflicts, permitted use, required corroboration and review date. Keep observations separate from evaluator inference so later evidence can revise the judgment cleanly.
Canonical locator, author or responsible organization, document title, publication or capture date, edition, version and preserved copy. Note whether the record is original, derivative, translated, summarized or transformed.
Exact proposition, subject, attribute, population, geography, time and required precision. If the source supports only part of a sentence, split the sentence into atomic claims before evaluation.
Visible evidence for competence, access, transparency, incentives, consistency and accountability. Record “unknown” where evidence is absent; do not silently convert missing information into a favorable or unfavorable assumption.
The dimension most capable of invalidating or narrowing the intended use. Examples include indirect access, an obsolete version, a nonrepresentative sample, an undisclosed transformation or a conflict that has not been resolved.
State the strongest claim the source can carry alone. Distinguish direct declaration, direct observation, controlled estimate, expert interpretation, contextual testimony and discovery lead.
Name the evidence family required next and why it adds information. More copies of the same upstream record do not repair missing independence; choose a source with a different access path or method.
Set a date or event that forces reassessment: policy revision, new release, changed methodology, correction, conflicting study, market shift or expiration of the claim’s validity window.
Stress the source before letting it carry weight.
These tests are designed to expose overreach. Passing one does not guarantee quality; failing one identifies a concrete repair, qualification or replacement requirement.
Would the standard change if the conclusion reversed?
Apply the same demand for sample quality, transparency and uncertainty to favorable and unfavorable outcomes. Selective skepticism is evidence of evaluator bias, not source strength.
PASS / SAME CONTROL, EITHER RESULTCould a famous logo replace the actual record?
If the judgment depends mainly on publisher reputation, inspect the document again. Determine who authored it, what evidence it contains, whether it is current and which claim it directly addresses.
PASS / RECORD INSPECTION SURVIVESWhat credible observation would contradict it?
A source is easier to evaluate when the claim can fail. If no possible evidence would change the conclusion, the statement is too vague, normative or insulated from testing.
PASS / DISCONFIRMING STATE NAMEDDoes the conclusion exceed the observed universe?
Check movement from one user to all users, one country to a global claim, one test to ordinary operation, one month to a durable trend or one source to an entire category.
PASS / INFERENCE MATCHES COVERAGEWould the same record answer the question next year?
Stable definitions may remain useful while prices, rankings, policies, leadership, interfaces and market states decay quickly. Set freshness according to volatility, not publication prestige.
PASS / VALIDITY WINDOW DECLAREDAre apparent confirmations truly independent?
Trace repeated figures, quotations, charts and claims upstream. Ten publications repeating one study form one originating evidence line plus nine distribution records.
PASS / ORIGINS COLLAPSEDCan the method support the number of decimals?
High precision may be cosmetic when sampling, classification or measurement uncertainty is large. Match claim precision to the weakest material uncertainty in the evidence chain.
PASS / PRECISION IS JUSTIFIEDWhat happens when an error is found?
Look for named responsibility, visible revisions, preserved prior versions and a correction channel. A source that silently overwrites its record weakens temporal and accountability controls.
PASS / ERROR PATH IS TRACEABLEUse the framework without flattening judgment.
These answers address the decisions that most often create weak research: confusing authority with truth, treating transparency as proof, counting duplicated sources, and assigning confidence before claim-source fit is established.
Is an official source always the highest-quality source?
No. An official source is usually preferred for its own identity, rules, specifications and declared actions because it has direct institutional access. It may be insufficient for claims about independent effectiveness, public impact, compliance in practice or comparative superiority. Match official access to the exact proposition.
Can a biased source still provide strong evidence?
Yes. Incentive pressure does not erase competence or access. A seller may be the best source for its current price and documented specification. The pressure becomes material when the claim concerns performance, superiority or outcomes that benefit the source. Disclosure, independent tests and correction costs can constrain that risk.
Does transparency make a source correct?
No. Transparency makes the basis inspectable. A fully documented study can still use a poor sample or invalid measure, but reviewers can identify the failure. Opaque sources restrict evaluation because unknown methods, exclusions and transformations cannot be tested.
How should anonymous sources be handled?
Separate the observable record from unverified identity claims. Anonymous testimony may provide a discovery lead or document a bounded experience when supporting artifacts are preserved. It should carry less weight where competence, access, conflicts of interest or accountability depend on knowing who produced the account.
Should source quality be converted into one score?
A score can help triage a large collection if its dimensions and thresholds remain visible, but it should not replace the profile. A high average can conceal a fatal weakness such as no access to the event, obsolete data or a method incapable of measuring the claimed attribute.
What if two high-quality sources disagree?
Do not average them automatically. Compare the claims at atomic level, then inspect time, geography, population, definitions, measurement, version, access and lineage. Both sources may be high quality within different boundaries. If the contradiction survives alignment, preserve it and lower conclusion confidence.
How many sources are enough?
Count independent evidence lines, not pages or citations. Sufficiency depends on decision stakes, uncertainty, source diversity and how directly each line tests the claim. One primary record may establish a declared policy; a broad causal or market claim may require multiple methods and populations.
Can user-generated content be high quality?
It can be strong for a preserved first-person experience and useful for recurring pattern discovery. It is weaker for prevalence, cause and market-wide conclusions unless identity, duplication, selection, timing and category rules are controlled. Evaluate the corpus construction separately from each testimony.
How does freshness affect source quality?
Freshness is claim-relative. A historical record may be ideal for a past state and poor for a current one. Define the validity window using volatility: technical interfaces, prices and rankings may decay quickly, while stable conceptual definitions can remain useful longer.
What is the difference between quality and corroboration?
Quality evaluates whether one source is fit for one claim. Corroboration asks whether independent evidence lines support the same bounded proposition. Several weak or derivative sources do not become strong through repetition, and one high-quality source does not automatically provide independent confirmation.
When should a source be rejected completely?
Reject it for the intended claim when identity cannot be resolved enough to evaluate it, the source had no plausible access, the record is materially altered, the method cannot observe the claimed attribute, or known fabrication destroys the relevant account. It may still remain useful as evidence of misinformation circulation or as a discovery lead, with that role stated explicitly.
What should a published evaluation disclose?
Disclose the bounded claim, source version, relevant dimensions, critical limitation, permitted use, unresolved unknowns, corroboration need and review date. Readers should be able to see why the source was used and where its authority stops without reverse-engineering an unexplained score.
Quality bounds the source. Collection preserves the record.
EVD/04 evaluates fitness for a claim. EVD/05 owns the controlled acquisition and preservation procedures needed to create stable evidence records.
Digital records, observations and claim-specific support.
EVD / 02 CLASS Evidence Types & ClassesOrigin, directness, form, independence and temporal state.
EVD / 03 ORIGIN Source ProvenanceIdentity, custody, version, authorship and transformation history.
EVD / 04 QUALITY Source QualityCompetence, access, transparency, incentives and accountability.
EVD / 05 CAPTURE Evidence Collection & PreservationAcquisition context, stable records, snapshots and chain of custody.
EVD / 06 CONFIRM Corroboration & TriangulationIndependent agreement across sources and observation modes.
EVD / 07 CONFLICT Conflicting Evidence ResolutionDiagnosing disagreement through scope, timing and lineage.
EVD / 08 TIME Temporal Validity & Evidence DecayFreshness windows, volatility, supersession and re-collection.
EVD / 09 SUPPORT Claim–Evidence MappingConnecting observations to exact claims and inference boundaries.
EVD / 10 CONFIDENCE Confidence CalibrationTransparent confidence states without false certainty.
EVD / 11 UNKNOWN Evidence Gaps & UnknownsMissing observations and unresolved alternatives.
EVD / 12 SYNTHESIS Evidence Synthesis & Decision ReadinessCombining support, conflict and uncertainty into a decision state.