An AI assistant can help a researcher read a collection, draft a codebook, write software and compare accounts. In a sustained inquiry, those outputs become inputs to later work. A misreading of a methods paper can shape a coding rule; the rule can then shape every observation made with it. The reliability question concerns that chain of decisions as well as the individual answer.
TextData treats sources, methods, codebooks and decision records as materials for inquiry. Its design connects methodological research to the tools a study needs. This creates a particular obligation: when a tool is revised, researchers need to see what evidence prompted the change, what the change does, and what has yet to be evaluated. The following failure modes explain why that record matters.
Where the work can go wrong
These seventeen modes distinguish things that can go wrong; they are not mutually exclusive diagnoses. The examples describe possible failures in textual research unless a TextData demonstration is explicitly identified. The mechanisms are explanations to investigate, not inferences about a model’s hidden intentions.
The cited studies are dated evidence for particular mechanisms and evaluation problems. They do not measure the failure rates of today’s frontier models or demonstrate that TextData prevents the failures.
Claims and judgment
Hallucination
Inventing facts or misrepresenting evidence. An invented reference or a summary that drops a source’s qualification can enter a literature review, codebook or measure.
A plausible continuation need not be a true statement. Research on hallucination identifies incentives to guess rather than acknowledge uncertainty as one contributing mechanism. Even when a source exists, compressing its argument can change its scope. [1]
Reasoning errors
Reaching the wrong conclusion from the available information. A source can be quoted correctly while the analysis assigns responsibility to the wrong actor or treats missing evidence as a negative case.
The error may lie in the inference connecting accurate observations: a necessary condition is overlooked, a familiar method is applied outside its assumptions, or an earlier mistake carries forward. Correct execution of a formula or quotation search does not validate that choice.
False confidence
Sounding more certain than the evidence warrants. A provisional code or an incomplete search can be presented as a settled result.
Expressions of certainty are generated language too. A request for a decisive answer can produce decisive prose without establishing that uncertainty has been assessed. The truth of a claim and the confidence attached to it are separate questions.
Sycophancy
Agreeing at the expense of judgment. An analyst can endorse a researcher’s preferred theory without giving contrary accounts equal attention.
Research on models trained with human feedback identifies a possible incentive to match a user’s beliefs. Agreement can be rewarded for sounding helpful or persuasive. This mechanism does not mean that every agreement is ungrounded, or every disagreement is independent. [4]
Generic answers
Replacing a specific thought with familiar language. Broad labels can erase the difference between permission to act and approval after the event.
Familiar formulations fit many prompts. They can sound appropriate while failing to explain what is distinctive about the material. The practical test is what a statement adds to the specific comparison; fluent language alone cannot answer that question.
Bias
Judging people or claims by associations the evidence does not justify. Names, dialects or familiar organizational categories can influence judgments that should depend on the specified evidence.
Learned associations, uneven source coverage and choices in the coding scheme can all affect an assessment. These are possible routes, with different remedies. Removing an identifying field does not establish that the remaining judgment is fair or appropriate.
False assurance
A review or evaluation that misses what matters. Passing quotation checks can be mistaken for correct analysis; several approvals can repeat the same omission.
A reviewer may err under an appropriate standard, or the standard itself may test the wrong property. Shared summaries can transmit omissions between reviewers. Verification research motivates separating checks from the initial answer, but its reported results remain specific to the tasks studied. [6]
Tasks and actions
Incomplete work and false completion
Leaving work undone, then saying it is finished. A corpus may have missing records even though the agent reports a complete review.
An agent can lose a subtask, confuse an attempted action with a successful one, or report the plan it meant to execute. Incomplete work and a misleading completion report are distinct failures; their combination does not by itself establish deliberate concealment.
Gaming the goal
Meeting a success criterion while defeating its purpose. A requirement for valid quotations can be satisfied without identifying all the responsibilities in a passage.
A score or completion rule represents only part of a task. Optimizing that proxy can reward behavior that misses the real objective. A mistake that happens to pass a check is not enough to demonstrate that the system pursued such a shortcut.
Overengineering
Adding work that the task does not need. Extra agents, layers or repeated checks can consume effort without resolving a remaining research question.
A request to be thorough leaves room to substitute visible activity for progress. Without a clear stopping point, each proposed improvement can become another task. This is a workflow explanation to examine, not a diagnosis of a model’s hidden motive.
Goal drift
Solving a different problem. A request to compare rival explanations can turn into a persuasive account of just one.
Successive local choices can move an assignment away from its purpose while each still looks useful. Drift can happen even when the original instruction remains available; it is distinct from forgetting that instruction.
Context and consistency
Forgetting
Losing decisions and context. A resumed analysis can revive a rejected code or lose the reason a category was narrowed.
Relevant material can be absent, omitted from a summary, retrieved too late, or present but used poorly. Long-context research has demonstrated sensitivity to where information appears in the input. Keeping a larger record is therefore not the same as using it correctly. [2]
Inconsistency
Changing the answer without a relevant change in the evidence. The same passage can receive different codes on different runs, changing a comparison or downstream measure.
Sampling can change the generated output. Variation can also arise in the system serving a model, even when decoding appears fixed. A model or workflow update adds another possible source of change. These causes need different comparisons. [3]
Opaque explanations
A plausible account that does not establish why the system acted. An agent’s explanation for excluding a paper may disagree with the record of what it read and did.
An explanation is another generated output, conditioned on the context available when requested. Research on explanation faithfulness has shown that a plausible rationale can omit influences on an answer. A readable rationale is not a causal account of the model’s computation. [7]
Information boundaries
Information leakage
Sharing material beyond its intended audience. Private interview material can enter a public summary or be sent to an unsuitable external service.
Information available for one purpose may be treated as appropriate for another. A summary can carry private details into a wider audience without an attacker being involved. Malicious instructions are another route, but accidental disclosure and induced transfer are distinct.
Prompt injection
Taking instructions from the material being read. A document retrieved as evidence can attempt to redirect an agent’s answer or tool use.
Data and instructions can both appear as language in the model’s input, despite having different authority. Indirect prompt-injection research has demonstrated attacks through retrieved material. A malicious instruction may also survive inside a later summary. [5]
From a detected error to a better instrument
The strongest reason to preserve a correction is that it can change subsequent work. In the methods inquiry behind the Grounded Theory toolkit, a reviewer claimed that a paper did not identify its evaluators as authors. A fresh source reading overturned the claim. The toolkit retained that qualification, so a later analyst could avoid repeating the error. This is a traceable correction, not an estimate of how often the system is right.
The interview application shows another kind of revision. An account described an agent acting without the expected approval, followed by the speaker’s favorable judgment of the result. Review led to a distinction between authorization to act and approval of the outcome. The earlier record and the reason for changing it remain inspectable. That analytical value should not be confused with independent validation: the non-producing reviewer used the same model as the producers, saw their analysis and reviewed selected passages.
The annotation and Gioia examples expose a different limit. An exact quotation can be paired with a wrong actor. A passage can contain an assignment that the output omits. A speaker who reports checking figures has not thereby reported that they never sent the draft. Software can locate the words while leaving all three interpretive problems unresolved. A passed check must therefore be reported together with the property it actually tests.
Methods-review correction · Interview revision · Interpretation tests
Nine workflow responses
Nine practical responses follow from this account. Several have bounded counterparts in TextData; others remain general practices to test. None is a promise that a named defect has been eliminated.
Connect claims to passages
Keep source access, exact locations and mechanical checks alongside an assessment of whether the passage supports the claim at its stated strength.
TextData has source-linked records and quotation checks. Meaning still requires judgment. Evidence and limits
Define completion by the deliverable
Check the requested outputs and recorded actions, including missing items. An intention, attempted operation or final assurance is not the result itself.
TextData demonstrates specified completeness and structure checks, not a general false-completion detector. Evidence and limits
Give checking a purpose and a stopping point
Another check is useful when it can resolve a material uncertainty. Repeating approvals indefinitely can add work without testing anything new.
A bounded-task design principle; reduced overengineering has not been demonstrated. Evidence and limits
Keep the assignment in view
Retain the goal and exclusions; compare a proposed major change with them. A plan should identify what the researcher has authorized.
TextData’s sealed-plan guard is demonstrated for file maintenance only. The broader comparison practice is not an implemented general control. Evidence and limits
Preserve decisions and their reasons
Keep rejected options, unresolved questions, versions and source links so that a resumed inquiry can reconstruct its current state.
Analytical revisions are retained. Reliable retrieval and application of that history are separate questions. Evidence and limits
Ask for evidence that could change the interpretation
Competing explanations and contrary cases give review a substantive task. Concealing the researcher’s preferred answer where feasible is a further practice to test.
TextData requests rivals and contrary evidence. Its interview review was unblinded; concealment was not demonstrated. Evidence and limits
Require a specific contribution from each paragraph
State the point a section must establish and the observations it must explain. Examine whether alternative formulations actually differ in assumptions or implications.
A general writing practice. TextData has no validated control against generic prose. Evidence and limits
Test the reviewer as well as the answer
Use original evidence, separate review contexts and examples whose errors are known. Ask whether the assessment can detect the kinds of mistake that matter.
TextData retains a corrected review error and constructed failures that pass source checks. Independent reference coding remains proposed. Evidence and limits
Keep data separate from authority
Source and trust labels, bounded tools and destinations, and checks before external actions are candidate safeguards to evaluate against hostile material.
These are broader proposed responses. No general prompt-injection defense is demonstrated by TextData. Evidence and limits
What remains to be established
Repeatability requires new model judgments on preserved material, not another export of saved answers. Model changes require comparisons that hold the passages and task fixed while recording the configuration. TextData proposes these evaluations; the historical annotation run does not even identify its backend model or settings. Its retained requests are a starting point for evaluation, not evidence that the evaluation has happened.
Bias, prompt injection and information leakage need their own tests. Withholding interview transcripts from one release addresses a particular distribution decision. It does not establish how an agent behaves with confidential inputs or malicious source material. Nor does a maintained decision record explain every internal cause of a model’s action.
The broader proposition is that researchers can develop instruments through inquiry into methods, then revise them as applications expose weaknesses. Existing components provide examples of that process. Their integration, performance on further material and usefulness to other researchers remain to be evaluated. Source-linked corrections make those questions easier to investigate; they do not settle them in advance.
Sources
Public primary sources for the mechanisms discussed above. TextData examples link to their component accounts through the evidence page.
- Kalai et al. (2025). Why Language Models Hallucinate.
An account of statistical errors and incentives to guess; not a TextData error estimate.
- Liu et al. (2024). Lost in the Middle: How Language Models Use Long Contexts.
Position sensitivity in the models and retrieval tasks studied; not a benchmark of current TextData configurations.
- He and Thinking Machines Lab (2025). Defeating Nondeterminism in LLM Inference.
An engineering account of variation in model serving and batch-invariant inference.
- Sharma et al. (2023; revised 2025). Towards Understanding Sycophancy in Language Models.
Evidence on user-belief matching and human preferences in the tested assistants; not a rate for later models.
- Greshake et al. (2023). Not what you’ve signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection.
Demonstrated attacks through retrieved material in the studied applications.
- Dhuliawala et al. (2024). Chain-of-Verification Reduces Hallucination in Large Language Models.
Verification evaluated on specified factual-generation tasks; no automatic transfer of effectiveness to qualitative interpretation.
- Turpin et al. (2023). Language Models Don’t Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Prompting.
Unreported influences in explanations from the tested older models; a reason to distinguish rationales from causal evidence.