Risk of Bias vs Reporting Quality: A Worked Nursing Evidence Appraisal

Ask two different questions about the paper

A trial report can be easy to read and still describe methods that could distort its result. Another report can omit a crucial detail, leaving the reader uncertain about what happened. Those are different problems. Reporting quality concerns whether the methods and findings are described adequately; risk of bias concerns whether the result may be systematically distorted by how the research was designed, conducted, analysed, or reported.

For a nursing evidence assignment, the distinction changes the sentence you can write. 'The report does not describe allocation concealment' identifies missing information. 'The next allocation was visible to the recruiter' identifies a documented feature that may allow selection to be influenced. The first statement does not prove the second, and neither should be replaced by a vague label such as 'poor study.'

Cochrane's guidance separates bias from other considerations, including precision and applicability. CONSORT provides reporting guidance for randomized trials; a completed checklist can help a reader locate information, but it does not certify that every reported method protects the result from bias. Use the appropriate assessment method for the study design and the purpose of the assignment.

The worked example below is entirely fictional. It follows a report of a discharge-information trial through the main article, protocol, registry, and supplement. The purpose is to produce a defensible evidence note and identify what a formal assessment still needs, not to recommend a clinical intervention or award a real study a risk-of-bias judgment.

Define the result before assessing its limitations

The fictional trial compares a plain-language discharge sheet plus usual explanation with a standard leaflet plus the same usual explanation. It reports two outcomes: a recall score at 48 hours and readmission within 30 days. Start by specifying which result your evidence note will use. A concern about how the recall interview was scored does not automatically apply in the same way to a separately recorded readmission outcome.

For this exercise, assess the between-group difference in the 48-hour recall score. Preserve the outcome instrument, measurement time, comparison, and analysis population. Do not change the target halfway through the assessment because the readmission result appears more favorable or because a different time point is more completely reported.

RoB 2, described in Cochrane's chapter on randomized trials, assesses a particular result using structured domains and signalling questions. Its use requires more than reading a title and ticking whether the article says 'randomized.' Select the appropriate version and effect of interest, follow the guidance, and record reasons for the answers. This article's tables are an extraction aid, not a substitute tool or a completed RoB 2 assessment.

If the study is non-randomized, do not force it into a randomized-trial checklist. If the assignment specifies a particular appraisal method, apply that method consistently and explain its scope. The comparison in this article helps separate concepts regardless of which appropriate tool you use.

Read the source pack without treating plans as proof of conduct

Suppose the fictional main article says participants were randomized but gives little detail about concealment or the outcome assessor. The supplement describes a central assignment service. The protocol planned an assessor who would not know the group, while the final report says the nurse who delivered the teaching also conducted and scored the recall interview. The documents answer different questions: what was planned, what was reported as done, and what remains uncertain.

The supplement resolves one information gap if it clearly documents the actual allocation process. The protocol alone would not establish that process was followed. Likewise, the planned independent assessor does not erase the final report's account of a different assessor. Keep the discrepancy visible and examine its implications for the particular outcome rather than choosing whichever document makes the study appear stronger.

Check that the documents refer to the same trial, version, and result. A registry entry for a related project or a protocol amended after results were known may not support the inference you intended. Record dates and relevant sections. If information remains missing after reasonable checks, say which sources were examined and what question they did not resolve.

Contacting authors can sometimes clarify conduct, but an unanswered request is not evidence that the method was defective, and a planned request is not a completed check. In an assignment, distinguish the sources actually obtained from the follow-up that would be needed.

Separate missing information from a plausible bias mechanism

The assessor detail illustrates the difference. If the report merely omitted who conducted the interview, the honest observation would be an information gap. Here the fictional report explicitly identifies the teaching nurse as the scorer and states that the nurse knew the assigned group. Because some answers require scoring judgment, knowledge of the intervention could influence measurement. That is a plausible mechanism to examine under the tool, not proof that the nurse intentionally altered scores.

Missing outcome data need their own reasoning. Fifteen of 100 versus two of 100 participants lack recall scores in the example. The imbalance merits investigation, but there is no universal missing-data percentage that settles the judgment. Ask why results are missing, whether missingness could depend on the true outcome, what information supports that possibility, and whether the analysis and sensitivity analyses address it appropriately.

Suppose the main report says some interviews were missed because participants were too unwell to take part, but does not connect reasons to group or expected recall. That creates a relevant uncertainty: missingness may be related to outcome, but the available information does not establish the direction or size of any distortion. Do not replace the uncertainty with an invented correction factor or assume that removing incomplete records is harmless.

A cluttered table is different again. It may make extraction difficult and should be checked carefully, but visual presentation alone is not a mechanism that biases the trial's effect estimate. If it conceals a discrepancy in denominators, investigate that discrepancy. The evidence problem is the unresolved data meaning, not the table's appearance by itself.

Use the appraisal tool instead of inventing a quality percentage

An article that reports nine out of ten items has not earned '90% trustworthy.' Reporting items do not all represent the same bias mechanism, and a severe problem in one area cannot be cancelled by attractive figures or complete background information. Keep the evidence for each relevant domain visible.

For the recall result, the extraction worksheet below identifies what the fictional source pack can and cannot support. It deliberately does not assign final domain labels. The supplied fragments are insufficient to answer every signalling question, and guessing the missing answers would turn a teaching example into a misleading completed assessment. A real assessment should use the full current tool and source material.

If using RoB 2, apply its own response options and judgment process. Do not introduce a homemade 'unclear' overall category or convert the domain judgments into an average score. An absence of information must be handled where the tool specifies, with a written justification. Record any departure from an algorithm's proposed judgment and the reason, as the guidance requires.

The finished appraisal should also be reproducible. Another reader should be able to locate the source passage, understand the response to the signalling question, and see how the final judgment follows. A coloured traffic-light diagram without those reasons is only a display of judgments, not an explanation of them.

Write an evidence note that preserves what remains unknown

The weak version is: 'This is a high-quality randomized trial because it follows CONSORT.' It gives no result, no mechanism, and no evidence for the judgment. Another weak version is: 'The trial is biased because blinding was not reported.' That converts an information gap into a claim about actual conduct.

A defensible note for the fictional pack is more specific: 'For the 48-hour recall result, the supplement describes central assignment, addressing the main report's allocation-information gap. The final report states that the teaching nurse knew allocation and scored the recall interview, creating a measurement concern that requires assessment against the outcome's judgment component. Missing recall scores were 15/100 versus 2/100; available reasons do not fully explain whether missingness depended on the true score. A complete result-specific risk-of-bias assessment is still required.'

This note does not claim that the result is definitely biased or that its effect is a particular amount too large. It identifies the source-backed concerns and the limit of the current appraisal. If the assignment requires a formal judgment, complete the relevant tool rather than presenting this preliminary note as the finished assessment.

Use the note to constrain the statement supported by the source. A recall result does not establish reduced readmission, and an appraisal of one trial does not by itself determine the certainty of an entire body of evidence. Clinical decisions also require evidence synthesis, applicability, patient priorities, and the relevant professional process; this exercise addresses only one part of evaluating a study result.

Keep a compact audit trail for the next paper

For the next source, start with the exact result, then gather the legitimate linked documents and extract the relevant methods. Separate a reporting gap, a documented methodological feature, and a concern about applicability. That sequence prevents a general impression of the paper from becoming the assessment.

Record source locations as you work. 'Supplement, allocation section' is useful; 'somewhere in the paper' forces another reader to repeat the search. Preserve contradictions rather than silently resolving them in favor of the abstract. If a later clarification changes your judgment, retain the earlier evidence and explain why the assessment changed.

The final summary should state the method used, result assessed, supported judgment or remaining incomplete work, and consequence for the claim you intend to make. It should neither praise reporting polish as protection against bias nor punish missing prose by inventing poor conduct. The value is a transparent connection between evidence, assessment, and the sentence you can responsibly write.