DIY Guide to Assessing Medical Information and Research (Part 2)
Monday December 2, 2013 | Abby White, WE C Hope CEO
This is the second part of a two-post blog. Follow this simple guidance to effectively read and assess medical information, medical news stories and published research articles.
Evaluating Published Clinical Study Research
We talk a lot about evidence-based research at World Eye Cancer Hope, but what does that really mean? Simply put, evidence-based research asks:
- What are the precise characteristics of patients in this study?
- What is the specific therapy, dosage and supportive care being tested?
- What specific outcomes are we measuring?
The evidence-based approach compels researchers to design studies around clearly defined questions that enable accurate analysis and transparent reporting of their findings. This generates recommendations that are underpinned by solid data.
Even though you may have the best doctor in the world, your personal ability to evaluate medical information is vital to your ability to advocate for yourself. No doctor can know or understand everything about a particular disease. When you understand medical research process and evidence, you can critically evaluate science and resulting data, and be a more active participant in deciding your / your child’s care.
Studies may be poorly designed, and reports may be poorly presented. Their authors sometimes draw conclusions that are not supported by their own data. Conflicts of interest such as current or future funding, media exposure, commercial drug sales and media sales influence the way in which the findings care presented. These conflicts can lead to misinterpreted and misrepresented data.
Research suggests 30-40% of published study abstracts misrepresent results.1 Following the guidance below as you assess the whole article can help you separate the evidence from myth, hype and bias, determine study quality and spot conclusions that may be misleading, inaccurate or incomplete.
Obtain the Full Article
Reading beyond the abstract is vital to gain in depth knowledge of the study’s design and findings. If you have read a news article or blog post about the research, but it does not reference the article, contact the author to request the authors, title and publication. Alternatively, contact the researcher or the institution that funded the research, if mentioned in the article. You can usually find the researcher’s contact information by searching for their faculty profile on the website of the institution at which they work.
The article’s author may send you the article, or a link to it. If you can access only the abstract, contact the lead author or the publication and request a copy of the full article. Explain that you are a parent or survivor affected by retinoblastoma who wishes to evaluate the evidence. Journals may provide the article at no cost, or may charge a per-article fee for online access or PDF download.
Evaluate the Publication
Check where the report is published. Studies published in peer-reviewed medical journals have more credibility than reports from the researcher’s web site or from a commercial entity. A finding reported at medical meeting usually requires additional research before it is ready for journal publication.
Check the Peer Review Process
Peer review is the evaluation of work by people with similar expertise to the authors. The process of review by scientific journals to determine which papers are worthy of publication is described in the Journal web site. If an article is not “peer reviewed” its credibility is considerably lessened. However, peer review does not guarantee that an objective view is presented.
Check for Conflicts of Interest
A conflicts of interest or disclosure statement is usually included as a side box towards the beginning or end of the report. This statement should disclose the study’s funders and any vested interests of the investigators or funders.
Evidence shows that industry-funded studies are often biased with inaccurately interpreted results.2 Independent trusts, foundations and agencies often employ a peer-review process to evaluate and award grants. Findings from studies funded in this way may be more credible and less biased than those generated from studies whose funders have a vested interest in the results.
Commercial sponsorship may influence the investigator’s findings, but this is not always the case. Industry-university research partnerships can be valuable, but this is a difficult factor in assessing outcome validity.
Assess the Investigators and Authors
Look for research conducted by a group of clinicians (doctors, nurses etc) and clinical trial scientists at a hospital or group of hospitals with a specialist retinoblastoma program. Assess with caution reports written by a single author, or by a team from single for-profit institution.
The first named author may not be the principal investigator. High level journals provide a paragraph detailing each author’s role in the study. Try to identify the principal investigators, lead author and their credentials. Google author names to identify their research credentials and experience of the question being investigated. University faculty profiles usually summarise qualifications, training and experience.
Identify the Study Question
What specific question is the research study attempting to answer? Proceed with caution if you cannot identify this question in the introduction. Balanced assessment of results relies on a precise question guiding the study.
Determine the Study Format
Was the study in vitro or in vivo, conducted in a lab or in clinical care? Was it randomized? Answering these questions is vital to assess relevance to your family.
In vitro (“within the glass”) describes experiments conducted in a controlled environment outside a living organism. Many cell biology studies are conducted in vitro. When test conditions don’t replicate conditions inside the human body, their results may not represent the situation arising in a human.
In vivo (“within the living”) describes experiments conducted on the tissue of a whole, living organism. Animal testing and clinical trials are forms of in vivo research.
Lab research is usually years from clinical application. Results of human clinical trial research are likely to be more meaningful to your child and family.
Randomized clinical trials are the gold standard for evidence based cancer research. These trials divide patients into two or more groups, comparing a standard treatment with experimental therapy. A computer randomly assigns the child to one treatment “arm”, preventing bias. Uncontrolled randomised trials involve two arms (standard and experimental). Controlled trials involve 3 or more arms, and the group receiving standard treatment becomes the control. They enable investigators to scientifically compare the two experimental therapies.
For an explanation of the different stages of clinical research see the Clinical Trials section of our website.
Carefully read the Methods section, which outlines the experimental design. As you read, look for the following information:
- How many children and how many eyes were studied, and for how long? A small number of participants does not invalidate a study, but the results may be questionable. When assessing a treatment, researchers need to study many patients over many years to validate effectiveness and safety of the therapy.
- What were the inclusion and exclusion criteria for participation? Treatments may be appropriate or effective only for certain groups of children – e.g. infants, bilateral diagnosis or eyes with disease of a particular classification. Studies may exclude children who received prior or concurrent therapy, both of which may impact of the therapy being studied. Are these criteria clearly stated? Are the characteristics of the study participants close to those of your child? If they are not, the study results may not be meaningful to your child’s care.
- What is the specific protocol of the therapy under investigation: Dosage, frequency, concurrent therapies and supportive care should be described.
- What are the success criteria? How do the researchers define effectiveness of the studied therapy? Do they assess only eye salvage or vision salvage also? Do they explore impact of therapy on the whole child? Does success criteria relate to short, medium or long term effects?
Assess the Results, Discussion and Conclusions
Do the described results correspond with the core research question and success criteria defined in the Methods section? If a therapy was tested, were side effects noted? If so, do they outweigh the therapy’s potential benefits? Has the therapy or drug been approved by regulators? If not, are further trials underway, and is there a timeframe for approval? A drug may be tested for many years before it is deemed safe for use in humans, especially in children.
Are results thoroughly analysed in the discussion section? Note particularly the discussion of any side effects or adverse events. Proceed with caution if such incidents noted in the results are only referenced in, or omitted from, the discussion.
Are the authors’ conclusions supported by strong data? Do the study results and the article conclusions reflect those presented in the abstract?
Is the report original work by the principal investigator, a different way of presenting previously published work by the same individual or team, or science gathered from studies by other people? Have the results been replicated by other researchers? Independent corroboration of results helps validate results, especially when the number of participants is small or there is a real or perceived conflict of interest.
- Pitkin R. M., Branagan M. A., Burmeister L.F. Accuracy of data in abstracts of published research articles. JAMA 1999 Mar 24-31;281(12):1110-1.
- Bekelman J. E., Li Y, Gross C.P. Scope and impact of financial conflicts of interest in biomedical research: a systematic review. JAMA 2003 Jan 22-29;289(4):454-65.