Quite often news about novel treatments for specific health conditions causes a stir in the public because of the reported interesting approaches, large treatment effects or side effects, convenience of administration, or even cure for once incurable illness.

One of the most recent misinterpretations of a published clinical study, that getting a haircut can reduce blood pressure levels, shows how unprepared our politicians are despite their professional background.

On the other extreme is the quoting of popular opinions or practices based on flimsy research evidence that has yet to be published or poorly conducted. Some examples of such studies are as follows:

Example 1: Men who do house chores live longer.

Example 2: Eating rice increases the risk of type 2 diabetes.

Example 3: Vitamin C supplementation improves blood pressure control.

Example 4: Cancer chemotherapy causes the spread of breast cancer.

Assessing and using reports of clinical studies

Short and incomplete mass communication of such clinical reports may misinform the public at large or mislead the people with the health conditions into changing health behaviours that could result in detrimental outcomes. Knowledge and skills in assessing and using reports of clinical studies are essential abilities nowadays to differentiate between truth from hype, marbles from pearls, and the most practical value to enable the person to have a more realistic expectation and understanding from the “novel” findings.

As a matter of fact, the knowledge and skills of assessing the quality of the published clinical studies are relatively simple, based on following three criteria: important samples of participants, standard measurement, and usefulness of the findings. This definitely requires you to read more than just the title of the report and, once grasped, they would help you to make the discernment and save you from much public embarassment.

Let’s return to the four examples above, and apply the three criteria of important samples of participants, standard measurement and usefulness of the findings.

Important samples of participants

On important samples (external validity), participants of the research are representative of the population where the findings of the study can be applied. Some of the must-have socio-demographic factors include similarity of proportions in age, gender, ethnicity, socio-economic status (or employment status) and one or two important features of the health condition being researched, such as the stage or severity level of the condition.

In a randomised control trial, the comparability between the groups takes precedence over representativeness of the sampled participants in getting the answer to the research question.

Example 1 is only partly true because the women in that study recorded a larger reduction in death rate i.e. more women gained longer life expectancy by doing house chores than men; and the different levels of physical activity were not observed in high-income countries.

Example 3 was a study on just 27 people, probably mostly the whites in Australia comprising over 80 per cent men aged over 60.

Example 4 became a “bombshell” when the research was just on animal models.

Accurate measurement

On accurate measurement (internal validity), all aspects of measurement in the research are completed using the standard or referent methods of recognised accuracy. This includes using calibrated and validated instruments and standardised measurement processes. The measurement is done by trained personnel and, if necessary, by blinded assessors who are not aware of the treatments the participants receive either in the intervention or control group.

Example 1 has concerns in this aspect - self-reported measure of physical activity using the long-form International Physical Activity Questionnaire (IPAQ), inconsistent outcomes (mortality and cardiovascular-related deaths consisting of incident myocardial infarction, stroke, or heart failure) between non-recreational and recreational physical activities with the group and between the two comparing groups.

Similarly, Example 2 results should be interpreted bearing in mind that self-reported dietary intakes are recorded only once at baseline in the decade-long study among the already high-risk population for type 2 diabetes. Therefore, the results are prone to occurrence by chance, at best suggestive and can not be construed as causal.

Usefulness of findings

On usefulness of findings (statistical validity), which means important outcomes and results that are relevant to the health condition being studied are reported. Numerical results truly represent the research objective, taking into consideration response rate, sample size achieved, fair missing data handling, proper statistical test and analysis strategy with no over-claimed or misleading conclusions. The unusual or unexpected analysis has to be justified.

Example 3 was a study where blood pressure was not the primary objective that the study was originally powered or designed to find an answer to, and it was analysed within-group pre- and post-supplementation not the more robust between-group analysis of differences. Therefore, it is at best hypothesis-generating and not to be taken as conclusive.

I believe the aforementioned criteria can help the general public in judging many clinical reports to an extent that is satisfying and educational, protecting as well as empowering towards overall health and communication.

-- BERNAMA

Associate Professor Dr Boon-How Chew (PhD) is with the Department of Family Medicine, Universiti Putra Malaysia.