jfa journal

AND option

OR option

ON SCHRÖDINGER’S CAT AND EVALUATION OF TRIALS DISRUPTED BY THE COVID19 PANDEMIC: A CRITICAL APPRAISAL

 

M. Cesari1,2, R. Calvani3,4, M. Canevelli5,6, I. Aprahamian7,8, P. de Souto Barreto9,10, D. Azzolino1,2, R.A. Fielding11, N. Vanacore5, M. Inzitari12,13, E. Marzetti3,14

 

1. Department of Clinical Sciences and Community Health, University of Milan, Milan, Italy; 2. Healthy Aging Lab, IRCCS Istituti Clinici Scientifici Maugeri, Milan, Italy;
3. Fondazione Policlinico Universitario «Agostino Gemelli» IRCCS, Rome, Italy; 4. Aging Research Center, Department of Neurobiology, Care Sciences and Society, Karolinska Institutet and Stockholm University, Stockholm, Sweden; 5. National Centre for Disease Prevention and Health Promotion, Italian National Institute of Health, Rome, Italy; 6. Department of Human Neuroscience, Sapienza University of Rome, Rome, Italy; 7. Geriatrics Division, Internal Medicine Department, Jundiaí Medical School, Jundiaí, Brazil; 8. University of Groningen, University Medical Center Groningen, Groningen, The Netherlands; 9. Gérontopôle de Toulouse, Institut du Vieillissement, Centre Hospitalo-Universitaire de Toulouse, Toulouse, France; 10. CERPOP INSERM 1295, University of Toulouse III, INSERM, UPS, Toulouse, France; 11. Nutrition, Exercise Physiology, and Sarcopenia Laboratory, Jean Mayer USDA Human Nutrition Research Center on Aging, Tufts University, Boston, MA, USA; 12. RE-FIT Barcelona research group, Parc Sanitari Pere Virgili and Vall d’Hebron Institute of Research (VHIR), Barcelona, Spain; 13. Department of Health Sciences, Universitat Oberta de Catalunya (UOC), Barcelona, Spain; 14. Università Cattolica del Sacro Cuore, Institute of Internal Medicine and Geriatrics, Rome, Italy.

Corresponding Author: Matteo Cesari, MD, PhD. IRCCS Istituti Clinici Scientifici Maugeri; Via Camaldoli 64, 20138 Milan, Italy. Email: macesari@gmail.com; Twitter: @macesari

J Frailty Aging 2021;in press
Published online May 24, 2021, http://dx.doi.org/10.14283/jfa.2021.23

 


Key words: Research, methodology, publishing, medicine, science.


 

From the beginning of 2020, the world has been fighting the SARS-Cov-2 outbreak. The life of each one of us has profoundly changed. Unavoidably, our clinical routine has drastically modified in its priorities and methodologies (1). The COVID-19 pandemic has also raised significant issues in the field of research. The investigators’ responsibility has increased with the need to thoughtfully weigh the risk-benefit ratio for each protocol in an emergency and evolving scenario (2).
Many projects that had started in the pre-COVID-19 era were halted in the past months, mainly because of 1) the restrictions that governments put in place to limit the spread of the Coronavirus, 2) the inadequacy/inappropriateness of the research activities as planned before the pandemic, and/or 3) the participants’ reluctancy at attending the scheduled visits in facilities usually located within the hospital premises. Some investigators managed to redesign their study methods and make them compatible with the new situation. Someone also proposed possible solutions to tackle the most common issues of the “new” research routine (3). Others faced challenging (if not insurmountable) stalls to solve and eventually decided to stop their studies. Indeed, it was not rare the closure of research activities in some countries, especially during the early period of the pandemic.
How should we judge the findings generated by studies that suffered the effects of the COVID-19 pandemic? In principle, the modification of a protocol “on the way” or the premature ending of a study are deviations that, in “normal” times, would be flagged as major methodological flaws. Purists of research methodology may indeed find these deviations hard to accept. But, can we judge today’s research activities without considering the new context of COVID-19 and the global consequences we all have been experiencing? What degree of flexibility should we apply when reviewing an article containing issues clearly due to the pandemic? In this context, it is also essential to consider the increasing interest around pragmatic research for achieving a wise compromise between the potentially decontextualized findings from rigid methodologies and the heterogenous “real life” scenarios where the study results will be applied (4). Perhaps, the pandemic may lead us to introduce sufficient doses of pragmatism and creativity as core elements of future research. For sure, scientific projects will have to include backup plans considering the possibility of postponing/modifying recruitment strategies, maintaining protocol adherence despite environmental changes, and designing alternative strategies for intervention delivery.
In this issue of The Journal of Frailty & Aging, Tavoian and colleagues (5) present the results of a pilot study initiated before the COVID-19 outbreak. The trial was focused on exploring whether stationary bicycle high-intensity interval training was a more efficient standalone exercise strategy to improve cardiovascular and muscular function compared with resistance or aerobic training in older adults. Unfortunately, the restrictions imposed by the emergent pandemic situation led to a premature stop of the program, precluding the achievement of the expected sample size and leaving the researchers with partial results.
The data could appear insufficient to convey a complete and clear message (at least coherent with the original study aims). For example, the unbalanced representation of women in the study sample (11 vs. only 3 men) might already render the findings of difficult interpretation. On the other hand, we might say that something is always better than nothing! Those partial results could still feed the discussion in the scientific community and serve to support further steps in the field (6). The Chinese philosopher Lao Tzu once said: “The journey of a thousand miles begins with one step”.
The study by Tavoian and colleagues was well-presented and conducted rigorously, methodologically speaking. It also had its record on clinicaltrials.gov, having been registered before the initiation of recruitment. The topic was of interest and novel. Last but not least, the trial was framed as a pilot study with the primary aim of collecting preliminary data for future larger-scale activities. Under these premises, it could have been unfair to reject it simply because an external event (i.e., the COVID-19 pandemic) had disrupted the researchers’ original plans. Are we sure that what is (partially) presented by Tavoian and colleagues may not be sufficient to address questions raised by other researchers? After all, when a study is well-conducted, it is always important to publish it, independently of issues that might have affected its conduct. Of course, the expected impact of an unforeseen event disrupting a pilot study is different from that occurring in a phase 3 trial. This implies that a case-by-case approach is crucial but needs to be structured on specific criteria. Otherwise, the risk could be to fall into a dangerous level of subjectivity or to reject everything in search of a utopic perfection.
No study is perfect, and every research has an own value. It is a matter of clearly and fairly presenting the experience, constructively discussing the strengths and weaknesses. At the same time, the readers should be transparently informed of the deviations imposed by the COVID-19 outbreak and provided with all the information necessary to judge the value of findings and discern potentially relevant results from “background noise”. The need for clarity and details is important for discriminating vague justifications (e.g., “due to the pandemic”) from founded decisions based on political, sanitary, and/or scientific choices (2).
It is also noteworthy that the numbers we report in our tables are persons who have trusted in our research and devoted part of their time to it. The restitution of the study results has to be considered an ethical duty of the researcher in front of the participant. Consistently, many investigations benefit from funds allocated by private or public institutions for specific research purposes and potentially taken from other priorities. It is, thus, also a matter of financial clarity and accountability behavior.
With the aim of providing the correct legitimacy to those valuable research activities critically affected by the COVID-19 pandemic, it has been felt that a set of standardizing criteria could be helpful. Table 1 presents aspects that might support clinicians, researchers and policymakers in better judging the quality of the information provided by partial scientific results.
The post-COVID-19 world we live in is putting us in the position of reconsidering many aspects of our (professional) lives. Different priorities are today modifying our decisions. How we conduct research today is not the same as before COVID-19. The population we are studying is not the same either, and its values and needs might be different (7). Some adaptations of research to the new scenario (e.g., more extensive use of technologies, higher involvement of participants in collecting their data) are necessary to continue advancing in research and avoiding that what was initiated before COVID-19 could go lost.

Table 1. Specific criteria to additionally consider when judging if an article is still worth being published despite COVID-19-related deviations from original plans

 

Conflicts of interest: MC has received honoraria from Nestlè Health Sciences for presentations at scientific meetings and serving as a member of Expert Advisory Boards. MI has received honoraria from Nestlé Health Sciences for serving as an expert advisor. EM has received honoraria from Abbott, Nestlè, Nutricia, and Thermofisher for presentations at scientific meetings. RAF received grants and personal fees from Nestlé. RAF also reports grants from National Institutes of Health (National Institute on Aging), during the conduct of the study; grants, personal fees and other from Axcella Health, other from Inside Tracker, grants and personal fees from Biophytis, grants and personal fees from Astellas, personal fees from Cytokinetics, personal fees from Amazentis, personal fees from Glaxo Smith Kline, personal fees from Juvicell, outside the submitted work. No conflict of interest reported by the other authors.

Acknowledgments: RAF is partially supported by the US Department of Agriculture (USDA), under agreement No. 58-8050-9-004 and by NIH Boston Claude D Pepper Center (OAIC; 1P30AG031679). Any opinions, findings, conclusions, or recommendations expressed in this publication are those of the authors and do not necessarily reflect the view of the USDA.

 

 

References

1. Astrone P, Cesari M. Integrated Care and Geriatrics: A Call to Renovation from the COVID-19 Pandemic. J Frailty Aging. 2021;10(2):182-183. doi: 10.14283/jfa.2020.59
2. Fleming TR, Labriola D, Wittes J. Conducting Clinical Research During the COVID-19 Pandemic: Protecting Scientific Integrity. JAMA. 2020;324(1):33-34. doi:10.1001/jama.2020.9286
3. Kroenke K, Bair MJ, Sachs GA. Continuing Research During a Crisis. J Gen Intern Med. 2021;36(4):1086-1088. doi:10.1007/s11606-021-06636-5
4. Ford I, Norrie J. Pragmatic Trials. N Engl J Med. 2016;375(5):454-463. doi:10.1056/NEJMra1510059
5. Tavoian D, Russ DW, Law TD, et al. Effects of three different exercise strategies for optimizing aerobic capacity and skeletal muscle performance in older adults: a pilot study. J Frailty Aging. doi:10.14283/jfa.2021.21
6. Karmakar S, Dhar R, Jee B. Covid-19: research methods must be flexible in a crisis. BMJ. 2020:m2668. doi:10.1136/bmj.m2668
7. Palmer K, Monaco A, Kivipelto M, et al. The potential long-term impact of the COVID-19 outbreak on patients with non-communicable diseases in Europe: consequences for healthy ageing. Aging Clin Exp Res. 2020;32(7):1189-1194. doi:10.1007/s40520-020-01601-4