Pragmatic Free Trial Meta
Pragmatic Free Trial Meta is a non-commercial, open data platform and infrastructure that supports research on pragmatic trials. It shares clean trial data and ratings using PRECIS-2 allowing for multiple and diverse meta-epidemiological studies to evaluate the effect of treatment on trials that employ different levels of pragmatism and other design features.
Background
Pragmatic studies provide real-world evidence that can be used to make clinical decisions. However, the usage of the term "pragmatic" is inconsistent and its definition and evaluation requires clarification. The purpose of pragmatic trials is to guide the practice of clinical medicine and policy decisions, not to prove a physiological or clinical hypothesis. A pragmatic trial should aim to be as close as is possible to real-world clinical practices that include recruitment of participants, setting, design, delivery and implementation of interventions, determination and analysis outcomes, and primary analysis. This is a major difference between explanatory trials, as described by Schwartz and Lellouch1 that are designed to confirm the hypothesis in a more thorough manner.
Trials that are truly pragmatic must be careful not to blind patients or clinicians as this could lead to bias in estimates of the effects of treatment. Practical trials should also aim to recruit patients from a variety of health care settings to ensure that the results can be compared to the real world.
Additionally, pragmatic trials should focus on outcomes that are crucial to patients, like quality of life or functional recovery. This is particularly important when trials involve surgical procedures that are invasive or may have serious adverse effects. The CRASH trial29, for example, focused on functional outcomes to compare a two-page report with an electronic system for monitoring of patients in hospitals suffering from chronic heart failure. Similarly, the catheter trial28 utilized urinary tract infections that are symptomatic of catheters as the primary outcome.
In addition to these aspects pragmatic trials should also reduce the procedures for conducting trials and requirements for data collection to reduce costs and time commitments. Finally pragmatic trials should strive to make their findings as relevant to actual clinical practice as possible by making sure that their primary analysis follows the intention-to treat approach (as described in CONSORT extensions for pragmatic trials).
Despite these guidelines, many RCTs with features that challenge the concept of pragmatism have been mislabeled as pragmatic and published in journals of all types. This can lead to false claims about pragmatism, and the use of the term should be standardised. The development of the PRECIS-2 tool, which offers an objective standard for assessing pragmatic characteristics, is a good first step.
Methods
In a pragmatic research study the aim is to inform clinical or policy decisions by demonstrating how an intervention could be integrated into routine care in real-world settings. This differs from explanation trials that test hypotheses regarding the cause-effect relationship in idealised conditions. Therefore, pragmatic trials might have less internal validity than explanatory trials, and could be more susceptible to bias in their design, conduct, and analysis. Despite these limitations, pragmatic trials may provide valuable information to decision-making in healthcare.
The PRECIS-2 tool scores an RCT on 9 domains, ranging between 1 and 5 (very pragmatic). In this study, the recruit-ment, organisation, flexibility: delivery, flexible adherence and follow-up domains received high scores, however, the primary outcome and the method for missing data were not at the limit of practicality. This indicates that a trial can be designed with effective practical features, but without harming the quality of the trial.
It is difficult to determine the amount of pragmatism in a particular trial because pragmatism does not have a binary characteristic. Some aspects of a research study can be more pragmatic than others. Moreover, protocol or logistic modifications made during a trial can change its pragmatism score. In addition 36% of the 89 pragmatic trials discovered by Koppenaal and co. were placebo-controlled or conducted prior to approval and a majority of them were single-center. Thus, they are not very close to usual practice and are only pragmatic in the event that their sponsors are supportive of the lack of blinding in these trials.
Additionally, a typical feature of pragmatic trials is that researchers try to make their results more meaningful by analysing subgroups of the sample. This can result in unbalanced analyses with less statistical power. This increases the possibility of missing or misdetecting differences in the primary outcomes. In the case of the pragmatic studies that were included in this meta-analysis this was a major issue since the secondary outcomes weren't adjusted for variations in the baseline covariates.
Additionally, pragmatic trials can also be a challenge in the gathering and interpretation of safety data. It is because adverse events are typically self-reported and are susceptible to errors, delays or coding differences. Therefore, it is crucial to improve the quality of outcome ascertainment in these trials, and ideally by using national registries rather than relying on participants to report adverse events in the trial's database.
Results
Although the definition of pragmatism may not require that all trials be 100% pragmatic, there are benefits to incorporating pragmatic components into clinical trials. These include:
By including routine patients, the trial results can be translated more quickly into clinical practice. However, Highly recommended Web-site may also have drawbacks. The right type of heterogeneity, for example could allow a study to expand its findings to different patients or settings. However the wrong kind of heterogeneity can decrease the sensitivity of the test and, consequently, reduce a trial's power to detect small treatment effects.
A variety of studies have attempted to classify pragmatic trials using different definitions and scoring methods. Schwartz and Lellouch1 created a framework to differentiate between explanation studies that confirm a physiological hypothesis or clinical hypothesis, and pragmatic studies that guide the choice for appropriate therapies in clinical practice. The framework consisted of nine domains scored on a 1-5 scale which indicated that 1 was more informative and 5 was more practical. The domains included recruitment, setting, intervention delivery and follow-up, as well as flexible adherence and primary analysis.
The original PRECIS tool3 included similar domains and scales from 1 to 5. Koppenaal et al10 created an adaptation to this assessment dubbed the Pragmascope that was easier to use in systematic reviews. They discovered that pragmatic reviews scored higher in all domains, but scored lower in the primary analysis domain.
This difference in the main analysis domain could be explained by the fact that most pragmatic trials analyse their data in the intention to treat method, whereas some explanatory trials do not. The overall score for pragmatic systematic reviews was lower when the domains of management, flexible delivery and following-up were combined.
It is important to understand that a pragmatic trial doesn't necessarily mean a low quality trial, and there is an increasing number of clinical trials (as defined by MEDLINE search, however it is neither sensitive nor specific) that employ the term "pragmatic" in their abstract or title. The use of these terms in titles and abstracts may suggest a greater awareness of the importance of pragmatism, but it is unclear whether this is reflected in the content of the articles.
Conclusions
As the value of real-world evidence becomes increasingly commonplace, pragmatic trials have gained popularity in research. They are randomized trials that evaluate real-world alternatives to new treatments that are being developed. They involve patient populations more closely resembling those treated in regular care. This method can help overcome the limitations of observational studies, such as the biases that arise from relying on volunteers and the lack of availability and coding variability in national registry systems.
Pragmatic trials also have advantages, like the ability to leverage existing data sources and a greater probability of detecting meaningful differences from traditional trials. However, these trials could be prone to limitations that compromise their validity and generalizability. Participation rates in some trials could be lower than anticipated because of the healthy-volunteering effect, financial incentives or competition from other research studies. A lot of pragmatic trials are restricted by the necessity to recruit participants in a timely manner. Certain pragmatic trials lack controls to ensure that any observed differences aren't due to biases during the trial.
The authors of the Pragmatic Free Trial Meta identified 48 RCTs that self-labeled themselves as pragmatic and were published up to 2022. They assessed pragmatism by using the PRECIS-2 tool that includes the domains eligibility criteria, recruitment, flexibility in adherence to intervention and follow-up. They discovered that 14 of the trials scored as highly or pragmatic practical (i.e. scores of 5 or higher) in one or more of these domains, and that the majority of these were single-center.
Trials with high pragmatism scores tend to have more lenient criteria for eligibility than conventional RCTs. They also contain populations from various hospitals. According to the authors, may make pragmatic trials more useful and relevant to everyday practice. However, they don't guarantee that a trial will be free of bias. In addition, the pragmatism that is present in a trial is not a predetermined characteristic; a pragmatic trial that doesn't possess all the characteristics of a explanatory trial can produce reliable and relevant results.