Comments on the 3-year work plan
for the Methodology Working Party (MWP) of the EMA

Uncategorized

The Methodological Working Party (MWP) of the EMA has published a draft 3-year work plan [1] on October 30, 2023, which I have commented below.

Content

  • L43-45:
    • The strategic goals of The Methodological Working Party (L43-45) are supported. However, innovation and best-practice efforts should be complemented by efforts to provide a standardized, “same practice” harmonized assessment of methodological aspects of clinical trials, e.g. the application of estimands for non-inferiority and equivalence trials in phase 2/3.
    • The MWP may add efforts to improve the influence and role of methodological experts, e.g. statisticians, throughout the regulatory procedures, e.g. scientific advice or marketing authorization, to ensure that methodological advice is presented correctly by the responsible subject matter experts to applicants.
    • The MWP may add efforts to support the open-source software development movement by providing regulatory guidelines, e.g. regulatory submission using R, or channels of communication with key opinion leaders and relevant working groups, e.g. ASA BIOP SWE, OpenStatsWare, https://rconsortium.github.io/asa-biop-swe-wg/.
    • The MWP may add efforts to improve the standard of clinical outcomes, esp patient-reported outcomes, focusing on modern psychometric methods, e.g. item-response theory.
    • The MWP may add efforts to support open access publication of methods and clinical trial results, which would support the dissemination of knowledge.
    • The MWP may add efforts to extend its scope to health-economics/HTA assessments or least to establish collaborations with concerned working groups within the EU network. This seems timely in anticipation of the Joint Clinical Assessment. The Joint Clinical Assessment will aim to meet requirements by payers, which are typically addressed in post-marketing studies, already in pivotal studies for regulatory approval, therefore, a mutual understanding of methodological concepts would be useful.
    • The MWP may add efforts to investigate methodological approaches to integrate environmental aspects of drug production and consumption into the benefit-risk assessment of drugs (see also HTA assessment).
  • L134: The MWP may consider reviewing the evidence on the validity of RWE approaches (external control groups) to replace RCTs from historical RCT data or commissioning such studies.
  • L134: The MWP may promote the development statistical methods of Federated Analyses, e.g. for pharmaco epidemiologic studies (cf Gedeborg et al. 2023, PMID: 36527437)
  • L167, 251, 362, 388: The MWP may provide clearer guidance on the application of Bayesian methods in drug development because currently the position sometimes seems to be a catch-22 (“You can use Bayesian methods, if they give the same results and conclusions as frequentist methods.”). The MWP may consider the comments by Igl & Constant:
    • Igl, W. & Constant, J. (2023). Hype, not hope – A comment on Ruberg et al. (2023) “Application of Bayesian approaches in drug development: Starting a virtuous cycle”,  https://wilmarigl.de/?p=689
    • Igl, W. & Constant, J. (2023). Hype, Not Hope: Using Bayesian Methods in Drug Development. ISPOR-EU, Copenhagen, Denmark, 13-15 November,  Abstract and Poster via https://wilmarigl.de/?page_id=7
    • Igl, W. & Constant, J. (2023). Hype, Not Hope: Using Bayesian Methods in Drug Development. Bayesian Biostatistics Conference (Bayes 2023), Utrecht, Netherlands, 25-27 October, 2023 Abstract and Presentation via https://wilmarigl.de/?page_id=7.
  • L161: The MWP should also consider the reflection paper on adaptive designs (CHMP/EWP/2459/02, 2007) for a high-priority update and upgrade to a guidance document (cf FDA Guidance, FDA-2018-D-3124, 2019!).
  • L170: The revision of the small population guideline may consider current developments in cell and gene therapy.
  • L190: The MWP may address principal questions on AI, for example, regarding the requirements for an explicit, human-readable statistical model to approval, reporting of results, e.g. statistical significance and clinical relevance (e.g. standardized effects sizes), which are not always available for such methods,  requirement for open-sourcing AI models to allow reproducibility and scientific discourse of the applied method, added value of AI models vs conventional statistical methods (for patients).
  • L200-203: International cooperations may be complemented by clearer, explicit guidance on differences between the legal and methodological frameworks by regulators, e.g. EMA vs FDA, for example, concerning the handling of censoring in time-to-event analysis.
  • L280-281, L461-462: The active and passive participation of subject matter experts at conferences is welcome to get insights into interpretation of guideline documents, preferences, and regulatory decision-making in general, also in direct communication.
  • L280, 461: The MWP may consider publishing a quarterly newsletter with relevant methods publications, legal changes, or impactful regulatory decisions, e.g. marketing approvals with a focus on methodological questions.

Wording, Form and Style

  • L106: The generic term “Modelling & Simulation” should be gradually replaced by a more descriptive term, e.g. “model-informed drug discovery and development (MID3)” or other.
  • L149, 213: The term “estimand” should be gradually replaced or at least complemented with the expression “treatment effect of interest” or “detailed clinical question”. The term is partially a misnomer because the concept comprises “what should be estimated”, but also “what can be estimated” (depending on methodological context).
  • L317, “accuracy, precision, reliability and comparability of device-based diagnostic tests” should say “accuracy, precision, reliability and validity of device-based diagnostic tests” or just “accuracy and precision of device-based diagnostic tests”
  • L340, passim: Explain all abbreviations, e.g. RP

References

[1] EMA (2023-10-30). Draft revised consolidated 3-year work plan for the 4 Methodology Working Party (MWP) (EMA/CHMP/478317/2023). https://www.ema.europa.eu/en/documents/work-programme/draft-revised-consolidated-3-year-work-plan-methodology-working-party-mwp_en.pdf