When the Radiomics Quality Score (RQS) was presented to the scientific community back in 2017, its authors aimed to introduce a tool for a rapid and effective evaluation of radiomics studies’ scientific/clinical merit. Conceived as a quality seal to be published alongside presented results and newly proposed radiomics models, the RQS has instead mostly been adopted by researchers as a scoring system to retrospectively evaluate the published literature (predominantly focused on specific radiomics applications). Working within the Radiomics Auditing Group Initiative (EuSoMII), we analyzed 44 of these topic-specific RQS-based systematic reviews to obtain a more comprehensive assessment of the literature and turn the spotlight onto the still open and pressing methodological challenges. Indeed, while the limitations of RQS must be acknowledged (e.g., unbalanced towards handcrafted radiomics, relatively low reproducibility, nonlinear scoring system), we also recognize that it has become the de facto standard for estimating the methodological quality of radiomics studies. As we looked at the evidence, we discovered that irrespective of which field of application (e.g., oncological vs. non-oncological, neuroimaging vs. body imaging), the overall methodological quality of radiomics studies is not satisfactory (with a median RQS of 21%). Some could argue that radiomics is still in its infancy; and at such a stage, exploratory and preliminary studies are somewhat expected to suffer from limitations, representing the lowest level of the evidence pyramid. While this view is partly shareable, we should remember that we are currently working on the premises upon which future studies will be designed. If the foundations are not sufficiently robust, it will be much harder to build a scientifically sound and reliable future for radiomics.
Key points
- Radiomics is a promising high-throughput method that may generate novel imaging biomarkers to improve clinical decision-making process, but it is an inherently complex analysis and often lacks reproducibility and generalizability.
- The Radiomics Quality Score serves a necessary role as the de facto reference tool for assessing radiomics studies.
- External auditing of radiomics studies, in addition to the standard peer-review process, is valuable to highlight common limitations and provide insights to improve future study designs and practical applicability of the radiomics models.
Authors: Gaia Spadarella, Arnaldo Stanzione, Tugba Akinci D’Antonoli, Anna Andreychenko, Salvatore Claudio Fanni, Lorenzo Ugga, Elmar Kotter & Renato Cuocolo
