Advertisement

Lung SBRT credentialing in the Canadian OCOG-LUSTRE randomized trial

Open AccessPublished:October 13, 2022DOI:https://doi.org/10.1016/j.ctro.2022.10.002

      Highlights

      • We report on the credentialing experience of a Phase III trial of SBRT versus hypofractionated RT for stage I lung cancer.
      • Elements included a site survey, phantom test, and a standardized case for a peripheral and central tumour.
      • Approximately 1/3 of plans required resubmission due to major deviations,but were not biased towards centre experience versus not.
      • There was contour variability demonstrated, whereas target coverage and dose constraints were largely per-protocol.
      • Such an exercise is important for studies that rely on high precision radiotherapy, and to ensure optimal trial quality.

      Abstract

      Purpose

      To report on the Stereotactic Body Radiation Therapy (SBRT) credentialing experience during the Phase III Ontario Clinical Oncology Group (OCOG) LUSTRE trial for stage I non-small cell lung cancer.

      Methods

      Three credentialing requirements were required in this process: (a) An institutional technical survey; (b) IROC (Imaging and Radiation Oncology Core) thoracic phantom end-to-end test; and (c) Contouring and completion of standardized test cases using SBRT for one central and one peripheral lung cancer, compared against the host institution as the standard. The main hypotheses were that unacceptable variation would exist particularly in OAR definition across all centres, and that institutions with limited experience in SBRT would be more likely to violate per-protocol guidelines.

      Results

      Fifteen Canadian centres participated of which 8 were new, and 7 were previously established (≥2 years SBRT experience), and all successfully completed surveys and IROC phantom testing. Of 30 SBRT test plans, 10 required replanning due to major deviations, with no differences in violations between new and established centres (p = 0.61). Mean contouring errors were highest for brachial plexus in the central (C) case (12.55 ± 6.62 mm), and vessels in the peripheral (P) case (13.01 ± 12.55 mm), with the proximal bronchial tree (PBT) (2.82 ± 0.78 C, 3.27 ± 1.06 P) as another variable structure. Mean dice coefficients were lowest for plexus (0.37 ± 0.2 C, 0.37 ± 0.14 P), PBT (0.77 ± 0.06 C, 0.75 ± 0.09 P), vessels (0.69 ± 0.29 C, 0.64 ± 0.31 P), and esophagus (0.74 ± 0.04 C, 0.76 ± 0.04 P). All plans passed per-protocol planning target volume (PTV) coverage and maximum/volumetric organs-at-risk constraints, although variations existed in dose gradients within and outside the target.

      Conclusions

      Clear differences exist in both contouring and planning with lung SBRT, regardless of centre experience. Such an exercise is important for studies that rely on high precision radiotherapy, and to ensure that implications on trial quality and outcomes are as optimal as possible.

      Keywords

      Introduction

      As technology continues to advance in the field of Radiation Oncology, clinical trials designed to evaluate new techniques in radiotherapy (RT) planning and delivery require high level quality assurance (RTQA) in order to ensure safe and effective treatment delivery [
      • Weber D.C.
      • Poortmans P.M.P.
      • Hurkmans C.W.
      • Aird E.
      • Gulyban A.
      • Fairchild A.
      Quality assurance for prospective EORTC radiation oncology trials: the challenges of advanced technology in a multicenter international setting.
      ]. Robust RTQA has been shown to not only improve protocol compliance, but also affect clinical outcomes in randomized controlled trials (RCTs) evaluating new RT techniques [
      • Jaccard M.
      • Zilli T.
      • Dubouloz A.
      • Escude L.
      • Jorcano S.
      • Linthout N.
      • et al.
      Urethra-sparing stereotactic body radiation therapy for prostate cancer: quality assurance of a randomized phase 2 trial.
      ,
      • Brade A.M.
      • Wenz F.
      • Koppe F.
      • Lievens Y.
      • San Antonio B.
      • Iscoe N.A.
      • et al.
      Radiation Therapy Quality Assurance (RTQA) of concurrent chemoradiation therapy for locally advanced non-small cell lung cancer in the PROCLAIM phase 3 trial.
      ,
      • Fairchild A.
      • Straube W.
      • Laurie F.
      • Followill D.
      Does quality of radiation therapy predict outcomes of multicenter cooperative group trials? A literature review.
      ,
      • Peters L.J.
      • O’Sullivan B.
      • Giralt J.
      • Fitzgerald T.J.
      • Trotti A.
      • Bernier J.
      • et al.
      Critical impact of radiotherapy protocol compliance and quality in the treatment of advanced head and neck cancer: results from TROG 02.02.
      ]. RTQA is also imperative to guide institutions not familiar with a new technology in safe implementation, allowing for adherence with protocol-derived metrics and expert review of plans prior to widespread adoption [
      • Corrigan K.L.
      • Kry S.
      • Howell R.M.
      • Kouzy R.
      • Jaoude J.A.
      • Patel R.R.
      • et al.
      The radiotherapy quality assurance gap among phase III cancer clinical trials.
      ].
      In Canada, an RCT of stereotactic body radiation therapy (SBRT) compared to more conventionally hypofractionated RT (CFRT) in early stage, medically inoperable non-small cell lung cancer (NSCLC), Ontario Clinical Oncology Group (OCOG) LUSTRE was conducted from February 2014 to January 2020. The trial randomized eligible patients in a 2:1 fashion to either SBRT (48 Gy/4 fractions for peripheral or 60 Gy/8 fractions for central tumours) or conventionally fractionated RT (CFRT) (60 Gy/15 fractions). The primary objective of the trial was to determine if SBRT would improve 3-year local control with acceptable morbidity compared to CFRT. It is to date the only randomized SBRT trial including patients with both peripheral and centrally located NSCLC [
      • Swaminath A.
      • Wierzbicki M.
      • Parpia S.
      • Wright J.R.
      • Tsakiridis T.K.
      • Okawara G.S.
      • et al.
      Canadian Phase III randomized trial of stereotactic body radiotherapy versus conventionally hypofractionated radiotherapy for stage I, medically inoperable non-small-cell lung cancer - rationale and protocol design for the Ontario Clinical Oncology Group (OCOG)-LUSTRE Trial.
      ].
      As part of the trial, an extensive prospective RTQA process was undertaken to help improve compliance in lung SBRT practice (especially with respect to central NSCLC, which at the time of study initiation was less widely adopted) and to help train new centres in SBRT lung delivery, as several centres in Canada prior to this study did not have lung SBRT capability. The initial aspect of the RTQA process was a credentialing exercise, which was mandated prior to each centre’s activation on study. The purpose of this credentialing exercise was to determine 1) each centre’s technical requirements for SBRT delivery, 2) if centres were able to contour and plan patients according to protocol-derived guidelines without major deviations, and 3) the degree of variation in SBRT target definition, organ at risk (OAR) delineation, and dosimetry. The main hypotheses were that unacceptable variation would exist particularly in OAR definition across all centres, and that institutions with limited experience in SBRT would be more likely to violate per-protocol guidelines.

      Material and methods

      Development of credentialing process

      In preparation for credentialing, a radiotherapy planning guide was developed (Appendix A) that contained the necessary instructions for contouring of targets, OARs, and dosimetric objectives including metrics of high and low-dose target conformality, as well as permitted OAR point and volume maximum doses (Table 1). The planning guide, contouring guide and dose metrics devised were based on consensus by the LUSTRE steering and RTQA committees with representation of two participating centres, in addition to external validation from a non-participating site. The guide and metrics were similar to previous guidelines for prospective/cohort trials in lung SBRT from the NRG Oncology Group, Dutch, and Japanese groups [
      • Videtic G.M.M.
      • Hu C.
      • Singh A.K.
      • Chang J.Y.
      • Parker W.
      • Olivier K.R.
      • et al.
      A randomized phase 2 study comparing 2 stereotactic body radiation therapy schedules for medically inoperable patients with stage I peripheral non-small cell lung cancer: NRG Oncology RTOG 0915 (NCCTG N0927).
      ,
      • Kong F.-M.-S.
      • Ritter T.
      • Quint D.J.
      • Senan S.
      • Gaspar L.E.
      • Komaki R.U.
      • et al.
      Consideration of dose limits for organs at risk of thoracic radiotherapy: atlas for lung, proximal bronchial tree, esophagus, spinal cord, ribs, and brachial plexus.
      ,
      • Nagata Y.
      • Takayama K.
      • Matsuo Y.
      • Norihisa Y.
      • Mizowaki T.
      • Sakamoto T.
      • et al.
      Clinical outcomes of a phase I/II study of 48 Gy of stereotactic body radiotherapy in 4 fractions for primary lung cancer using a stereotactic body frame.
      ,
      • Hiraoka M.
      • Ishikura S.
      A Japan clinical oncology group trial for stereotactic body radiation therapy of non-small cell lung cancer.
      ].
      Table 1Dose/fractionation and conformality considerations for lung SBRT.
      OrganMaximum Point Dose (dose per fraction) [Gy]Critical Volume [cm3]Critical Volume Dose (dose per fraction) [Gy]
      SBRT 48 Gy in 4 Fractions
      SPINAL CANAL27 (6.75)118 (4.5)
      ESOPHAGUS30 (7.5)519 (4.75)
      BRACHIAL PLEXUS27 (6.75)
      HEART35 (8.75)1529 (7.25)
      VESSELS (SVC/IVC/AORTA)48 (12)1040 (10)
      TRACHEA40 (10)532 (8)
      PROXIMAL BRONCHIAL TREE40 (10)532 (8)
      SKIN36 (9)1033 (8.25)
      RIBS50 (12.5)540 (10)
      STOMACH28 (7)121 (5.25)
      BOTH LUNGS100013 (3.25)
      Critical Volume [%]Critical Volume

      Dose [Gy]
      BOTH LUNGS1020
      SBRT 60 Gy in 8 Fractions
      SPINALCANAL32(4)122(2.75)
      ESOPHAGUS40 (5)522 (5)
      BRACHIAL PLEXUS38 (4.75)
      HEART64 (8)1060 (7.5)
      VESSELS (SVC/IVC/AORTA)64 (8)1060 (7.5)
      TRACHEA64 (8)560 (7.5)
      PROXIMAL BRONCHIAL TREE64 (8)560 (7.5)
      SKIN45 (5.6)1040 (5)
      RIBS60 (7.5)550 (6.25)
      STOMACH40 (5)136 (4.5)
      BOTH LUNGS100018 (2.25)
      Critical Volume [%]Critical Volume

      Dose [Gy]
      BOTH LUNGS1020
      Dose Conformity Metrics
      PTV (cm3)R100R50D2 cm (%)
      DEVIATIONDEVIATIONDEVIATION
      NONEMINORNONEMINORNONEMINOR
      0–20<1.251.25–1.40<1212–14<6565–75
      20–40<1.151.15–1.25<99–11<7070–80
      >40<1.101.10–1.20<66–8<7070–80

      Credentialing process and standardized cases

      The SBRT credentialing process for each centre consisted of the following three essential elements:
      • 1.
        Successful completion of an institutional survey describing SBRT planning, delivery, image guidance, and program experience (ie number of years of experience in using SBRT for NSCLC).
      • 2.
        Completion of an end-to-end evaluation using an established Imaging and Radiation Oncology Core (IROC) phantom. Centres with previously successful IROC phantom testing were asked to provide proof of completion.
      • 3.
        Successful contouring and planning of SBRT for two standardized cases - one peripheral and one central NSCLC case, approved for use by the institutional ethics board (Fig. 1). In order to pass, both plans required meeting contours and dosimetry goals as outlined in the planning guide. Centres were required to contour using the available 4-dimensional computed tomography (4DCT) datasets in the benchmark cases, and with window/level settings as recommended within the planning guide. Centres were advised that target contours including gross tumor volume (GTV), internal target volume (ITV), and planning target volume (PTV) should be delineated on the 4D image datasets provided, while OARs were to be contoured on the primary image dataset (in this case the average-4D dataset). They were then asked to plan using the method/algorithm that was selected in the site survey. Plans were then uploaded in Digital Imaging and Communications in Medicine (DICOM) format along with the dose and structure list using a secure file transfer protoctol (FTP) (instructions provided in planning guide) to the host institution for evaluation within the MIM (Cleveland, Ohio) platform.
        Figure thumbnail gr1
        Fig. 1Standardized cases including both peripheral (left) and central (right) lung cancers.

      Evaluation of submitted plans

      Each submitted standardized plan was reviewed for both contour and plan objectives by one medical physicist and one radiation oncologist. Plans that violated either target/OAR volume delineation, and/or coverage/dosimetric constraints as outlined in the planning guide were sent back to the submitting institution for revision. As per the trial protocol, reviews were required to be completed within three to five working days to allow for quick feedback and amendments (if necessary). Upon final approval, the centre was considered to be fully credentialed and allowed to proceed with trial activation. The final approved plans were then compared across centres, using the host institution as the “de-facto” gold standard based on previously agreed upon consensus target and OAR structure volumes by the trial steering committee. These same experts who provided consensus also approved the contours and plan for the host institution “gold standard” against which other centres were benchmarked.

      Statistical analysis

      Simple descriptives were used to summarize the site survey, IROC phantom results, and percentage of cases that did not meet initial per-protocol guidelines. With respect to contouring of targets, volume ratios of submitted ITV and PTV volumes were compared individually to the host institution as well as summarized among institutions, with the group mean and standard deviation calculated. Dice coefficients, which are measures of similarity between two volumes (or the degree of overlap between two volumes, in this case each centre (B) versus the host institution (A)) were also utilized to determine concordance using the following formula; Dice = (2 × |A ∩ B|)/(|A| + |B|): a value closer to 1 would indicate a high degree of correlation between contours, a value of 0.8 or higher would be considered good concordance, 0.7–0.8 moderate concordance, and <0.7 poor concordance. With respect to OAR contouring, in addition to dice coefficients, the maximum mean distance (in mm) between contours as compared to the host institution were calculated. In order to compute the distance between contour A (host) and B (each institution) each point in A was located, along with a corresponding point on B. This was done by finding the point on B that was the closest. This exercise was repeated over all points on A. The mean distance represented the mean distance between contour A and B. This process was repeated for each institution and was also averaged out among institutions, with mean and standard deviation reported as well. A distance of within 2 mm was considered to be good concordance, 2–5 mm moderate concordance, and >5 mm poor concordance.

      Results

      Centre demographics

      A total of 17 Canadian institutions participated in the prospective credentialing process prior to trial activation. Two centres were excluded from this analysis - 1 participated only in CFRT (not SBRT) credentialing (patients from this centre were treated with SBRT if randomized at the host institution), and a second closed the study soon after activation, leaving a total of 15 accruing centres that participated. Table 2 outlines the results of the centre survey. All 15 participating institutions planned to utilize advanced planning algorithms for the trial, in addition to 4DCT based lung SBRT volume and plan generation. All centres also planned to use daily image guidance using either cone-beam computed tomography (CBCT) or tumor tracking with/without fiducial marker guidance. Of the 15 centres, 7 had previously established lung SBRT programs (>2 years of experience). Only one of the 15 centres failed the initial IROC phantom end-to-end test; initially the reasons for this were unclear, but after delivery of a 2nd phantom within a short period of time, this centre was successful.
      Table 2Results of Site Surveys.
      Centre# of ROs# SBRT pts per monthLength of SBRT ProgramPrimary Planning DatasetCT Slice ThicknessPrimary Planning MethodDose Grid (mm)AlgorithmIGRTIGRT FrequencyMotion Management
      174–51 yearavg 4DCT2–3 mmVMAT2x2x3AAAKV CBCTDailyITV, 4DCT, AC
      2300 monthsavg 4DCT2 mmVMAT2x2x2AAAKV CBCTDailyITV, 4DCT, AC
      353–45 monthsavg 4DCT2–3 mmVMAT2.5x2.5x2.5AAAKV CBCTDailyITV, 4DCT, AC
      4100 monthsavg 4DCT2–3 mmVMAT2.5x2.5x2.5AAAKV CBCTDailyITV, 4DCT, AC
      5500 monthsFB CT/avg 4DCT2–3 mmVMAT2.5x2.5x2.5AAAKV CBCTDailyITV, 4DCT, AC
      648–105 yearsFB CT/avg 4DCT2–3 mmVMAT2.5x2.5x2.5AAAKV CBCTDailyITV, 4DCT
      786–85 yearsavg 4DCT2–3 mmVMAT2.5x2.5x2.5AAAKV CBCT

      KV Ortho
      DailyITV, 4DCT, AC
      8455 yearsavg 4DCT

      exh 4DCT

      exh BH CT
      1–2 mmVMAT

      Cyberknife
      2.5x2.5x3

      1x1x1
      AAA

      MMC
      KV CBCT

      KV Ortho

      Dual KV
      DailyITV, 4DCT, AC, Gating, Voluntary BH, Fiducials
      9677 yearsavg 4DCT1–2 mmVMAT2.5x2.5x2.5AAAKV CBCT

      KV Ortho
      DailyITV, 4DCT
      10682.5 yearsavg 4DCT1–2 mmVMAT2.5x2.5x2.5ACSKV 4DCBCTDailyITV, 4DCT, AC
      11200 monthsavg 4DCT2–3 mmVMAT2.5x2.5x2.5AAAKV CBCTDailyITV, 4DCT
      124104 yearsFast FB CT1–2 mmVMAT2x2x2AAAKV CBCTDailyITV, 4DCT
      13587 yearsavg 4DCT

      gated BH CT
      2–3 mmVMAT3x3x3ACSKV/gated CBCTDailyITV, 4DCT, Gating
      142214 monthsavg 4DCT2–3 mmVMAT2.5x2.5x2.5ACSKV 4DCBCTDailyITV, 4DCT, AC
      15126 monthsFast FB CT1–2 mmVMAT2x2x2ACSKV CBCT, 4DCBCTDailyITV, 4DCT, AC
      HOST453 yearsavg 4DCT

      exh BH CT
      1–2 mmVMAT

      Cyberknife
      2.5x2.5x2.5

      1x1x1
      ACS

      MMC
      KV CBCT

      KV Ortho
      DailyITV, 4DCT, AC, BH, Fiducials
      AAA – Analytical Anisotropic Algorithm; ACS – Adaptive Convolution/Superposition; MMC – Multiplan Monte Carlo; BH – Breath-hold; FB – Free-breathing; ITV – Internal Target Volume; AC – Abdominal Compression.

      Protocol compliance with standardized cases

      A total of 30 SBRT plans (15 peripheral, 15 central) were submitted for initial credentialing. Of the 30 plans submitted, 10 (33 %) required resubmissions due to major deviations. Six of the resubmissions were for the centrally located case, and four for the peripheral case. Centres with previously established SBRT programs contributed 14 initial plans, of which 4 were resubmitted due to deviations (28.5 %), whereas new programs submitted 16 plans, of which 6 were not per protocol (37.5 %). No statistical difference was seen between rates of proportions of resubmission with respect to established versus new lung SBRT programs (chi-squared p = 0.61). Reasons for resubmission were mainly related to contouring issues such as: under-contouring of the brachial plexus, not including cartilage rings of trachea and proximal bronchial tree (PBT), skin contours contained the immobilization device, ribs not contoured within 5 cm of PTV, lung contours included the PBT and/or target, contours were not cleaned leaving many stray voxels on the submitted plans, missing slices on OARs either superiorly or inferiorly, and incorrect nomenclature/naming of structures as per the planning guide. In terms of plans/dosimetry, one submitted plan was calculated on the maximum intensity projection image; while the dosimetry in that case appeared reasonable, the resubmitted plan was computed on the correct dataset (in this case the average 4DCT).

      Final submitted plan contouring – target volume ratios

      For the peripheral case, the mean volume ratio of the ITV (± standard deviation) across all institutions was 1.11 (± 0.16), with a range of 0.87–1.51, compared to the host institution. PTV mean volume ratios were 1.13 (± 0.10), with a range of 0.95–1.38. For the central case, the mean volume ratios for ITV were 1.13 (+/-0.20, range 0.89–1.55), and PTV were 1.05 (± 0.12, range 0.90–1.27). A representative overlay of GTVs and PTVs across all centres is demonstrated in Fig. 2a.
      Figure thumbnail gr2a
      Fig. 2aGTV (above) and PTV (below) overlays compared to the host institution (turquoise contour).

      Final submitted plan contouring – min/max distances and dice coefficients

      Table 3 presents the min/max distances and dice coefficients for both targets and OARs for both the peripheral and central cases. In general, good concordance was seen for targets (ITV, PTV), spinal canal, and trachea across all cases. Moderate concordance was consistently observed in PBT and rib contours for both cases. Finally, poor concordance was seen in brachial plexus and vessel contours across most institutions. Mixed concordance occurred for heart volumes (high dice coefficient, moderate max/min distance) likely due to it being a relatively large OAR, and esophageal volumes (moderate dice coefficient, high max/min difference), likely due to the narrow organ structure. Fig. 2b shows examples of contouring difference among centres compared to the host institution with representative examples of high and low concordant OAR structures.
      Table 3Contouring Variations on Final Submitted Test Plans.
      Figure thumbnail gr2b
      Fig. 2bOverlays of contours with moderate to poor concordance compared to host institution (in yellow) – from top left going clockwise: Ribs, Proximal Bronchial tree, Vessels, and Brachial Plexus. (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)

      Final submitted plan evaluation

      In terms of target coverage, there was observed variation with respect to both high dose and low dose heterogeneity in both cases as seen with the dose at 2 cm from the PTV (D2cm) and the maximum dose in the PTV. This was evident in both cases, with some institutions prescribing up to a maximum point dose of 80 Gy within the target (in 8 fractions) in the central case, and up to 70 Gy (in 4 fractions) for the peripheral case. Fig. 3 demonstrates examples of both high and low dose isodoses across all institutions in the context of the peripheral case.
      Figure thumbnail gr3
      Fig. 3Representative DVHs and variations in dosimetry across all centres for both standardized cases (substantial variations highlighted in yellow).
      While OAR protocol constraints were met as per final submitted plans, some variation was observed. Representative dose-volume histograms (DVHs) are described in Fig. 3. For the peripheral case, DVHs were predictably less variable, apart from PBT and vessels, however doses to these structures were overall still quite low given the location of the target. For the central cases, DVHs showed some more variability particularly with the trachea and vessels, which were closer to the target.

      Discussion

      The credentialing process in preparation for the OCOG-LUSTRE trial was prospectively designed to evaluate institutional practices with respect to lung SBRT, and to provide a platform for newer centres to participate. The results of this credentialing exercise clearly showed that unacceptable deviations existed with respect to contouring, plan delivery and differences in terms of plan objectives and these were not necessarily commensurate on centre experience. While all plans eventually passed as per protocol, other variations, while appropriate for this exercise, were observed. Whether such variations are clinically significant are uncertain but need to be studied further. In particular, contouring and dosimetry related to key central and apical OAR structures were most variable (ie brachial plexus, vessels, trachea, and PBT).
      It should be noted that the benchmarking process of the European Organization for Research and Treatment of Cancer (EORTC) LUNGTECH trial of central SBRT observed similar issues with the majority of plans failing initially due to contouring variations (in the current study all requirements for resubmission were due to contouring errors) [
      • Lambrecht M.
      • Clementel E.
      • Sonke J.-J.
      • Nestle U.
      • Adebahr S.
      • Guckenberger M.
      • et al.
      Radiotherapy quality assurance of SBRT for patients with centrally located lung tumours within the multicentre phase II EORTC Lungtech trial: Benchmark case results.
      ]. Other radiotherapy trials involving benchmark or “dry-run” cases have also demonstrated this trend, for SBRT and other techniques [
      • Kron T.
      • Chesson B.
      • Hardcastle N.
      • Crain M.
      • Clements N.
      • Burns M.
      • et al.
      Credentialing of radiotherapy centres in Australasia for TROG 09.02 (Chisel), a Phase III clinical trial on stereotactic ablative body radiotherapy of early stage lung cancer.
      ,
      • Matsuo Y.
      • Takayama K.
      • Nagata Y.
      • Kunieda E.
      • Tateoka K.
      • Ishizuka N.
      • et al.
      Interinstitutional variations in planning for stereotactic body radiation therapy for lung cancer.
      ,
      • Trada Y.
      • Kneebone A.
      • Paneghel A.
      • Pearse M.
      • Sidhom M.
      • Tang C.
      • et al.
      Optimizing radiation therapy quality assurance in clinical trials: A TROG 08.03 RAVES substudy.
      ,
      • Boustani J.
      • Rivin Del Campo E.
      • Blanc J.
      • Peiffert D.
      • Benezery K.
      • Pereira R.
      • et al.
      Quality assurance of dose-escalated radiation therapy in a randomized trial for locally advanced oesophageal cancer.
      ]. Furthermore, SBRT peer review studies in lung and other sites have also demonstrated high revision rates, mainly due to OAR contouring as the main reason for plan revision/rejection [
      • Lo A.C.
      • Liu M.
      • Chan E.
      • Lund C.
      • Truong P.T.
      • Loewen S.
      • et al.
      The impact of peer review of volume delineation in stereotactic body radiation therapy planning for primary lung cancer: a multicenter quality assurance study.
      ,
      • Matuszak M.M.
      • Hadley S.W.
      • Feng M.
      • Hayman J.A.
      • Brock K.K.
      • Burger P.
      • et al.
      Enhancing safety and quality through preplanning peer review for patients undergoing stereotactic body radiation therapy.
      ]. A crowd-sourced lung planning study recently exemplified similar discrepancies in central OAR contours, including in that case both PBT and esophagus [
      • Moghanaki D.
      • Slotman B.
      • Swaminath A.
      • Nelms B.
      • Wang B.
      Assessing the variability and quality of lung stereotactic radiation therapy treatment plans using a web-based crowdsourcing platform.
      ].
      Poor concordance of structures such as vessels and brachial plexus were possibly due to a few factors. First, the farther distance of the brachial plexus to the targets and lack of experience in routine contouring of this organ probably led to inaccurate delineation. Furthermore, the brachial plexus contours were largely based on vascular surrogates, but some centres elected to include the actual plexus bundle if visible to them, which increased the volumes of the plexus overall. This included more lateral vessels which affected the overall volume, but did not impact dose to the plexus, due to the location of the target in both cases. Second, with respect to the vessel contours, many institutions included pulmonary artery and vein, although this was not required per protocol. Additionally, some centres elected to contour both ipsilateral and contralateral vessels, although it was not necessary in the planning guide to do so. This is why, especially for vessel contours, there was a discrepancy between poor concordance and non-violation of per-protocol guidelines. There was flexibility allowed in the vessel structure based on institutional practices that, while aligning with general RTQA guidelines, led to variability in contour definition compared to the host institution.
      Reasons for OARs with moderate concordance such as PBT and to a lesser extent esophagus were more likely variation in interpretation of the radiological imaging provided. Such differences could have potential implications given concern regarding radiation related toxicity especially for central and “ultra-central” NSCLC, when these organs are close to high dose gradients [
      • Rim C.H.
      • Kim Y.
      • Kim C.Y.
      • Yoon W.S.
      • Yang D.S.
      Is stereotactic body radiotherapy for ultra-central lung tumor a feasible option? A systemic review and meta-analysis.
      ,
      • Tekatli H.
      • Haasbeek N.
      • Dahele M.
      • De Haan P.
      • Verbakel W.
      • Bongers E.
      • et al.
      Outcomes of hypofractionated high-dose radiotherapy in poor-risk patients with “ultracentral” non-small cell lung cancer.
      ,
      • Chen H.
      • Laba J.M.
      • Zayed S.
      • Boldt R.G.
      • Palma D.A.
      • Louie A.V.
      Safety and effectiveness of stereotactic ablative radiotherapy for ultra-central lung lesions: A systematic review.
      ]. With emerging reports of even modest biologically equivalent SBRT doses potentially increasing risk of bronchial toxicity it is clear that ensuring avoidance of uninvolved bronchus and other central OARs with careful delineation is paramount to ensuring safe SBRT for these high-risk patients both on trial and in clinical practice [
      • Lindberg K.
      • Grozman V.
      • Karlsson K.
      • Lindberg S.
      • Lax I.
      • Wersäll P.
      • et al.
      The HILUS-Trial-a prospective nordic multicenter phase 2 study of ultracentral lung tumors treated with stereotactic body radiotherapy.
      ,
      • Lodeweges J.E.
      • van Rossum P.S.N.
      • Bartels M.M.T.J.
      • van Lindert A.S.R.
      • Pomp J.
      • Peters M.
      • et al.
      Ultra-central lung tumors: safety and efficacy of protracted stereotactic body radiotherapy.
      ].
      Interestingly, no institution failed due to under/overcoverage of target volumes, this is in contradistinction to the LUNGTECH and trans-Tasman Radiation Oncology Group (TROG) CHISEL credentialing experience, where some plans did not pass due to PTV underdosing [
      • Lambrecht M.
      • Clementel E.
      • Sonke J.-J.
      • Nestle U.
      • Adebahr S.
      • Guckenberger M.
      • et al.
      Radiotherapy quality assurance of SBRT for patients with centrally located lung tumours within the multicentre phase II EORTC Lungtech trial: Benchmark case results.
      ,
      • Kron T.
      • Chesson B.
      • Hardcastle N.
      • Crain M.
      • Clements N.
      • Burns M.
      • et al.
      Credentialing of radiotherapy centres in Australasia for TROG 09.02 (Chisel), a Phase III clinical trial on stereotactic ablative body radiotherapy of early stage lung cancer.
      ]. These differences could be due to the priority of more stringent OAR constraints in these trials resulting in lower target coverage in submitted plans. With respect to PTV coverage in particular, there was considerable variability in the “heterogeneity” of dose distributions within the target. While all institutions passed minimum requirements for PTV coverage, the maximum dose within the target was quite variable. Perhaps the choice of cases permitted this spectrum of heterogeneity, however it was clear that some institutions preferred a more uniform dose distribution across the target, especially when in proximity to a critical OAR such as the trachea, or vessels. Whether this was by design or intentional based on target location is somewhat unclear based on this study. Furthermore, the impact of more heterogeneity within the target at a clinical level is less clear. Additionally, the impact of intrafraction motion on both target coverage and high dose gradient spillage into potential adjacent OARs is still the subject of continued investigation [
      • Jasper K.
      • Liu B.
      • Olson R.
      • Matthews Q.
      Evidence-Based Planning Target Volume Margin Reduction for Modern Lung Stereotactic Ablative Radiation Therapy Using Deformable Registration.
      ]. While these issues are beyond the scope of this analysis, future guidance on the effect of the planned PTV dose to what is delivered will be important for future dose response studies, especially for central or ultra-central NSCLC.
      At a larger level, the results of this credentialing study emphasize the need for continued, robust RTQA measures particularly when introducing new technologies in radiotherapy. A recent systematic review identified that within the past 25 years, while two-thirds of RCTs involving radiation did have some measure of RTQA, less than half had reported initial credentialing, and there was no increase in utilization of RTQA in studies that utilized more advanced techniques (intensity-modulated RT/volumetric modulated RT/SBRT) compared to those that did not [
      • Corrigan K.L.
      • Kry S.
      • Howell R.M.
      • Kouzy R.
      • Jaoude J.A.
      • Patel R.R.
      • et al.
      The radiotherapy quality assurance gap among phase III cancer clinical trials.
      ]. Even with lung SBRT, which is more widely adopted in general, there were still important variations observed that the credentialing process helped in terms of streamlining practices. This should hopefully ensure that accrued patient plans on trial are more compliant with the protocol and will improve the quality of radiotherapy delivered in the main study. As part of the trial, we have mandated real-time review for the first 4 patients treated with SBRT. Any major deviations will be recorded and the treatment centre requested to make changes to the plans before treatment is started. Centres will continue to submit plans for real-time review until 4 SBRT cases are submitted consecutively without major deviations. Following this process, all treatment plans will be sent for final review. This prospective real-time review process is similar in spirit to other lung and oligometastatic SBRT trials that have been conducted, based on the analysis by the Global Quality Assurance of Radiation Therapy Clinical Trials Harmonisation Group [
      • Clark C.H.
      • Hurkmans C.W.
      • Kry S.F.
      • Global Quality Assurance of Radiation Therapy Clinical Trials Harmonisation Group
      The role of dosimetry audit in lung SBRT multi-centre clinical trials.
      ]. Our next steps are to analyse the real-time and final review aspects of the study itself; the hypothesis is that because of the initial effort involved in credentialing, the downstream effect on plan quality and rates of major deviations will be relatively low.
      In conclusion, the prospective credentialing experience in the OCOG-LUSTRE trial was an important step to understand differences and help develop an approach to align practices for high quality lung SBRT. Such an approach can be utilized for other studies that rely on high precision radiotherapy techniques, and to ensure that the implications on trial quality and outcomes are as optimal as possible.

      Funding

      Canadian Cancer Society Research Institute Impact Grant # 701596 .

      Data Sharing

      Research data are not available at this time.

      Declaration of Competing Interest

      The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

      Appendix A. Supplementary data

      The following are the Supplementary data to this article:

      References

        • Weber D.C.
        • Poortmans P.M.P.
        • Hurkmans C.W.
        • Aird E.
        • Gulyban A.
        • Fairchild A.
        Quality assurance for prospective EORTC radiation oncology trials: the challenges of advanced technology in a multicenter international setting.
        Radiother Oncol J Eur Soc Ther Radiol Oncol. 2011; 100: 150-156https://doi.org/10.1016/j.radonc.2011.05.073
        • Jaccard M.
        • Zilli T.
        • Dubouloz A.
        • Escude L.
        • Jorcano S.
        • Linthout N.
        • et al.
        Urethra-sparing stereotactic body radiation therapy for prostate cancer: quality assurance of a randomized phase 2 trial.
        Int J Radiat Oncol Biol Phys. 2020; 108: 1047-1054https://doi.org/10.1016/j.ijrobp.2020.06.002
        • Brade A.M.
        • Wenz F.
        • Koppe F.
        • Lievens Y.
        • San Antonio B.
        • Iscoe N.A.
        • et al.
        Radiation Therapy Quality Assurance (RTQA) of concurrent chemoradiation therapy for locally advanced non-small cell lung cancer in the PROCLAIM phase 3 trial.
        Int J Radiat Oncol Biol Phys. 2018; 101: 927-934https://doi.org/10.1016/j.ijrobp.2018.04.015
        • Fairchild A.
        • Straube W.
        • Laurie F.
        • Followill D.
        Does quality of radiation therapy predict outcomes of multicenter cooperative group trials? A literature review.
        Int J Radiat Oncol Biol Phys. 2013; 87: 246-260https://doi.org/10.1016/j.ijrobp.2013.03.036
        • Peters L.J.
        • O’Sullivan B.
        • Giralt J.
        • Fitzgerald T.J.
        • Trotti A.
        • Bernier J.
        • et al.
        Critical impact of radiotherapy protocol compliance and quality in the treatment of advanced head and neck cancer: results from TROG 02.02.
        J Clin Oncol Off J Am Soc Clin Oncol. 2010; 28: 2996-3001https://doi.org/10.1200/JCO.2009.27.4498
        • Corrigan K.L.
        • Kry S.
        • Howell R.M.
        • Kouzy R.
        • Jaoude J.A.
        • Patel R.R.
        • et al.
        The radiotherapy quality assurance gap among phase III cancer clinical trials.
        Radiother Oncol J Eur Soc Ther Radiol Oncol. 2021; 166: 51-57https://doi.org/10.1016/j.radonc.2021.11.018
        • Swaminath A.
        • Wierzbicki M.
        • Parpia S.
        • Wright J.R.
        • Tsakiridis T.K.
        • Okawara G.S.
        • et al.
        Canadian Phase III randomized trial of stereotactic body radiotherapy versus conventionally hypofractionated radiotherapy for stage I, medically inoperable non-small-cell lung cancer - rationale and protocol design for the Ontario Clinical Oncology Group (OCOG)-LUSTRE Trial.
        Clin Lung Cancer. 2017; 18: 250-254https://doi.org/10.1016/j.cllc.2016.08.002
        • Videtic G.M.M.
        • Hu C.
        • Singh A.K.
        • Chang J.Y.
        • Parker W.
        • Olivier K.R.
        • et al.
        A randomized phase 2 study comparing 2 stereotactic body radiation therapy schedules for medically inoperable patients with stage I peripheral non-small cell lung cancer: NRG Oncology RTOG 0915 (NCCTG N0927).
        Int J Radiat Oncol Biol Phys. 2015; 93: 757-764https://doi.org/10.1016/j.ijrobp.2015.07.2260
        • Kong F.-M.-S.
        • Ritter T.
        • Quint D.J.
        • Senan S.
        • Gaspar L.E.
        • Komaki R.U.
        • et al.
        Consideration of dose limits for organs at risk of thoracic radiotherapy: atlas for lung, proximal bronchial tree, esophagus, spinal cord, ribs, and brachial plexus.
        Int J Radiat Oncol Biol Phys. 2011; 81: 1442-1457https://doi.org/10.1016/j.ijrobp.2010.07.1977
        • Nagata Y.
        • Takayama K.
        • Matsuo Y.
        • Norihisa Y.
        • Mizowaki T.
        • Sakamoto T.
        • et al.
        Clinical outcomes of a phase I/II study of 48 Gy of stereotactic body radiotherapy in 4 fractions for primary lung cancer using a stereotactic body frame.
        Int J Radiat Oncol Biol Phys. 2005; 63: 1427-1431https://doi.org/10.1016/j.ijrobp.2005.05.034
        • Hiraoka M.
        • Ishikura S.
        A Japan clinical oncology group trial for stereotactic body radiation therapy of non-small cell lung cancer.
        J Thorac Oncol Off Publ Int Assoc Study Lung Cancer. 2007; 2: S115-S117https://doi.org/10.1097/JTO.0b013e318074de1b
        • Lambrecht M.
        • Clementel E.
        • Sonke J.-J.
        • Nestle U.
        • Adebahr S.
        • Guckenberger M.
        • et al.
        Radiotherapy quality assurance of SBRT for patients with centrally located lung tumours within the multicentre phase II EORTC Lungtech trial: Benchmark case results.
        Radiother Oncol J Eur Soc Ther Radiol Oncol. 2019; 132: 63-69https://doi.org/10.1016/j.radonc.2018.10.025
        • Kron T.
        • Chesson B.
        • Hardcastle N.
        • Crain M.
        • Clements N.
        • Burns M.
        • et al.
        Credentialing of radiotherapy centres in Australasia for TROG 09.02 (Chisel), a Phase III clinical trial on stereotactic ablative body radiotherapy of early stage lung cancer.
        Br J Radiol. 2018; 91: 20170737https://doi.org/10.1259/bjr.20170737
        • Matsuo Y.
        • Takayama K.
        • Nagata Y.
        • Kunieda E.
        • Tateoka K.
        • Ishizuka N.
        • et al.
        Interinstitutional variations in planning for stereotactic body radiation therapy for lung cancer.
        Int J Radiat Oncol Biol Phys. 2007; 68: 416-425https://doi.org/10.1016/j.ijrobp.2006.12.012
        • Trada Y.
        • Kneebone A.
        • Paneghel A.
        • Pearse M.
        • Sidhom M.
        • Tang C.
        • et al.
        Optimizing radiation therapy quality assurance in clinical trials: A TROG 08.03 RAVES substudy.
        Int J Radiat Oncol Biol Phys. 2015; 93: 1045-1051https://doi.org/10.1016/j.ijrobp.2015.08.029
        • Boustani J.
        • Rivin Del Campo E.
        • Blanc J.
        • Peiffert D.
        • Benezery K.
        • Pereira R.
        • et al.
        Quality assurance of dose-escalated radiation therapy in a randomized trial for locally advanced oesophageal cancer.
        Int J Radiat Oncol Biol Phys. 2019; 105: 329-337https://doi.org/10.1016/j.ijrobp.2019.06.2542
        • Lo A.C.
        • Liu M.
        • Chan E.
        • Lund C.
        • Truong P.T.
        • Loewen S.
        • et al.
        The impact of peer review of volume delineation in stereotactic body radiation therapy planning for primary lung cancer: a multicenter quality assurance study.
        J Thorac Oncol Off Publ Int Assoc Study Lung Cancer. 2014; 9: 527-533https://doi.org/10.1097/JTO.0000000000000119
        • Matuszak M.M.
        • Hadley S.W.
        • Feng M.
        • Hayman J.A.
        • Brock K.K.
        • Burger P.
        • et al.
        Enhancing safety and quality through preplanning peer review for patients undergoing stereotactic body radiation therapy.
        Pract Radiat Oncol. 2016; 6: e39-e46https://doi.org/10.1016/j.prro.2015.09.009
        • Moghanaki D.
        • Slotman B.
        • Swaminath A.
        • Nelms B.
        • Wang B.
        Assessing the variability and quality of lung stereotactic radiation therapy treatment plans using a web-based crowdsourcing platform.
        Pract Radiat Oncol. 2020; 10: e118-e127https://doi.org/10.1016/j.prro.2019.12.004
        • Rim C.H.
        • Kim Y.
        • Kim C.Y.
        • Yoon W.S.
        • Yang D.S.
        Is stereotactic body radiotherapy for ultra-central lung tumor a feasible option? A systemic review and meta-analysis.
        Int J Radiat Biol. 2019; 95: 329-337https://doi.org/10.1080/09553002.2019.1552375
        • Tekatli H.
        • Haasbeek N.
        • Dahele M.
        • De Haan P.
        • Verbakel W.
        • Bongers E.
        • et al.
        Outcomes of hypofractionated high-dose radiotherapy in poor-risk patients with “ultracentral” non-small cell lung cancer.
        J Thorac Oncol Off Publ Int Assoc Study Lung Cancer. 2016; 11: 1081-1089https://doi.org/10.1016/j.jtho.2016.03.008
        • Chen H.
        • Laba J.M.
        • Zayed S.
        • Boldt R.G.
        • Palma D.A.
        • Louie A.V.
        Safety and effectiveness of stereotactic ablative radiotherapy for ultra-central lung lesions: A systematic review.
        J Thorac Oncol Off Publ Int Assoc Study Lung Cancer. 2019; 14: 1332-1342https://doi.org/10.1016/j.jtho.2019.04.018
        • Lindberg K.
        • Grozman V.
        • Karlsson K.
        • Lindberg S.
        • Lax I.
        • Wersäll P.
        • et al.
        The HILUS-Trial-a prospective nordic multicenter phase 2 study of ultracentral lung tumors treated with stereotactic body radiotherapy.
        J Thorac Oncol Off Publ Int Assoc Study Lung Cancer. 2021; 16: 1200-1210https://doi.org/10.1016/j.jtho.2021.03.019
        • Lodeweges J.E.
        • van Rossum P.S.N.
        • Bartels M.M.T.J.
        • van Lindert A.S.R.
        • Pomp J.
        • Peters M.
        • et al.
        Ultra-central lung tumors: safety and efficacy of protracted stereotactic body radiotherapy.
        Acta Oncol Stockh Swed. 2021; 60: 1061-1068https://doi.org/10.1080/0284186X.2021.1942545
        • Jasper K.
        • Liu B.
        • Olson R.
        • Matthews Q.
        Evidence-Based Planning Target Volume Margin Reduction for Modern Lung Stereotactic Ablative Radiation Therapy Using Deformable Registration.
        Adv Radiat Oncol. 2021; 6: 100750https://doi.org/10.1016/j.adro.2021.100750
        • Clark C.H.
        • Hurkmans C.W.
        • Kry S.F.
        • Global Quality Assurance of Radiation Therapy Clinical Trials Harmonisation Group
        The role of dosimetry audit in lung SBRT multi-centre clinical trials.
        Phys Med. 2017; 44: 171-176