Article PDF first page preview

Article PDF first page preview

Introduction: Visual Analysis of Swallowing Efficiency and Safety (VASES) and Dynamic Imaging Grade of Swallowing Toxicity for Flexible Endoscopic Evaluation of Swallowing (DIGEST-FEES) are two complimentary methods for assessing swallowing during FEES. Whereas VASES is intended to facilitate trial-level ratings of pharyngeal residue, penetration, and aspiration, DIGEST-FEES is intended to facilitate protocol-level impairment grades of swallowing safety and efficiency. The aim of this study was to assess the validity of using VASES to derive DIGEST-FEES impairment grades. Methods: DIGEST-FEES grades were blindly analyzed from 50 FEES – first using the original DIGEST-FEES grading method (n = 50) and then again using a VASES-derived DIGEST-FEES grading method (n = 50). Weighted Kappa (κw) and absolute agreement (%) were used to assess the relationship between the original DIGEST-FEES grades and VASES-derived DIGEST-FEES grades. Spearman’s correlations assessed the relationship between VASES-derived DIGEST-FEES grades with measures of construct validity. Results: Substantial agreement (κw = 0.76–0.83) was observed between the original and VASES-derived grading methods, with 60–62% of all DIGEST-FEES grades matching exactly, and 92–100% of DIGEST-FEES grades within one grade of each other. Furthermore, the strength of the relationships between VASES-derived DIGEST-FEES grades and measures of construct validity (r = 0.34–0.78) were similar to the strength of the relationships between original DIGEST-FEES grades and the same measures of construct validity (r = 0.34–0.83). Conclusion: Findings from this study demonstrate substantial agreement between original and VASES-derived DIGEST-FEES grades. Using VASES to derive DIGEST-FEES also appears to maintain the same level of construct validity established with the original DIGEST-FEES. Therefore, clinicians and researchers may consider using VASES to increase the transparency and standardization of DIGEST-FEES ratings. Future research should seek to replicate these findings and explore the simultaneous use of VASES and DIGEST-FEES in a greater sampling of raters and across other patient populations.

Flexible endoscopic evaluation of swallowing (FEES) is an instrumental procedure used to aid in the comprehensive assessment of functional (bolus-related) swallowing outcomes, including impairments in swallowing efficiency (pharyngeal residue) and safety (penetration and aspiration) [1]. Given the widespread use of FEES in clinical and research practices [2, 3], it is important to standardize how functional swallowing outcomes are interpreted and reported during FEES. Two complimentary methods used to guide standardized assessment of functional swallowing outcomes during FEES include the Dynamic Imaging Grade of Swallowing Toxicity for FEES (DIGEST-FEES) [4] and the Visual Analysis of Swallowing Efficiency and Safety (VASES) [5, 6].

DIGEST-FEES is a rating scale originally adapted from its videofluoroscopic swallow study counterpart – DIGEST [7, 8]. DIGEST-FEES uses trial-level ratings of pharyngeal residue, penetration, and aspiration to ultimately derive protocol-level impairment grades in swallowing safety (DIGESTSafety), swallowing efficiency (DIGESTEfficiency), and overall pharyngeal swallow dysfunction (DIGESTTotal). One reason DIGEST-FEES may be of interest for clinicians and researchers is because it facilitates a standardized method to characterize the severity of a person’s swallow impairment and dysphagia diagnosis.

Alternatively, VASES is an interpretation framework which establishes (1) explicit boundaries for six anatomic structures and four swallowing-related temporal phases and (2) distinct methods to rate swallowing safety and efficiency. The intent of VASES is to facilitate standardized, transparent, and reliable trial-level ratings of pharyngeal residue, penetration, and aspiration. Importantly, preliminary normative data have recently been published for VASES which may further aid clinicians in accurate diagnosis of swallowing impairments [9].

Using trial-level ratings with VASES to simultaneously derive protocol-level grades from DIGEST-FEES has important clinical and research implications. First, it has the potential to increase workflow efficiency for people using both DIGEST-FEES and VASES by eliminating the need to analyze each swallow twice – once with VASES and then again with DIGEST-FEES. Second, it has the potential for people who use DIGEST-FEES to further enhance the transparency and reliability of DIGEST-FEES grades through the use of VASES explicit interpretation framework. Third, it has the potential for people who use VASES to obtain protocol-level grades to summarize a person’s swallow dysfunction. However, methodological differences exist between DIGEST-FEES and VASES which may impact the validity of using VASES to derive DIGEST-FEES grades. For example, DIGEST-FEES residue ratings are made by estimating bolus clearance through the pharynx after the initial swallow attempt, whereas VASES residue ratings are made by estimating the amount of residue filling or covering a structure at the offset after the swallow phase.

Given this, the primary aims of this study were to (1) characterize the agreement between original DIGEST-FEES grades and VASES-derived DIGEST-FEES grades and (2) characterize the relationship between VASES-derived DIGEST-FEES grades with measures of construct validity. We hypothesized that substantial agreement would be present between original DIGEST-FEES grades and VASES-derived DIGEST-FEES grades. We also hypothesized that the strength of the relationships between VASES-derived DIGEST-FEES grades and measures of construct validity would be similarly strong as the relationships observed between original DIGEST-FEES grades and the same measures of construct validity.

This was a prospective, within-subject study design comparing DIGEST-FEES grades assigned using the originally published method to grades assigned using VASES. The inclusion criterion for study participation was having been a rater in the original DIGEST-FEES publication [4]. The exclusion criterion was having knowledge of the VASES rating framework prior to DIGEST-FEES publication. The purpose of these inclusion-exclusion criteria was to prevent including raters with any direct or indirect exposure to the VASES framework which could potentially bias DIGEST-FEES results. As a result, two of the three raters from the original DIGEST-FEES publication were included in the present study. The two raters were PhD-level speech-language pathologists, one of whom had 12 years of experience performing and interpreting FEES, and the other of whom had 20 years of experience performing and interpreting FEES.

Each of the two raters had previously used DIGEST-FEES to rate the 100 FEES included in the original DIGEST-FEES publication [4] – prior to the development of VASES. Original DIGEST-FEES grades were made by assigning trial-level ratings of the Penetration-Aspiration Scale (PAS) [10] and trial-level ratings of pharyngeal residue for each FEES video. Pharyngeal residue ratings were made using a 4-point categorical scale: <10%, 10–33%, 34–66%, and >66%.

Two years after completing the original DIGEST-FEES ratings, the raters completed 5 h of VASES training led by the first author. The first author is a speech-language pathologist with board certification in swallowing and swallowing disorders and is one of the original developers of the VASES framework. Upon completion of VASES training, the last author randomly assigned 25 FEES videos to each of the two raters. A total of 50 unique FEES were assigned which represented a range of swallow impairment levels, including normal to mild impairment, moderate impairment, and severe to profound impairment. Videos were selected from FEES previously analyzed by the two raters from the original DIGEST-FEES publication. Raters were blinded to their original DIGEST-FEES ratings from 2 years prior. FEES exams utilized a standardized swallowing protocol of two trials of each of 5 mL, 10 mL, and self-selected volume cup sips of blue-dyed thin liquid; a spoonful of blue-dyed applesauce; and a self-selected volume bite of graham cracker.

Raters analyzed each FEES using VASES by first viewing each video clip in real time and then using frame-by-frame analysis as needed. VASES outcome measures included trial-level ratings of PAS and percentage-based residue ratings for the oropharynx, hypopharynx, epiglottis, laryngeal vestibule, vocal folds, and subglottis. VASES-derived DIGEST-FEES grades were then established for each FEES exam using these new VASES ratings.

Deriving “Pharyngeal Residue” Ratings

To establish VASES-derived DIGEST-FEES grades, it was first necessary to develop one singular measure of “pharyngeal residue” (as is done with DIGEST-FEES) from VASES’ six residue ratings. Combinations of VASES residue ratings were developed a priori, which included (from most parsimonious to least parsimonious):

  • 1.

    Maximum rating between the oropharynx and hypopharynx

  • 2.

    Average rating between the oropharynx and hypopharynx

  • 3.

    Weighted ratings of 33% for the oropharynx, 33% for the hypopharynx, and 33% for the maximum rating across the epiglottis, laryngeal vestibule, vocal folds, and subglottis

  • 4.

    Maximum rating across the oropharynx, hypopharynx, epiglottis, laryngeal vestibule, vocal folds, and subglottis

  • 5.

    Average rating across the oropharynx, hypopharynx, epiglottis, laryngeal vestibule, vocal folds, and subglottis

  • 6.

    Weighted ratings of 25% for the oropharynx, 25% for the hypopharynx, 12.5% for the epiglottis, 12.5% for the laryngeal vestibule, 12.5% for the vocal folds, and 12.5% for the subglottis.

Spearman’s rank correlation coefficients (ρ) were used to assess the relationship between each combination outlined above with the pharyngeal residue ratings made using the original DIGEST-FEES method. A correlation was sufficiently stronger than another if its coefficient was higher than the upper boundary of the 95% confidence interval (CI) of the contrasting correlation. The strongest and most parsimonious correlation was used as the measure of pharyngeal residue for DIGEST-FEES. Notably, only one-third of the total dataset was randomly selected for this preliminary analysis to avoid overfitting data for the primary aims.

Statistical Analyses and Reliability Assessments

All data were analyzed using R version 4.2.3 [11]. Each rater blindly re-rated 20% of their assigned FEES to assess intra-rater reliability. The first author rated 20% of the FEES videos to assess inter-rater reliability. Intra- and inter-rater reliability were statistically analyzed using weighted Kappa and absolute agreement. To remain consistent with interpretation from the original DIGEST-FEES publication, weighted Kappa was interpreted as “no agreement” when κw ≤ 0, “none-to-slight” agreement when κw = 0.01–0.20, “fair agreement” when κw = 0.21–0.40, “moderate agreement” when κw = 0.41–0.60, “substantial agreement” when κw = 0.61–0.80, and “almost perfect” agreement when κw = 0.81–1.00 [4].

Aim 1: Characterizing the Concurrent Validity of VASES-Derived DIGEST-FEES

Concurrent validity was assessed by comparing the raters’ original and VASES-derived DIGEST-FEES grades. Concurrent validity between the DIGEST-FEES grades was analyzed using weighted Kappa (κw) and absolute agreement. To be consistent with the original DIGEST-FEES publication, weighted Kappas were assessed using quadratic weights and were interpreted as previously outlined. Additionally, nonparametric Wilcoxon signed rank tests were used to determine if significant differences were present between original and VASES-derived DIGEST-FEES grades, with an alpha of p < 0.05 set as the level of statistical significance.

Aim 2: Characterizing the Construct Validity of VASES-Derived DIGEST-FEES

Construct validity was assessed by comparing VASES-derived DIGEST-FEES grades to the same measures of construct validity from the original DIGEST-FEES publication: the MD Anderson Dysphagia Inventory (MDADI) total, functional, physical, and emotional scores; Functional Oral Intake Scale (FOIS) [12]; Secretion Severity Scale (SSS) [13]; Yale Pharyngeal Residue Severity Rating Scale for the Valleculae (Yale Valleculae) and Piriforms (Yale Piriforms) [14]. Spearman’s correlation coefficients were used to assess the relationship between VASES-derived DIGEST-FEES grades and the measures of construct validity. Spearman’s correlation coefficients were also used to assess the relationship between the raters’ original VASES-derived DIGEST-FEES grades and the same measures of construct validity. VASES-derived DIGEST-FEES grades were considered significantly different from the original DIGEST-FEES grades if the original DIGEST-FEES correlation coefficient was above or below the 95% CI of the VASES-derived DIGEST-FEES grades.

A total of 50 FEES were included in this study, yielding an analysis of 544 swallowing trials. Of the 50 FEES included in this study, 6 (12%) had no impairment in swallow function, 16 (32%) had mild impairment in swallow function, 16 (32%) had moderate impairment in swallow function, 11 (22%) had severe impairment in swallow function, and 1 (2%) had profound impairment in swallow function. Demographic data are outlined in Table 1.

Table 1.

Patient characteristics (N = 50)

Age M = 65.2 years 
 (range: 42–84; SD = 11.2) 
Sex 
 Male 35 
 Female 15 
Primary site 
 Oropharynx 28 
 Oral cavity 10 
 Nasopharynx 
 Larynx/hypopharynx 
 Other 
T stage 
 Tx, 0–2 41 
 T3–4 
N stage 
 N0–1 26 
 N2–3 24 
Treatment 
 Surgery only 
 Nonsurgical 27 
 Both 19 
Timing of exam 
 Pretreatment 10 
 Posttreatment 40 
MDADI M = 73.1 
 (range: 30–100; SD = 18.2) 
FOIS scores 
 1 
 2 
 3 
 4 
 5 15 
 6 10 
 7 19 
Secretion Severity scores 
 0 18 
 1 20 
 2 11 
 3 
Yale Pharyngeal Residue scores 
 Valleculae  
  1 
  2 
  3 
  4 14 
  5 22 
 Piriforms  
  1 
  2 20 
  3 10 
  4 
  5 
Age M = 65.2 years 
 (range: 42–84; SD = 11.2) 
Sex 
 Male 35 
 Female 15 
Primary site 
 Oropharynx 28 
 Oral cavity 10 
 Nasopharynx 
 Larynx/hypopharynx 
 Other 
T stage 
 Tx, 0–2 41 
 T3–4 
N stage 
 N0–1 26 
 N2–3 24 
Treatment 
 Surgery only 
 Nonsurgical 27 
 Both 19 
Timing of exam 
 Pretreatment 10 
 Posttreatment 40 
MDADI M = 73.1 
 (range: 30–100; SD = 18.2) 
FOIS scores 
 1 
 2 
 3 
 4 
 5 15 
 6 10 
 7 19 
Secretion Severity scores 
 0 18 
 1 20 
 2 11 
 3 
Yale Pharyngeal Residue scores 
 Valleculae  
  1 
  2 
  3 
  4 14 
  5 22 
 Piriforms  
  1 
  2 20 
  3 10 
  4 
  5 

Number in cells reflects the number of participants, unless otherwise specified.

MDADI, MD Anderson Dysphagia Inventory; FOIS, Functional Oral Intake Scale.

Intra-rater reliability for VASES-derived DIGEST grades was “almost perfect” for DIGESTSafetyw = 0.914; absolute agreement = 88.9%), DIGESTEfficiencyw = 0.962; absolute agreement = 88.9%), and DIGESTTotalw = 0.903; absolute agreement = 77.8%). Inter-rater reliability was also “almost perfect” agreement for DIGESTSafetyw = 0.914; absolute agreement = 88.9%), DIGESTEfficiencyw = 0.887; absolute agreement = 66.7%), and DIGESTTotalw = 0.903; absolute agreement = 77.8%).

All six combinations of the approaches to derive a composite VASES residue rating exhibited “strong” correlational relationships [15] with the 4-point DIGEST-FEES ratings of pharyngeal residue (r = 0.75–0.77) – with no relationship sufficiently stronger than another. Therefore, the most parsimonious combination, which involved the maximal residue rating between either the oropharynx or hypopharynx, was used to establish the singular measure of pharyngeal residue for DIGEST-FEES grades (Fig. 1).

Fig. 1.

Left: Spearman’s correlation coefficients (dots) and their 95% CIs demonstrating the relationship between six combinations of VASES residue ratings with DIGEST-FEES pharyngeal residue ratings. Right: relationship between the VASES-derived pharyngeal residue rating (maximum rating between the oropharynx and hypopharynx) and DIGEST-FEES pharyngeal residue rating.

Fig. 1.

Left: Spearman’s correlation coefficients (dots) and their 95% CIs demonstrating the relationship between six combinations of VASES residue ratings with DIGEST-FEES pharyngeal residue ratings. Right: relationship between the VASES-derived pharyngeal residue rating (maximum rating between the oropharynx and hypopharynx) and DIGEST-FEES pharyngeal residue rating.

Close modal

Aim 1: Characterizing the Concurrent Validity of VASES-Derived DIGEST-FEES

Agreement between original and VASES-derived DIGEST-FEES grades is outlined in Table 2. Weighted Kappa revealed “substantial” agreement for DIGESTEfficiency, and “almost perfect” agreement for DIGESTSafety and DIGESTTotal. Descriptively, 60–62% of the original and VASES-Derived DIGEST-FEES grades matched exactly. All VASES-derived DIGESTSafety and DIGESTTotal grades were within one grade of the original, and 92% of the VASES-derived DIGESTEfficiency grades were within one grade of the original. Of the 19 VASES-derived DIGESTSafety grades that did not agree exactly, 10 (52%) were less severe than the original and nine (48%) were more severe than the original. Similarly, of the 20 VASES-derived DIGESTEfficiency grades that did not agree exactly with the original, 10 (50%) were less severe than the original and 10 (50%) were more severe than the original. Of the 20 VASES-derived DIGESTTotal grades that did not agree exactly, 12 (60%) were less severe than the original and eight (40%) were more severe than the original. Lastly, no significant differences were observed between original and VASES-derived grades for DIGESTSafety (p = 0.51, r = 0.16, 95% CI: −0.24, 0.51; Fig. 2), DIGESTEfficiency (p = 0.34, r = 0.24 95% CI: −0.53, 0.10; Fig. 3), DIGESTTotal (p = 0.38, r = 0.20, 95% CI: −0.14, 0.50; Fig. 4).

Table 2.

Agreement between VASES-Derived DIGEST-FEES and original DIGEST-FEES

DIGEST-FEESWeighted Kappa (κw)Exact agreement (%)Percent agreement within 1 grade (%)Percent agreement within 2 grades (%)
Safety grade 0.829 62.0 100 100 
Efficiency grade 0.769 62.0 92.0 100 
Total grade 0.832 60.0 100 100 
DIGEST-FEESWeighted Kappa (κw)Exact agreement (%)Percent agreement within 1 grade (%)Percent agreement within 2 grades (%)
Safety grade 0.829 62.0 100 100 
Efficiency grade 0.769 62.0 92.0 100 
Total grade 0.832 60.0 100 100 
Fig. 2.

Combined boxplots and violin plots representing the relationship between the original DIGESTSafety grade (left) and the VASES-derived DIGESTSafety grades (right). Dots represent raw data points, with dashed lines representing the change in grades, with darker dots and lines representing more frequent observations.

Fig. 2.

Combined boxplots and violin plots representing the relationship between the original DIGESTSafety grade (left) and the VASES-derived DIGESTSafety grades (right). Dots represent raw data points, with dashed lines representing the change in grades, with darker dots and lines representing more frequent observations.

Close modal
Fig. 3.

Combined boxplots and violin plots representing the relationship between the original DIGESTEfficiency grades (left) and the VASES-derived DIGESTEfficiency grades (right). Dots represent raw data points, with dashed lines representing the change in grades, with darker dots and lines representing more frequent observations.

Fig. 3.

Combined boxplots and violin plots representing the relationship between the original DIGESTEfficiency grades (left) and the VASES-derived DIGESTEfficiency grades (right). Dots represent raw data points, with dashed lines representing the change in grades, with darker dots and lines representing more frequent observations.

Close modal
Fig. 4.

Combined boxplots and violin plots representing the relationship between the original DIGESTTotal grades (left) and the VASES-derived DIGESTTotal grades (right). Dots represent raw data points, with dashed lines representing the change in grades, with darker dots and lines representing more frequent observations.

Fig. 4.

Combined boxplots and violin plots representing the relationship between the original DIGESTTotal grades (left) and the VASES-derived DIGESTTotal grades (right). Dots represent raw data points, with dashed lines representing the change in grades, with darker dots and lines representing more frequent observations.

Close modal

Aim 2: Characterizing the Construct Validity of VASES-Derived DIGEST-FEES

Correlation coefficients and their 95% CIs are outlined in Table 3. The correlations between the MDADI, FOIS, SSS, Yale Valleculae, and Yale Piriforms ranged from r = 0.34–0.83 for original DIGEST-FEES grades and r = 0.34–0.78 for VASES-derived DIGEST-FEES grades. There was insufficient evidence to demonstrate that the relationships between VASES-derived DIGEST-FEES grades and measures of construct validity were significantly stronger or weaker than the relationships between the original DIGEST-FEES grades and the same measures of construct validity.

Table 3.

Correlations between DIGEST-FEES and construct validity measures

DIGEST-FEES grading methodDIGESTSafetyDIGESTEfficiencyDIGESTTotal
ρ95% CIρ95% CIρ95% CI
MDADITotal 
 VASES derived −0.53 −0.71 to −0.32 −0.49 −0.68 to −0.25 −0.50 −0.69 to −0.26 
 Original −0.47 −0.67 to −0.24 −0.47 −0.68 to −0.22 −0.46 −0.67 to −0.20 
MDADIEmotional 
 VASES derived −0.46 −0.66 to −0.21 −0.46 −0.66 to −0.21 −0.43 −0.63 to −0.18 
 Original −0.40 −0.61 to −0.14 −0.42 −0.64 to −0.15 −0.40 −0.63 to −0.15 
MDADIFunctional 
 VASES derived −0.51 −0.69 to −0.27 −0.34 −0.56 to −0.10 −0.42 −0.62 to −0.18 
 Original −0.40 −0.63 to −0.16 −0.34 −0.57 to −0.07 −0.37 −0.57 to −0.11 
MDADIPhysical 
 VASES derived −0.56 −0.73 to −0.35 −0.52 −0.72 to −0.29 −0.54 −0.73 to −0.31 
 Original −0.49 −0.67 to −0.26 −0.52 −0.72 to −0.24 −0.50 −0.70 to −0.23 
FOIS 
 VASES derived −0.63 −0.80 to −0.43 −0.52 −0.70 to −0.28 −0.60 −0.77 to −0.37 
 Original −0.57 −0.75 to −0.35 −0.46 −0.67 to −0.18 −0.51 −0.70 to −0.27 
SSS 
 VASES derived 0.65 0.49–0.78 0.51 0.29–0.69 0.56 0.37–0.72 
 Original 0.51 0.25–0.69 0.46 0.21–0.67 0.54 0.31–0.72 
YaleValleculae 
 VASES derived 0.70 0.54–0.81 0.78 0.62–0.89 0.78 0.65–0.88 
 Original 0.74 0.60–0.84 0.82 0.68–0.91 0.83 0.70–0.91 
YalePiriforms 
 VASES derived 0.73 0.57–0.84 0.67 0.50–0.80 0.71 0.53–0.83 
 Original 0.64 0.44–0.78 0.63 0.43–0.78 0.69 0.50–0.83 
DIGEST-FEES grading methodDIGESTSafetyDIGESTEfficiencyDIGESTTotal
ρ95% CIρ95% CIρ95% CI
MDADITotal 
 VASES derived −0.53 −0.71 to −0.32 −0.49 −0.68 to −0.25 −0.50 −0.69 to −0.26 
 Original −0.47 −0.67 to −0.24 −0.47 −0.68 to −0.22 −0.46 −0.67 to −0.20 
MDADIEmotional 
 VASES derived −0.46 −0.66 to −0.21 −0.46 −0.66 to −0.21 −0.43 −0.63 to −0.18 
 Original −0.40 −0.61 to −0.14 −0.42 −0.64 to −0.15 −0.40 −0.63 to −0.15 
MDADIFunctional 
 VASES derived −0.51 −0.69 to −0.27 −0.34 −0.56 to −0.10 −0.42 −0.62 to −0.18 
 Original −0.40 −0.63 to −0.16 −0.34 −0.57 to −0.07 −0.37 −0.57 to −0.11 
MDADIPhysical 
 VASES derived −0.56 −0.73 to −0.35 −0.52 −0.72 to −0.29 −0.54 −0.73 to −0.31 
 Original −0.49 −0.67 to −0.26 −0.52 −0.72 to −0.24 −0.50 −0.70 to −0.23 
FOIS 
 VASES derived −0.63 −0.80 to −0.43 −0.52 −0.70 to −0.28 −0.60 −0.77 to −0.37 
 Original −0.57 −0.75 to −0.35 −0.46 −0.67 to −0.18 −0.51 −0.70 to −0.27 
SSS 
 VASES derived 0.65 0.49–0.78 0.51 0.29–0.69 0.56 0.37–0.72 
 Original 0.51 0.25–0.69 0.46 0.21–0.67 0.54 0.31–0.72 
YaleValleculae 
 VASES derived 0.70 0.54–0.81 0.78 0.62–0.89 0.78 0.65–0.88 
 Original 0.74 0.60–0.84 0.82 0.68–0.91 0.83 0.70–0.91 
YalePiriforms 
 VASES derived 0.73 0.57–0.84 0.67 0.50–0.80 0.71 0.53–0.83 
 Original 0.64 0.44–0.78 0.63 0.43–0.78 0.69 0.50–0.83 

(ρ), Spearman’s correlation coefficient; 95% CI, 95% confidence interval; MDADI, MD Anderson Dysphagia Inventory; FOIS, Functional Oral Intake Scale; Yale, Yale Pharyngeal Residue Severity Rating Scale.

This study aimed to determine if trial-level VASES ratings could be used to simultaneously derive valid and reliable protocol-level DIGEST-FEES impairment grades. From a concurrent validity standpoint, there was a high level of agreement between original DIGEST-FEES grades and VASES-derived DIGEST-FEES grades – with no statistically significant differences between the two grading methods. While these findings support the concurrent validity of VASES-derived DIGEST-FEES, it should be noted that the lack of statistically significant differences does not imply the two grading methods are necessarily equivalent. In fact, 38–40% of the VASES-derived DIGEST-FEES grades did not match exactly with the original DIGEST-FEES grades. However, there was no observable trend of VASES-derived DIGEST-FEES grades being consistently higher or lower than original DIGEST-FEES grades. There were also no observable trends of VASES-derived DIGEST-FEES grades being consistently higher or lower than original DIGEST-FEES grades as an effect of demographics, oncological medical history, or dysphagia severity (online suppl. material; for all online suppl. material, see https://doi.org/10.1159/000538935). Therefore, the differences that were present in DIGEST-FEES grades between the two rating methods may reflect a combined effect of methodological differences and rater reliability.

From a construct validity standpoint, original DIGEST-FEES grades and VASES-derived DIGEST-FEES showed similarly strong relationships with important metrics of swallow-related function, including management of secretions, functional oral intake, and impact of dysphagia on swallowing-related quality of life. Therefore, even while VASES-derived DIGEST-FEES grades may not perfectly align with original DIGEST-FEES, using VASES to derive DIGEST-FEES grades appears to be a valid and meaningful way to characterize swallow impairment at the protocol level.

VASES is intended to increase the transparency, standardization, and reliability of FEES analysis within clinical and research practices. In the present study, both intra- and inter-rater reliability were characterized as “almost perfect” (κw = 0.88–0.96), with exact agreement ranging from 66.7% to 88.9%. Descriptively, the rater reliability in the present study appears higher than the reliability results which have been previously reported for DIGEST-FEES [4, 16, 17]. While reliability can be affected by factors beyond rater reliability (e.g., number of raters, number of FEES, and variability in grades used for reliability analysis) [18], the use of VASES may be one reason for descriptively higher reliability results. These findings support the notion that VASES may be one way to increase transparency, standardization, and reliability of FEES analysis at both the bolus and protocol (DIGEST-FEES) levels.

There are several limitations to this study. First, the methods used to combine VASES residue ratings into one pharyngeal residue rating were created a priori through expert panel discussion. All the methods in the study were similarly strong, and therefore, the simplest method was used for data analysis. However, it is possible that other unexplored combinations of VASES residue ratings may be more accurate. It is also possible that the method for deriving one overall rating of residue with VASES may vary across consistencies, which was unexplored in the current study. Second, all FEES included in the current study were in patients with head and neck cancer. This may limit the generalizability of results to other patient populations. Lastly, this study was not designed to compare differences in intra- and inter-rater reliability between the original and VASES-derived DIGEST-FEES grading methods. Therefore, future research should more comprehensively assess the effects of VASES on intra- and inter-rater reliability of DIGEST-FEES by including a greater number of raters, with a range of FEES interpretation experience, who rate FEES using both grading methods.

Findings from this study demonstrate that analyzing FEES at the trial level with VASES may be a valid and reliable approach to derive global ratings of swallowing safety, swallowing efficiency, and overall swallow function using DIGEST-FEES. While results indicate that DIGEST grades derived from VASES do not perfectly align with DIGEST-FEES, nearly all participants were within one grade. Future research should expand on this study in different patient populations.

All procedures were performed in accordance with the ethical standards of the institutional research committee and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards. This study protocol was reviewed and approved by the Institutional Review Board of Stanford University (Approval #IRB-51439). The need for informed consent for this study was waived by the Institutional Review Board of Stanford University.

The authors have no relevant financial disclosures or conflicts of interest.

This study was supported in part by the American Speech-Language-Hearing Foundation’s 2018 Researcher-Practitioner Collaboration Grant originally awarded to Heather Starmer and Dr. Kate Hutcheson.

All authors met the minimum criteria for authorship status, as proposed by the International Committee of Medical Journal Editors. Authorship contributions are outlined below using CRediT (https://credit.niso.org/).

James Curtis: conceptualization, data curation, formal analysis, investigation, methodology, project administration, resources, software, visualization, and writing – original draft. Loni Arrese and Lauren Tabor Gray: data curation, project administration, resources, and writing – reviewing and editing. James Borders: conceptualization, investigation, methodology, validation, visualization, and writing – reviewing and editing. Heather Starmer: conceptualization, data curation, funding acquisition, investigation, methodology, resources, supervision, and writing – reviewing and editing.

The data that support the findings of this study are not publicly available due to institutional restrictions. However, data may be available upon reasonable request and completion of a data use agreement. Requests for data can be made to the senior author (H.S.).

1.
Giraldo-Cadavid
LF
,
Leal-Leaño
LR
,
Leon-Basantes
GA
,
Bastidas
AR
,
Garcia
R
,
Ovalle
S
, et al
.
Accuracy of endoscopic and videofluoroscopic evaluations of swallowing for oropharyngeal dysphagia
.
Laryngoscope
.
2017
;
127
(
9
):
2002
10
.
2.
Kasle
DA
,
Torabi
SJ
,
Savoca
EL
,
Lerner
M
,
Manes
RP
.
High-volume billing and reimbursement trends for endoscopic swallowing studies in the medicare population
.
Dysphagia
.
2021
;
36
(
5
):
919
24
.
3.
Langmore
SE
.
History of fiberoptic endoscopic evaluation of swallowing for evaluation and management of pharyngeal dysphagia: changes over the years
.
Dysphagia
.
2017
;
32
(
1
):
27
38
.
4.
Starmer
HM
,
Arrese
L
,
Langmore
S
,
Ma
Y
,
Murray
J
,
Patterson
J
, et al
.
Adaptation and validation of the dynamic imaging grade of swallowing toxicity for flexible endoscopic evaluation of swallowing: DIGEST-FEES
.
J Speech Lang Hear Res
.
2021
;
64
(
6
):
1802
10
.
5.
Curtis
JA
,
Borders
JC
,
Perry
SE
,
Dakin
AE
,
Seikaly
ZN
,
Troche
MS
.
Visual Analysis of Swallowing Efficiency and Safety (VASES): a standardized approach to rating pharyngeal residue, penetration, and aspiration during FEES
.
Dysphagia
.
2022
;
37
(
2
):
417
35
.
6.
Curtis
JA
,
Borders
JC
,
Troche
MS
.
Visual analysis of swallowing efficiency and safety (VASES): establishing criterion-referenced validity and concurrent validity
.
Am J Speech Lang Pathol
.
2022
;
31
(
2
):
808
18
.
7.
Hutcheson
KA
,
Barrow
MP
,
Barringer
DA
,
Knott
JK
,
Lin
HY
,
Weber
RS
, et al
.
Dynamic imaging grade of swallowing toxicity (DIGEST): scale development and validation
.
Cancer
.
2017
;
123
(
1
):
62
70
.
8.
Hutcheson
KA
,
Barbon
CEA
,
Alvarez
CP
,
Warneke
CL
.
Refining measurement of swallowing safety in the dynamic imaging grade of swallowing toxicity (DIGEST) criteria: validation of DIGEST version 2
.
Cancer
.
2022
;
128
(
7
):
1458
66
.
9.
Curtis
JA
,
Borders
JC
,
Dakin
AE
,
Troche
MS
.
Normative reference values for FEES and VASES: preliminary data from 39 nondysphagic, community-dwelling adults
.
J Speech Lang Hear Res
.
2023
;
66
(
7
):
2260
77
.
10.
Rosenbek
JC
,
Robbins
J
,
Roecker
EB
,
Coyle
JL
,
Wood
JL
.
A penetration-aspiration scale
.
Dysphagia
.
1996
;
11
(
2
):
93
8
.
11.
R Core Team
.
R: a language and environment for statistical computing [Internet]
.
Vienna, Austria
:
R Foundation for Statistical Computing
;
2023
. Available from: https://www.R-project.org/.
12.
Crary
MA
,
Mann
GDC
,
Groher
ME
.
Initial psychometric assessment of a functional oral intake scale for dysphagia in stroke patients
.
Arch Phys Med Rehabil
.
2005
;
86
(
8
):
1516
20
.
13.
Ota
K
,
Saitoh
E
,
Baba
M
,
Sonoda
S
.
The secretion severity rating scale: a potentially useful tool for management of acute-phase fasting stroke patients
.
J Stroke Cerebrovasc Dis
.
2011
;
20
(
3
):
183
7
.
14.
Neubauer
PD
,
Rademaker
AW
,
Leder
SB
.
The Yale pharyngeal residue severity rating scale: an anatomically defined and image-based tool
.
Dysphagia
.
2015
;
30
(
5
):
521
8
.
15.
Schober
P
,
Boer
C
,
Schwarte
L
.
Correlation coefficients: appropriate use and interpretation
.
Anesth Analg
.
2018
;
126
(
5
):
1763
8
.
16.
Labeit
B
,
Lapa
S
,
Muhle
P
,
Suntrup-Krueger
S
,
Claus
I
,
Gandor
F
, et al
.
Validation of the DIGEST-FEES as a global outcome measure for pharyngeal dysphagia in Parkinson’s disease
.
Dysphagia
.
2023
;
22
:
1
8
.
17.
Simon
SR
,
Wieland
MWM
,
Hendriks
C
,
Pilz
W
,
Schindler
A
,
Winkens
B
, et al
.
Intra and interobserver agreement of the Dynamic Imaging Grade of Swallowing Toxicity Scale (DIGEST) in fiberoptic endoscopic evaluation of swallowing (FEES): the importance of observer-tailored training
.
Eur Arch Otorhinolaryngol
.
2023
;
280
(
6
:
2865
76
.
18.
Koo
TK
,
Li
MY
.
A guideline of selecting and reporting intraclass correlation coefficients for reliability research
.
J Chiropr Med
.
2016
;
15
(
2
):
155
63
.