Objective: Medical students rely on the internet as a resource to gather information about residency programs, although little data exist on the quality or completeness of these websites. Therefore, we sought to evaluate the accessibility of educational and recruitment content of radiation oncology residency websites in the US.
Methods and Materials: The names of radiation oncology residency programs were obtained from the Electronic Residency Application Service. Websites were evaluated for the presence of 20 unique features related to categories of application process, work incentives, educational instruction, research, clinical training, and program leadership introduction. Programs were organized by geographic location, size and ranking for further analysis. Univariate logistic regression was performed to assess predictors of matching in the 2019 cycle.
Results: A total of 92 analyzable websites were identified. Individual program websites contained a mean (SD) of 9.75 (3.8) of the 20 factors sought (49%). Ten (11%) program websites had at least 75% of the 20 features. In addition, 37 (40%) programs had less than 50% of the features listed on their websites. When evaluated by geographic region, no differences in the amount of information available on each website were noted (p = 0.102). Furthermore, there was no significant difference in mean number of features reported by large or small programs (10.80 vs 9.15, p = 0.114) and by ranking (9.96 vs 9.68, p = 0.760). Large programs were more likely to fill all their spots in the 2019 match (OR 3.85, p = 0.013) and there was a nonsignificant trend in increased likelihood of matching with 6 to 15 features on program websites (OR 2.07-2.14).
Conclusion: With the recent high unmatched rate in radiation oncology residency programs, methods to improve the recruitment process are of even greater importance. Many radiation oncology residency websites appear to be incomplete. Improvement in the comprehensiveness and accessibility of radiation oncology websites may improve the recruitment process and allow for medical students to make more informed decisions.
Medical students interested in obtaining a radiation oncology residency position often use the internet as a resource to gather information about residency programs. Although there have been no surveys specific to radiation oncology applicants, studies in other medical specialties have confirmed the importance of online program information.1-8 Therefore, it is important for residency programs to maintain informative and comprehensive websites for prospective radiation oncology applicants.
Prior studies in other medical specialties have demonstrated that residency program websites are often suboptimal and that missing information can be crucial for applicants to determine which programs are a better “fit” for them.9,10 Given that program websites may be the only novel program-specific resource medical students have before applying, completeness of program information may be a significant factor in allowing residency programs to remain competitive for applicants, particularly with the recent increase in unmatched radiation oncology program spots in 2019.11
Previous studies in other medical specialties have found deficiencies in online content on residency program websites; however, similar information regarding the availability and quality of current radiation oncology residency website content is not available. We sought to evaluate the accessibility of educational and recruitment content of radiation oncology residency websites in the US.
A list of US radiation oncology residency programs was obtained from the Electronic Residency Application Service (ERAS). All websites were publicly available. This study did not require institutional review board approval per the SUNY Downstate Medical Center.
The program websites were accessed through the link provided by ERAS or through an online search. Websites were evaluated for the presence of 20 unique features related to categories of application process, work incentives, educational instruction, research, clinical training, and program leadership introduction (Table 1). The 20 features were derived from published studies in other specialties evaluating residency websites and were considered to be relevant to the field of radiation oncology by the authors.4,5,12 Inclusion of information required its presence directly on the radiation oncology residency or department website. However, information on salary, benefits, parking, and faculty listing was considered present if it was accessible by a direct link from the residency website.
Programs were organized by geographic location and residency size for further analysis. Programs were divided into institutions based in the Northeast (Maryland, Pennsylvania, Delaware, New Jersey, New York, Connecticut, Massachusetts, Vermont, Rhode Island, New Hampshire, Maine, and the District of Columbia), South (Virginia, Kentucky, Arkansas, Oklahoma, Texas, North Carolina, South Carolina, Georgia, Florida, Louisiana, Mississippi, Alabama, Tennessee), West (New Mexico, Colorado, Washington, Oregon, Nevada, Utah, Arizona, California, Hawaii), and Midwest (Nebraska, Kansas, Minnesota, Iowa, Missouri, Wisconsin, Illinois, Michigan, Indiana, Ohio, West Virginia). States/territories without programs included Alaska, Wyoming, Montana, Idaho, North Dakota, Puerto Rico, and South Dakota. Program size was determined by the median number of residents in the programs, with ≤ 7 residents considered to be small and ≥ 8 considered large. Programs were then categorized by the ranking of the cancer program per US News and World Report as a “Top 25” vs a “Not Top 25” program and further categorized using the National Resident Matching Program data as having filled or not filled in the 2019 cycle. Chi-square and Mann-Whitney/Kruskal-Wallis tests were used to compare categorical and continuous variables, respectively. Univariate logistic regression was performed to assess predictors of matching in the 2019 cycle. SPSS version 21.0 (IBM Inc., Armonk, New York) was used for statistical analysis.
A total of 94 radiation oncology residency programs was obtained from ERAS. Of the 94 programs, 92 websites were evaluated (2 websites were inaccessible or nonexistent). Individual program websites contained a mean (SD) of 9.75 (3.8) of the 20 factors sought (49%).
Only 10 (11%) of the program websites had at least 75% of the 20 features. In addition, 37 (40%) programs had < 50% of the features listed on their websites. Websites ranged from having 15% of the features to as much as 90%. Most sites had contact e-mail (98%), research opportunities (86%), and a comprehensive faculty listing (82%), while fewer than a quarter of programs had a message from the chairperson (24%), call schedule (15%), or parking information (9%) (Table 1).
After organizing by geographical location, the Northeast contained 29% of programs, the South 29%, the West 19%, and the Midwest 23%. There were no differences in the amount of information available on each website based on geographic region (p = 0.102). There were 45 large programs and 47 small programs, and there was no significant difference in mean number of features reported by large or small programs (10.90 vs 9.15, p = 0.114). Comparison of Top 25 vs Not Top 25 programs by the US News and World Report found no difference in mean number of features reported (9.96 vs. 9.68, p = 0.760) (Table 2).
There were 88 programs that entered the NRMP in 2019 of which 22 (25%) did not fill all positions. On univariate logistic regression, large programs were more likely to fill their spots in the 2019 match (OR 3.85, p = 0.013) and there was a nonsignificant trend in increased likelihood of matching with 6 to 15 features on program websites (OR 2.07-2.14)(Table 3).
The 2019 Match Day results, with 22 programs (25%) going unfilled for the first time in many years, was a surprising development after years of a competitive match in radiation oncology. The decline in applications is likely multifactorial, with causes including an anticipated future oversupply of radiation oncologists and a much higher-than-usual failure rate on the 2018 radiation biology and physics qualifying board examinations.13,14
As future generations of medical trainees undoubtedly will continue to use the internet as a resource for investigating residency specialties and individual programs, having comprehensive program websites will continue to grow in importance. In this study, we evaluated current program websites based on 20 criteria and note that on average, programs met about half of these predefined criteria. Furthermore, the geographical location and program size were not related to website completeness, implying that the issue is widespread and overlooked among a variety of programs.
While the deficiencies in online content available for other medical specialties has been reported extensively,1-8 information on availability and quality of online information regarding radiation oncology programs is limited.12 We found that important information for prospective residents, including the number of match spots, selection criteria, and alumni job placement, is omitted in the majority of websites. Previous studies have noted that websites influence prospective applicants’ decisions5,7 and that an easily navigable site may be an important factor in deciding where to apply.3 The lack of information on radiation oncology residency websites may leave applicants with insufficient information with which to gauge their interest in a particular program.
It is likely that another resource programs use is social media, which may be used in recruitment. A survey study of prospective anesthesia residents showed that the majority (52.8%) felt a residency-based social media account impacted their evaluation of programs. Specifically, the most popular platforms included Doximity and Facebook.15 With an increasing Twitter presence in oncology,16 its utilization by programs may also be an emerging trend in resident recruitment.
This study has several limitations. First, the choice of program website features by the study team was completed through extensive literature review of desired features in other specialties as well as consensus on factors specific to radiation oncology; however, additional factors of interest to medical students may not have been included. Second, due to website variability, available features may have been overlooked despite thorough review. There was also no official way to verify the accuracy of the information posted on the websites. Furthermore, intangible factors such as website design and ease of use were not assessed in this study. Nonetheless, these results highlight several areas for potential improvement.
The recent match results indicate that individual residency programs, and even our field as a whole, cannot be complacent when it comes to attracting the best medical students. We demonstrate that residency program websites, a medical student’s first and sometimes final look at a program, often lack completeness. Enhancing the quality and completeness of residency program websites may be a very high-yield first step toward optimizing future matches and reversing the recent concerning increase in unfilled radiation oncology residency spots.
Sayan M, Youssef I, Gupta A, Ohri N, Jabbour SK, Parikh R, Wagman R, Haffty BG, Kim S, Weiner J, Lee A. An Assessment of the Comprehensiveness of Radiation Oncology Residency Websites. Appl Rad Oncol. 2020;9(1):24-27.
Dr. Sayan and Dr. Gupta are residents; Dr. Ohri, Dr. Wagman, and Dr. Weiner are assistant professors; Dr. Jabbour and Dr. Haffty are professors; and Dr. Parikh and Dr. Kim are associate professors; all in the Department of Radiation Oncology, Rutgers Cancer Institute of New Jersey, New Brunswick, NJ. Dr. Youssef is a resident, Department of Radiation Oncology, SUNY Downstate Medical Center, Brooklyn, NY. Dr. Lee is a fellow, Department of Radiation Oncology, Memorial Sloan Kettering Cancer Center, New York, NY. Disclosure: The authors have no conflicts of interest to disclose. None of the authors received outside funding for the production of this original manuscript and no part of this article has been previously published elsewhere. This study was presented at the Annual Meeting for the American Society for Radiation Oncology (ASTRO), Chicago, IL, September 15-19, 2019.