Second Trial - FAQs
Who is conducting the SECOND Trial?
Like the FIRST Trial, the SECOND Trial is a joint effort between the Accreditation Council for Graduate Medical Education (ACGME), the American College of Surgeons (ACS), the Association of Program Directors in Surgery (APDS), and the American Board of Surgery (ABS). As in the FIRST Trial, the ACGME and ACS will not have access to the data; this is clearly stated in our contracts with each program and with the ACGME and ACS. The Surgical Outcomes and Quality Improvement Center (SOQIC) at the Northwestern University Feinberg School of Medicine will serve as the coordinating center for the trial.
When will the SECOND Trial start and end?
Trial enrollment began in fall of 2019. All programs will be given access to the Wellness Toolkit in early 2023.
How long will the data for reports be collected?
Data will be collected at the time of the ABSITE survey from now until 2022. There will be an interim analysis in 2021.
How will you track wellness initiatives implemented by programs throughout the duration of the study?
We will conduct an annual Program Director survey that captures all initiatives or policies programs implemented in the past year.
Are there any associated costs?
There are no costs associated with the participation of this study. The SECOND Trial provides data, "how to" guidelines for interventions, and access to us/content experts free of charge. Certain interventions may have associated costs, which vary depending on the specific intervention. These are not charges collected by the SECOND Trial, but estimates for your budgeting purposes. For example, we provide a free protocol for integrating a psychologist into the residency. This is potentially a very expensive intervention, as the department will have to cover that person's FTE; however, these are not charges collected by the SECOND Trial.
Can we share the program report results with residents?
We encourage sharing with your residents and faculty, but we ultimately leave this decision up to each program's leadership. Please note that this data cannot be shared outside of your program any way, including verbally, during presentations, on social media. We require this in order to maintain confidentiality of all programs (i.e., if the high performing programs advertised their data, the low performing ones would be "outed" by inference). We will remove anyone who violates this requirement from the Trial.
Can all of our residency programs do this or is this only for general surgery?
You are welcome to share the Toolkit with any residency programs WITHIN YOUR INSTITUTION to help improve resident well-being and learning environment. Sharing with people or programs outside of your institution is strictly prohibited; violations will result in removal from the Trial.
Is the survey of residents only through ABSITE or will there be a second survey and/or phone calls?
The SECOND Trial does not work with residents directly. The only survey residents will be completing is the ABSITE. We will ask program directors to complete a Program Director Survey annually so we may track which policies/interventions they implement.
How will the SECOND Trial examine whether program-specific data and/or the Toolkit work?
At the conclusion of the SECOND Trial, metrics of the learning environment and resident well-being (e.g., burnout) will be compared between the intervention and control groups to assess the intervention’s effectiveness. Both baseline and follow-up data will be obtained from the ABSITE survey, which is administered to all residents in ACGME-approved general surgery programs and typically has a high response rate. We will also conduct an annual Program Director survey to assess what interventions programs in both arms have implemented.
Where do the interventions come from?
We combed the literature looking for novel interventions and best practice guidelines, consulted with authors/experts, and adapted them for surgical residency programs. We also conducted a Program Director Survey and conducted phone interviews of program directors who reported novel and interesting interventions at their institutions. Finally, we conducted 2 day Program Tours at representative programs, identified through a combination of notoriety in wellness, ABSITE data performance, and Program Director Survey reports. On these Program Tours, we brought a multidisciplinary team (e.g., residents, psychologists, psychiatrists, qualitative researchers) to perform structured observations of educational conferences and conduct focus groups and individual interviews with residents, faculty, support staff, and departmental, institutional, and educational leadership, with the goal of learning about their experiences, infrastructures, policies, interventions, leadership priorities, and culture. Everything we learned has been adapted into a Toolkit intervention.
I attended the webinar, but I need more help with my intervention.
We encourage you to use the Comments section of each intervention to communicate with your coach as well as other programs that are also working on implementation. We will also coordinate additional coaching sessions, as the need arises. Please email us at SENTteam@iu.edu with your requests, and we will make the appropriate arrangements.
We have an intervention we would like to contribute to the Wellness Toolkit!
We are eager to hear about any and all new interventions! We'd love to include yours. We also encourage collaborative research. If you serve as a coach for your intervention, you may conduct your own research projects on the cohort of programs that choose to implement it. We may also be able to provide you with some of our survey data on that cohort, as well. Please reach out to the SECOND Trial team at SENTteam@iu.edu for more information.
Who can participate in our wellness committee?
Diverse perspectives are important. We recommend that your wellness committee involve attendings and residents. We've been asked about whether it'd be appropriate to include residents' family members. As they are an integral part of work-life integration, we certainly encourage soliciting their perspective and efforts, but we would also recommend careful attention to balance; resident voices should be primary.
I heard the SECOND Trial is giving everyone bad reports so that everyone is motivated to improve!
While we a little bit enjoy this characterization of us as evil geniuses (of wellness?), this one is pure fiction. Your data is real, and the benchmarking is real. We would love to give everyone great reports! (And if everyone was great, we actually wouldn’t quartile it; we don’t force arbitrary data cut-offs just for the sake of differentiating between programs. We look at the statistical distribution of programs and decide what cut points make sense.)
Green reports probably reflect programs where the culture is so unsafe that people are afraid to be honest even on an anonymous survey. So at least our residents feel safe.
There are indeed programs that rank in the best quartile on many metrics whose residents are not lying. We have seen them in person on our Program Tours. And yes, after 2 days of observations and interviews, we always get the whole story. We recognize that this may be hard data to see. As our experience feeding back quality improvement data to hospitals tells us, the first stage is denial: I operate on sicker patients, my residents are more honest, etc, etc. That’s why we benchmark your data against programs like yours. And as with grief, the next stage is often anger. It may be tempting to try to scapegoat a few people at your program whom you perceive to be the problem and/or your residents for reporting it, but we strongly advise against this. Everyone contributes to institutional culture, and everyone should take responsibility. Moreover, there is a name for blaming the reporters: retaliation. And it often backfires, leaving an even bigger wellness hole to dig out of. Encourage your program to move as quickly as you can towards acceptance so you can get to productive action.
How will the SECOND Trial examine whether program-specific data and/or the Toolkit work?
At the conclusion of the trial, metrics of the learning environment and resident well-being (e.g., burnout) will be compared between the intervention and control groups to assess the intervention’s effectiveness. Both baseline and follow-up data will be obtained from the ABSITE survey, which is administered to all residents in ACGME-approved general surgery programs and typically has an 85-99% response rate.
Why randomize? Isn’t it unethical to withhold resources from the control group?
Both trial arms have access to data about their residents’ well-being (e.g., burnout) and interventions regarding suicide, but only intervention arm programs have access to the Learning Environment Report, Wellness Toolkit, and the implementation support. Because there is currently little data about the effectiveness of any wellness interventions and many are expensive and/or time/effort-intensive, our Bioethics Panel concluded that the SECOND Trial has equipoise.
Randomization allows us to address the secular trend of increasing emphasis on wellness, as programs who are independently embarking on wellness initiatives should be evenly divided between intervention and control arms. We closely track the wellness interventions implemented at all programs in both study arms annually.
After the conclusion of the Trial, access to the Wellness Toolkit will be expanded to all enrolled programs.
How will the confidentiality of individual residents be protected?
We recognize that these are sensitive topics. All individual resident identifiers are removed from survey data prior to being transferred to SOQIC; identification of individual residents by the study team is not possible. Data metrics on our Program-Specific Reports are provided as quartiles (i.e., for burnout, your program ranks in the first (best) quartile of programs in the country). We do not provide programs with the responses of individual residents or even the proportion of their residents that reported any particular metric, thus precluding attempts to identify the residents who might have reported any particular issue. During Program Tours, all interviews and focus groups are conducted confidentially.
How will the confidentiality of programs be protected?
ABSITE survey data are sent to SOQIC, and programs are immediately de-identified. It is maintained in de-identified form throughout all analyses. Program-Specific Reports will be generated using program-level linkages, which are securely maintained.
Much like other clinical quality improvement programs, we will mandate that programs cannot disseminate or publicize/advertise the data in their reports outside of their institutions. We encourage sharing within each institution (i.e., with residents and faculty), but this decision will be left to the discretion of each Program Director, Chair, and DIO.
We have worked hard on our interventions. How can we participate in the research?
We at the SECOND Trial are supportive of collaborative research efforts. If you contribute novel interventions to the Wellness Toolkit and serve as coaches or topical experts for programs that decide to implement them, you may participate in scholarship in the following way: We can analyze and give you ABSITE survey results to write up for the subset of programs that choose your intervention from the Wellness Toolkit. (Our data use agreements prohibit us from sharing the raw data.) Of course, then your metrics are limited to the ones that are already measured in the ABSITE survey.
We envisioned the Wellness Toolkit to be an interactive application, to which programs can add their tips, tricks, and experiences as they begin implementing the interventions that you contributed. In this way, the SECOND Trial weaves together the best pieces from all programs to maximize our collective impact on resident well-being.