Chaos and the Match

This forum made possible through the generous support of SDN members, donors, and sponsors. Thank you.

Hitch

Alfred J.
10+ Year Member
5+ Year Member
15+ Year Member
Joined
Jan 10, 2006
Messages
36
Reaction score
0
I'm wondering how reproducible the match results are. Theoretically the computer should spit out the same answer no matter how many times it's asked to run the algarithm. But there are an aweful lot of rank lists for it to consider. What if it made even one little glitch? That's not so unlikely is it? Couldn't that throw everything off, one step and create an entirely different match result? I'm just curious about this. Can it happen?
 
Thats why they run the algorithim hundreds of times in different order, and take so long to release results, to verifiy the reproducibility.
 
tigershark said:
Thats why they run the algorithim hundreds of times in different order, and take so long to release results, to verifiy the reproducibility.

Really? That's reassuring. 👍 I assumed that by "verify" they just meant that they checked they put the right messeges in the right envelopes.
 
Hitch said:
I'm wondering how reproducible the match results are. Theoretically the computer should spit out the same answer no matter how many times it's asked to run the algarithm. But there are an aweful lot of rank lists for it to consider. What if it made even one little glitch? That's not so unlikely is it? Couldn't that throw everything off, one step and create an entirely different match result? I'm just curious about this. Can it happen?

The uro match got f*cked up last year. Ultimately, things were corrected. Not really sure of the details, but yes, apparently it can happen.

-PB
 
PickyBicky said:
The uro match got f*cked up last year. Ultimately, things were corrected. Not really sure of the details, but yes, apparently it can happen.

from http://www.auanet.org/residents/resinvestigate_Old.cfm

Executive Summary of Investigative Report

At its scheduled February meeting in Baltimore, the AUA Board of Directors sought detailed explanation of the serious error encountered in the 2005 Urology Residency Match on January 24. The AUA President appointed a committee of the Board, chaired by Secretary Dr. Carl Olsson, to seek out the root causes of the mismatch, and to suggest corrective measures for the future.

The committee secured relevant AUA internal documents including e-mails and memos, interviewed staff responsible for the Match process, and reviewed the comments of Program Directors and residency candidates directly affected by the error. The committee reached the following findings and conclusions:

1. Shortly after the Match results were disseminated on January 24, a problem was first noted by several Program Directors who called attention to unexpected vacancies in filling their residency slots. The initial staff response was that the "Match process was fair and accurate" because it was based on the logic of past Matches, and had been "manually audited." However, it became clear by later that day that there was in fact a serious error.

2. The mismatch was caused not by a computer glitch but by human error. AUA staff misapplied the Match algorithm, giving primary importance to the programs' preference rather than to the candidates' preference. Both the original Match and the "manual audit" were conducted by junior staff, who lacked full understanding of the Match Guidelines.

3. Three major staff errors were identified:
* Senior staff responsible for the Match failed to follow standard IT procedures, which required testing and validation of the system and its results, before those results were released. "Red flags" were dismissed when they appeared.
* No one sufficiently knowledgeable about the residency programs reviewed the Match outcomes to judge their conformance with expected results.
* When the Match error was confirmed, senior staff did not respond with timely and appropriate apologies, or with explanations to those adversely affected by the results.

4. The mismatch was corrected within hours of first notice, and the corrected and verified Match results were communicated to all participants the same week (January 28). Unfortunately, because of a fax equipment problem, the candidates learned their results before the Program Directors were informed. Later apologies were sent to all concerned by AUA President Dr. Brendan Fox.

5. Importantly, the final Residency Match results were tested and accuracy was confirmed by an independent auditor.

Appropriate steps have been taken to ensure there will be no recurrence of the problems experienced this year. In the future, the Residency Match Review Process will include oversight by urologists from AUA Board of Directors and from the Society of Urology Chairpersons and Program Directors, an affiliate of the Society for University Urologists.
 
scrappy said:
1. Shortly after the Match results were disseminated on January 24, a problem was first noted by several Program Directors who called attention to unexpected vacancies in filling their residency slots. The initial staff response was that the "Match process was fair and accurate" because it was based on the logic of past Matches, and had been "manually audited." However, it became clear by later that day that there was in fact a serious error.

2. The mismatch was caused not by a computer glitch but by human error. AUA staff misapplied the Match algorithm, giving primary importance to the programs' preference rather than to the candidates' preference. Both the original Match and the "manual audit" were conducted by junior staff, who lacked full understanding of the Match Guidelines.

3. Three major staff errors were identified:
* Senior staff responsible for the Match failed to follow standard IT procedures, which required testing and validation of the system and its results, before those results were released. "Red flags" were dismissed when they appeared.
* No one sufficiently knowledgeable about the residency programs reviewed the Match outcomes to judge their conformance with expected results.
* When the Match error was confirmed, senior staff did not respond with timely and appropriate apologies, or with explanations to those adversely affected by the results.

4. The mismatch was corrected within hours of first notice, and the corrected and verified Match results were communicated to all participants the same week (January 28). Unfortunately, because of a fax equipment problem, the candidates learned their results before the Program Directors were informed. Later apologies were sent to all concerned by AUA President Dr. Brendan Fox.

5. Importantly, the final Residency Match results were tested and accuracy was confirmed by an independent auditor.

Appropriate steps have been taken to ensure there will be no recurrence of the problems experienced this year. In the future, the Residency Match Review Process will include oversight by urologists from AUA Board of Directors and from the Society of Urology Chairpersons and Program Directors, an affiliate of the Society for University Urologists.

:laugh:
 
Considering the size of the NRMP match staff >>>>>>> AUA match staff I'm pretty sure that such a scenario is extremely unlikely to repeat itself. Hindsight is 20/20 and I think the NRMP can learn a valuable lesson from the AUA's gaffe. Also, I'm sure they are extra careful since that whole lawsuit vs. the Match blew over.
 
Gfunk6 said:
Considering the size of the NRMP match staff >>>>>>> AUA match staff I'm pretty sure that such a scenario is extremely unlikely to repeat itself. Hindsight is 20/20 and I think the NRMP can learn a valuable lesson from the AUA's gaffe. Also, I'm sure they are extra careful since that whole lawsuit vs. the Match blew over.

So whatever happened with that lawsuit anyway?
 
Gfunk6 said:
Considering the size of the NRMP match staff >>>>>>> AUA match staff I'm pretty sure that such a scenario is extremely unlikely to repeat itself. Hindsight is 20/20 and I think the NRMP can learn a valuable lesson from the AUA's gaffe. Also, I'm sure they are extra careful since that whole lawsuit vs. the Match blew over.

Yes. Knowing you can't be sued does tend to make you extra careful. :laugh: :laugh:
 
Top