 |
| Is Greg celebrating another district win, or about to smash the trophy on someone's head after getting DQ'ed at states? |
In the previous post, I proved that SFS being DQ'ed 5 out of 61 relays cannot be explained by random chance. So if this large number of DQ's cannot be explained by random chance, is there anything else besides referee bias that can explain this event? Here are some possibilities:
- SFS swimmers are more prone to false starts. As mentioned previously, the 5 DQ's were all blamed on different swimmers. In fact, only 2 swimmers were on more than one DQ'ed relay (and they were only on 2 DQ'ed relays). Unless the murky water in the St. Francis pool is causing the swimmers' muscle fibers to twitch too quickly, I don't think the swimmers can explain the difference. But I will back this up with numbers in a minute.
- While the swimmers have changed over the past 11 years, the coach, Keith Kennedy, has not. Is Keith to blame for these DQ's, or is he the muse of Lady (Bad) Luck? In my 4 years being coached by him, he never instructed us to false start. In fact, if anything, all of these DQ's have caused him to stress the importance of not pushing the starts. So I'm not buying this explanation either.
You may be thinking that I am too close to the situation to blame Keith or the swimmers. So let me finish with one last analysis that will put the "!" on this debate. In order to qualify for the state tournament, every relay must place high enough at the district tournament to advance. The OHSAA website also shows the Northwest Ohio District swimming results for 8 of the last 9 years (2006 is missing: apparently they are still waiting for one of the other districts to finish swimming before posting results). The coach and, for the most part, swimmers on the relay do not change from districts to states.
For the available data, 15 of the 529 non-SFS relay swims (2.8%) have been DQ'ed at the district meet. This is about twice as frequent as the perennial contenders at the state meet (1.3%), which should be expected as the quality of swimmers at districts are not as high as the swimmers and relays at states. Still, this shows that officials are DQ'ing more relays at the district meet than the state meet. SFS has not been DQ'ed at districts in the past 11 years for a total of 33 relays (even though I don't have the district results for all years, had a relay been DQ'ed at districts, they would not have qualified for the state meet, which I have data for). The probability that a team is not disqualified in 33 relays at the district meet if all relays are swam independently is
(1 - 0.028)33 = 0.405
Let's do the same analysis, but using the DQ frequency of SFS at the state meet. That is, the probability that a team that is DQ'ed 8.2% of the time at states would go 33 straight relays without getting DQ'ed at districts is
(1 - 0.082)33 = 0.059
In other words, it is highly probable that a given school will not be DQ'ed in 11 years at districts (probability of about 40%). But schools that are DQ'ed as often as SFS at states would expect to be DQ'ed at least once at districts 94% of the time.
Now, one can argue that because the competition is less at districts than states, that the swimmers play the relay starts safe at districts but push them at states. But I would also expect this to hold for all of the perennial contenders, so this doesn't explain the difference in DQ's between SFS and other perennial contenders at the state meet. One could also argue that there is a referee bias for the perennial contenders at districts. Why? The officials may want the district to have a good showing at the state meet, so it would hurt the district to DQ one of the top relays even if there really was a false start. However, it is tough to quantify this bias in favor of SFS at the district meet.
This look of the district data should show that the large number of DQ's is most likely not fully explained by the coach or swimmers, as you would expect to see similar DQ patterns at the district meet. This three-part series conclusively shows that it is extremely unlikely that there is no referee bias against the SFS swim team at the state meet.
Now, one can argue that because the competition is less at districts than states, that the swimmers play the relay starts safe at districts but push them at states. But I would also expect this to hold for all of the perennial contenders, so this doesn't explain the difference in DQ's between SFS and other perennial contenders at the state meet. One could also argue that there is a referee bias for the perennial contenders at districts. Why? The officials may want the district to have a good showing at the state meet, so it would hurt the district to DQ one of the top relays even if there really was a false start. However, it is tough to quantify this bias in favor of SFS at the district meet.
This look of the district data should show that the large number of DQ's is most likely not fully explained by the coach or swimmers, as you would expect to see similar DQ patterns at the district meet. This three-part series conclusively shows that it is extremely unlikely that there is no referee bias against the SFS swim team at the state meet.
Solid analysis; I'm sold. It would also be cool to do a Hierarchical Bayesian model to estimate each team's DQ rate, for both the district and state meets. Under the null, the posterior distributions should be about the same for each team. My guess would be that this would hold up for the other perennial contenders, but that, even with a pooling of the data, the posteriors for the SFS team would be significantly different... Good read, thanks for sharing!
ReplyDeleteGood suggestion Matt. I'm not going to follow up with that analysis for 2 reasons:
ReplyDelete1. Hierarchical Bayesian modelling is more complicated than I want to get on this blog.
2. I'm not a Bayesian and my brain shuts off whenever I hear that B-word.
But I agree that it should probably lead to very different posteriors.