Winter Olympics Medal Predictions Gone Wrong

I recently came across an article about an analytics company, MicroStrategy, that used their dashboard to predict the number of medals that each country will win at the 2014 Winter Olympics. They predicted that Canada will win the most medals (35), followed by Germany (31), with the US in 7th place (16 medals). They explain the results in their blog post. I think the company was just trying to do a simple, light-hearted analysis to show off the types of analyses that their product can do. Unfortunately for the company, there are several flaws in their model/analysis.  Two minor flaws have to do with the data itself:

• In their post, they state "Keep in mind that the algorithm is based on historical data, and doesn’t necessarily reflect more current information such as emerging stars, recent funding boosts, and an unexpectedly large addition of new events to the program."
• On a related note, they fail to explain how they handled the dissolution of old countries to form new countries. For example, are the medals from past Olympics won by the USSR now contributed to Russia? What about newer countries, such as Ukraine, that used to be a part of USSR?

The biggest problem is over-fitting of the model. Over-fitting occurs when you design a complex model using many variables that is more likely to describe random noise than the true signal. Because the model is describing the noise and not true signal of the historical data, it often results in inaccurate predictions (in this case, inaccurate medal predictions). If you are interested in performing predictions based on a model, one simple way to determine if a model is over-fit is by leaving out a subset of the data, fitting a model, then using that model to predict the data that you left out. Since you know the "truth" of these predicted data, you can assess the prediction accuracy. For example, MicroStrategy could have used all data prior to 2010, fit the model using that data, then predicted medal counts for 2010. We would be able to determine the accuracy by comparing to the true 2010 medal counts. 

The 2014 Winter Olympics are only half way over, but let's compare their predictions to current medal counts for selected countries:

	Predicted Medal Count	Actual Medal Count (as of 2/16)
Canada	35	14
Germany	31	12
Russia	18	16
USA	16	16
Netherlands	7	17

While it is possible that Canada and Germany can still reach their predicted medal counts, it looks like Russia, USA, and the Netherlands will all greatly surpass their predicted medal count.

Again, I realize that the company was doing this for fun and to generate some press. I just hope that customers don't look at this analysis, see how bad these predictions are, and ultimately decide to not buy the product. This goes to show how a little statistical knowledge can go a long way!

Swimming and the Fast Suit Aftermath

At the end of 2009, the governing body of competitive swimming, FINA, banned the use of high-tech full-body fast suits (try saying that 5 times!).  See here for a summary.  As proof of the influence of fast suits, all but 2 world records (both men's and women's) were broken in either 2008 or 2009!  The general consensus was that these world records would be untouchable for a long time, and for the most part, that has been true.  However, 8 world records were broken at the 2012 Olympics.  I am now going to answer the question, "Were the winning Olympic times significantly slower than the world records?"

First, let's look at a box plot of the difference between the world record and the winning Olympic time (a value less than zero denotes that the world record was broken).  Note: 2 men's world records were set in 2011, so I am comparing to these current records rather than pre-2010 records.

While the times are generally above zero (slower than WR time), the boxplot whiskers do extend below zero.  There is one clear outlier for the men, and this ocurred when the 1500m free WR was broken by 3 seconds.  Most of the variation is due to events being different distances (50, 100, 200, 400, 800 and 1500m).  To account for this difference, I have normalized all times to 100m (multiply 50m time differences by 2, divide 200m time differences by 2, etc.).  The normalized times are reported in the following box plots.  Now the times are much less variable and there are no clear outliers.

To officially answer our question of whether times were significantly slower without fast suits, I performed a t-test for mean difference.  Our null hypothesis is:

Ho: no difference between average world record time and winning Olympic time.

Leaving out the details, we obtain p-values of 0.18 for the men and 0.26 for the women.  Thus, since these p-values are large (> typical cutoff of 0.05), we fail to reject the null hypothesis.  We can conclude that there is no significant evidence that the winning swimming times in the 2012 Olympics were significantly slower than the world records.  I also repeated the calculations after removing the 3 relays from the analysis and arrived at the same conclusion.

We cannot tell from this analysis if the fast suits has a smaller influence on time decreases as originally thought, or if swimmers are just training harder and getting stronger (I tend to believe the latter).  It's also too early to tell if any of these records will be thought of as unbreakable (example: Phelp's 2008 Olympic performance + fast suit = some really fast world records).  But, I think we can safely conclude that, unlike the steroid era in baseball, world records set in the fast suit era will not require an asterisk.

How Usain Bolt can rival Michael Phelps

OK, so this post isn't exactly statistics related, but watching Olympic coverage talk comparing Usain Bolt to Michael Phelps is ridiculous.  First, let me say that without a doubt, Bolt is the fastest man alive.  His performances are the highlight of track and field Olympics.  But, winning back to back gold medals in the 100 and 200 is nowhere near Phelp's 22 medals (18 gold) over 3 Olympics.  Here are 3 ways, in my opinion, for Bolt to end his career on the same page as Phelps.

1. Compete in at least 4 Olympic games.  Phelps competed as a 15 year old at Sydney, swimming in the 200 fly.  Combined with Athens, Beijing and London, Phelps swam in 4 Olympics.  Bolt is only half way there with 2 Olympics.

2. Win both the 100 and 200 at Rio 2016.  Phelps became the first swimmer to (twice) win gold in the same event in 3 consecutive Olympics (100 fly, 200 IM), while just missing out on three-peating with the 200 fly.

3.  Add additional events.  Every commentator who says that Bolt does not have as many opportunities to race as Phelps should be fired on the spot.  Here are other reasonable events for him to race.

• 400 m: This is only running 2-200's in a row.
• 4 x 400 relay: This is the most reasonable race for him to add.  Phelps swims the 4x100 free relay (turning in the 2nd fastest split this Olympics), yet he has never swum the 100 free as an individual event.  Bolt doesn't need to be the fastest 400 runner to win a medal, but be part of the fastest team.
• 110 m hurdles: Yes, this involves hurdles, but he's tall enough to make it over the hurdles and would definitely be the fastest pure runner in the race.
• Long jump: Jesse Owens won 4 gold medals in the 1936 Olympics including the 100, 200, 4x100 and long jump.  Bolt has never had a single Olympics as successful as Owen's performance.
• High jump and triple jump: see above.

I just listed 6 additional events for Bolt to possibly compete in.  Yes, many of the events would take him out of his comfort zone, but winning these off events is what distinguishes legends from greats.  If he added the 4x400 relay with another individual event and won golds in those events, I would then start to think of Bolt as competing on equal footing with Phelps.

And please don't argue that the schedule wouldn't work.  Phelps (and Lochte, Franklin, etc.) won gold medals within an hour of swimming in another final or semi-final.  I've yet to see a top Olympic track athlete push themselves and race multiple finals/semi races in the same day.  So Bolt could be innovative in this manner too.

Finally, Ranomi Kromowidjojo is a female swimmer from the Netherlands won gold in both the 50 and 100 free and silver in the 4x100 free relay this Olympics.  If she wins gold in all 3 events next Olympics and sets a few records in the process, will she be considered the greatest female swimmer ever? NO.  But isn't her event schedule comparable to Bolt's (minus the relay)? YES! 

Why Roger Federer (and the entire women's tour) should never hit a second serve

In the 2012 Wimbledon Men's Championship match, Roger Federer beat Andy Murray to win his record tying 7th Wimbledon title and record 17th major overall.  But it was a rough start for Federer, as he lost the first set, which included losing his first 7 second serve points (he won 3/11 overall in the first set).  Federer was having a much easier time with his first serve (as do most players), winning 65% of the points when he made his first serve.  This got me thinking ... does it ever make strategical sense for a player to never hit a slower second serve but always hit a first serve?  The answer may surprise you (although I guess I give it away in the title).

I went through several matches for both men and women at this year's Wimbledon.  Let's use the men's final as an example.  Over the whole match, Federer made his first serve 68.7% of the time, winning 75.6% of those first serve points.  Multiplying these two numbers together, we see that 51.9% of time time Federer made his first serve and won the point (conversely, 48.1% of the time he either made his first serve and lost the point or had to hit a second serve).  Federer won 48.8% of the points when he hit a second serve.  This shows that Federer would have been better off going for his first serve even when hitting a second serve, as he would be expected to win an additional 3.1% (51.9 - 48.8) of second serve points. Federer hit 41 second serves, so he would expect to win an additional (41)*(3.1%) = 1.3 points in the match had he not hit his regular second serve.  This doesn't sound like much, especially since Federer won the match, but it could translate to winning an additional game.

One drawback of only hitting first serves is an increased number of double faults.  Because Federer made 68.7% of his first serves, we would expect him to miss his first serve 31.3% of the time.  If we consider two serves as independent events, the probability that he would miss two in a row and double fault is (31.3%)(31.3%) = 9.8%, resulting in an expected 12.8 double faults (he served 131 points in the match), much higher than the 3 double faults he actually served.  But he would have also been expected to serve an additional 3.75 aces (work not shown, but trust me).  So this turn out to a net decrease of (12.8 expected double faults) - (3 actual df's) - (3.75 additional aces) = 6.05 points.  How can this make sense if I just said that we expect Federer to win a net of 1.3 additional points?  When he does make his first serve, he wins a much higher percentage of those points compared to second serve points (75.6% to 48.8%) that it more than makes up for the double faults.  In other words, the risk of only hitting first serves pays off for Roger Federer.

I decided to look at all men's matches starting with the 4th round of Wimbledon.  Of the 30 matches (or 60 players, some counted more than once), 10 of the players would have expected to benefit by only hitting first serves.  Roger Federer was the only player to show up twice on the list (Finals and QF matches).  Of these 10, 7 lost the match (Federer won both matches and Djokovic won his QF).  In 3 of Andy Murray's 4 matches that I looked at, his opponent would have been better off only hitting first serves (Federer, Tsonga and Cilic), showing how great of a returner Murray is (the other opponent was Ferrer, who doesn't have a big first serve).  The largest expected gain was for Tsonga, who could have expected to win an additional 7.7 points (2 games) against Murray in the semis.

I wanted to look a little more closely at how this strategy may impact the match.  My original thought was that a player probably loses a set when he is serving poorly, so only hitting first serves would result in a lot more double faults and a net loss of points.  I was surprised to see that this was not the case.  Looking at the QF through the Finals, players lost 26 sets.  In 10 of these 26 sets, the player who lost the set would have expected to win more points by serving only first serves.  This definitely supports players only hitting first serves when they lose the set, but I guess this doesn't help much retrospectively.

Looking at sets won, for 19 of 26 sets, the winner would have expected to lose more points by only hitting first serves.  However, only 10 of those 19 would be expected to lose more than 2 points.  Interestingly, Federer could have won more points in 3 of the 9 sets he won by only hitting first serves and never would have expected to lose an additional 2.5 points in the 9 sets that he won.  This shows how great his first serve was working during the Championships.

The players are back at Wimbledon for the next 2 weeks, but this time trying to win an Olympic medal.  My recommendations would be for Federer to only hit first serves, but he is probably the only male player I would recommend this to regardless of opponent.  Plus, this may have the added benefit of screwing up the opponent's game plan.  I would also suggest that all of Andy Murray's opponents to employ this strategy.  But as I said before, no coach will tell their player to use this strategy.  My guess is that this is due to aesthetic reasons - no coach wants to see his player double fault in the double digits even if he would win more points in the long run.

I also performed the same analysis for the women.  To keep things short, every women player should only hit first serves.  Of the final 7 matches (14 players) that I looked at, only 5 players would not have benefited by this strategy.  In fact, 4 of those 5 players who would not have benefited lost the match anyway (the only exception being Serena Williams in the SF).  On average, the women would expect to win an additional 1.1 points per match.  Compare this to the men, who on average would expect to lose an additional 2.5 points.

Will any players employ this strategy at the Olympics?  I'm guessing not.