Sunday, September 15, 2013

Why women should not challenge more in tennis

In last week's Sports Illustrated, Jon Wertheim argues that women should use the challenge system more at Grand Slam tennis tournaments.  Although he includes data in his article to make his case, I'm going to argue that he is misinterpreting the data and show that convincing women to challenge more often might actually hurt the game.  Unfortunately, this article is not available online, so I will begin with a summary and include some direct quotes (*Update - the article is available here).

A little background: "In 2006 tennis instituted a replay challenge system not unlike the NFL's.  Provided the court is equipped with the technology, players can appeal line calls for review."  The rules are the same for men and women; however, men and women have different challenge behavior.  "Men challenge their points 25% more often than women - though their success rates are virtually the same."  For example, at Wimbledon this year, women challenged 2.6% of the points played, while men challenged 3.3% of the time. This trend holds over all Grand Slam tournaments over the past year.  Men won their challenges 27.73% of the time while women won 27.37% of challenges.

Supplied with this data, the author concludes that "men are more prone to question their authority" and "women are more reluctant to challenge and be assertive or confrontational".  Many assume that men are more likely to challenge because they hit the ball harder, so linespersons are more likely to make a mistake.  The author's response: "Sounds logical.  But if this were true - if it were harder for linespersons to trace 140-mph serves, as opposed to 120-mph serves - we would expect to see a disparity in accuracy of line calls".  But because the challenge accuracy is equivalent between gender, the author concludes that women challenge less (and thus accept incorrect calls) because "women are uncomfortable with confrontation and negotiation".  The article concludes with a quote from Martina Navratilova, "Women need to be more comfortable challenging.  Here's one area where there's no reason we shouldn't be like the men."

The author's whole argument hinges on one point: there is no disparity in the accuracy of line calls for men's and women's matches.  I agree that the author's conclusions would hold if the data could show this.  However, we have no way of knowing the accuracy of all linepersons calls from the data - we only know that the accuracy of player challenges is roughly equivalent between genders (27%).  What we cannot know is how many of the unchallenged points contained an incorrect call by a linesperson.  The only way we could figure this out is if someone watching the match (presumably in the TV booth) "challenges" every point to determine how many points contained incorrect calls, then look at how many of these points were challenged by players.  This would allow us to construct the following table for each gender:

# Points containing
all correct line calls
# Points containing
an incorrect call
# Points
# Points With
No Challenge

From the data presented, we know the values for A and B.  With the Wimbledon 2013 men's data, A = 2.38% of all men's points (3.3% of challenged points x 72.27% of incorrect chalenges) and B = 0.92% of all men's points (3.3% of challenged points x 27.73% of overturned points).  For women, A =  1.89% and B = 0.71% of women's points.

To determine whether or not linespersons make more incorrect calls in men's matches, we need to know D (the total number of incorrect calls is B + D).  Without this knowledge, we cannot determine if more challenges from women would result in more overturned calls.  However, we know that the value of B is larger for men than women by 0.21% of all points played.  For the author's assertion that linespersons are equally as likely to be incorrect for women as for me, D would need to be larger for women than men by 0.21% of all points played.

Let's look at 3 examples using the data from Wimbledon 2013:

  1. Assume women got every single wrong call overturned (D = 0).  Therefore, using additional challenges will not get any call overturned.  If women challenge every as frequently as men without any additional incorrect calls, then women's accuracy would drop to 21.56%.  So urging women to challenge more would make them appear to have worse judgement than men (not a good thing when trying to argue for gender equality).
  2. Assume the proportion of incorrect men's and women's calls are equivalent (D is equal for men and women).  If this is the case, then you could argue that men and women should both be challenging more.  However, because B+D is still smaller for women than men, their accuracy would decrease if they challenged the same number of times as men.
  3. Assume the total number of points containing an incorrect call is equivalent between genders (D is larger by 0.21% of all points for women than men, resulting in B+D being equivalent between men and women).  If this is true, as the author assumes, then his conclusion than women need to challenge more is correct.  However, assuming that women challenge the same number of times as men, their accuracy on the "new" challenges would actually increase from 27.37% to 29.08% if they were to remain as successful as men at getting line calls overturned.  This means that women are currently challenging calls that are less likely to be overturned than the calls that they are are missing (try explaining this to the women players!).

Therefore, we cannot make any assumptions as to whether linespersons miss the same number of calls for women as men.  This also means that we cannot determine whether women can challenge more and still be as accurate as men in getting calls overturned.  So what conclusions can be made from this data?

  • Men and women are equally successful at challenging.  This means that one sex does not have better "eyes" than the other.
  • Men and women are equally bad at challenging.  On average, less than 1 out of 3 challenges will result in a call being overturned.  This number is likely due to "throw away" challenges when a player knows the call was correct but has challenges to waste at the end of a set (or needs a longer break to catch his/her breath after a long point).
  • Assuming that men and women are getting every incorrect call overturned (D=0), linespersons are 29.6% more likely to make a mistake on a men's point than a women's point (0.92% vs 0.71%).