Tuesday, August 9, 2016

How much does losing the first round of a tournament hurt your final result?

This is the second half of a two-part piece about arguments I’ve had with Shengwu Li. Part 1 gives a statistical argument for why in some states it is rational to vote.

Today’s question: can you sleep through the first round of a 9-round debate tournament, like the World or European championships, without hurting your final result? You might think this question is only interesting to debaters, but the mathematical way of answering it is potentially applicable to any tournament with some randomness in pairings in which the winners play the winners and the losers play the losers; this also happens in chess and lots of other competitions.

(Brief note for non-debaters: in each round of the debate world championships, you compete against three other teams, earn up to three points, want to accumulate as many points as possible. In each round, you’re paired against teams with roughly the same number of points, so if you win, you face better teams.)
The argument that the first round doesn’t matter is that, if you lose, you face worse teams in the next eight rounds, an advantage you can capitalize on. Perhaps this evens out.

A first glance at the data would imply this argument is wrong. In the three debate world championships from 2013 - 2015, teams who earned three points in their first round ended up with about six more points by the end of the tournament than teams who earned zero points in their first round. This seems pretty amazing, because you only get three points from winning your first round, and then you have to face better teams; how do you end up six points better? Perhaps winning your first round gives you a confidence boost that improves your performance?

This reasoning is wrong. Winning your first round doesn’t necessarily cause you to do better in later rounds; it’s just a sign you’re a better team. (Similarly, getting in an ambulance doesn’t cause you to die; it’s just a sign that you’re sick.) The teams who win their first rounds would’ve done better whether they won their first rounds or not.

If we want to figure out whether winning your first round causes you to do better overall, we need to control for the fact that teams that win their first rounds are better. In an ideal world, we’d just do an experiment: instead of running the first round, we’d divide teams into four random, equally-sized groups, give one group 3 points, one group 2 points, one group 1 point, and one group 0 points, run the rest of the tournament normally, and see whether the initial advantage ended up mattering. But we can’t run a real random experiment, so we need to find a random factor that affects who wins the first round. We call the random factor an instrumental variable. The math behind how exactly this works is beyond the scope of the post (here’s a less mathy reference, here’s a more mathy one), but the basic recipe is straightforward. If you want to know how cause X affects outcome Y, you need to find a random factor Z which affects X (and is uncorrelated with Y when controlling for X). Then there’s some math that lets you put those three ingredients together to figure out how X affects Y. That’s a lot of symbols, so here are some examples.
How does X...
Affect Y...
Random factor Z which affects X
Reference
Serving in the Vietnam War
Future earnings
Draft lottery number
Tea party protests
Election outcomes
Rain on tax day
Iron metabolism
Risk of Parkinson’s
Genetic variants affecting iron metabolism
Height and BMI
Socioeconomic status
Genetic variants affecting height and BMI


To return to our debate problem: our X is how well you do in round 1, our outcome Y is how well you do in the tournament overall, and now we need a random factor Z that affects how well you do in round 1. One random factor that affects a team’s probability of winning round 1 is how good the other teams in the round are, since teams are randomly matched in the first round. We estimate the likelihood that each team will win the round using a number of other factors, including their record in past tournaments, their school’s record in past tournaments, and their EFL / ESL status [1], and use these as our instruments. Then we use these instruments to estimate the true causal effect of winning round 1.  (Another source of randomness you could potentially use is which position you’re assigned to; we discuss this further here [2]).

Here are the results. The red line shows the estimated effect of your round one result on your score after each round (x-axis) using the naive, incorrect method we describe above; the black line shows the estimated effect using the more sophisticated instrumental variable method.
A couple interesting things here. First, the red line and the black line are very different; in particular, the naive approach (red line) suggests that winning round 1 has a much larger effect, and this gets bigger as the tournament progresses, which is wrong for the reasons discussed above. The more sophisticated analysis implies that while winning round 1 affects your performance for a couple rounds, by the end of the tournament (round 9) it doesn’t make much difference. (The 95% confidence interval on the estimate is unfortunately large [3], because the instruments aren’t that strong.)
Takeaways:

  1. Don’t freak out too much if you do badly in an early round in a long tournament like Worlds or Euros -- our estimates suggest it doesn’t affect your final performance very much, if at all. On the other hand, in shorter 4 - 6 round tournaments, losing an early round can have a larger effect.
  2. If early rounds have a smaller effect on your final performance, it might be better to run controversial debates in earlier rounds; that way if teams had to opt out, they could do so without taking as large a hit to their final result.
  3. Storing comprehensive data from all Worlds in a single centralized repository, with a standard format, would make these analyses easier and is worth doing.
  4. Naive correlational estimates of effects are often different than those estimated using causal methods. For the love of God, bear this in mind when reading popular social science coverage. Even if the authors claim to have “controlled for other factors”, this phrase does not work magic. It is very difficult to control completely for other factors.

Notes:
[1] This isn’t, of course, a perfect measure of team strength -- since Oxford A is usually composed of different people from year to year, and many teams do not appear in other Worlds.
[2] In each round, every team will debate the same statement -- for example, “We should ban abortions at all stages of pregnancy” -- but because there are four teams in the same room, two teams will be randomly assigned to argue for the statement, and two teams will be assigned to argue against the statement. It’s hard to come up with statements which are perfectly balanced, so it’s often better to be on one side or the other. We found that a team’s score after the first four rounds was significantly associated with the positions they had been randomly assigned in those rounds, so position in the first four rounds could be used as an instrument. You have to use the first four rounds, not just the first round, to satisfy the exclusion restriction, since most debate teams cycle through all four positions in the first four rounds.
[3] I think you should actually compute standard errors clustering by round, since results within round are highly correlated; when you do this, the errorbars are slightly larger but the conclusions are the same.


55 comments:

  1. As opposed to quickest-finger-first to fill this workshop, we've got determined to mild attendees. To register your interest, please rsvp for the waiting listing with a paragraph reaction to the questions "How do you technique regression testing on your enterprise? Question shared by Intelligentessays.com

    ReplyDelete
  2. Tiers of sleep and sleep cycles. Commonly sleepers bypass thru four stages: 1, 2, three, and rem (speedy eye motion) sleep. Those ranges progress cyclically from 1 thru rem then begin once more with degree 1. A complete sleep cycle takes an average of 90 to a hundred and ten mins, with each stage lasting between five to 15 mins. Info delivered by Do My Assignment for Me Cheap - All Assignment Help.

    ReplyDelete
  3. Looking great work dear. These tips may help me in the future. If you are looking for help with game theory assignment and meet the requirement before the deadline, then you are at the right place. Avail the Nursing Assignment Help service from GoAssignmentHelp Portal.

    ReplyDelete
  4. Schauffele shot the pleasant score by using two strokes, however, my sleeper pick, Hideki Matsuyama, shot a sixty-six and is in high function to make a circulate on Friday and Saturday.

    Source by Cheap Essay Writing Services | 6DollarEssay.

    ReplyDelete
  5. At our CheapEssay writing provider, that has been in the commercial enterprise of writing instructional works for students for decades, and know nicely all of the intricacies of this enterprise. You may make sure that whoever is assigned to jot down your particular assignment at our service, the results are going to be outstanding.

    ReplyDelete
  6. You could make sure that whoever is assigned to jot down your unique project at our service, the outcomes are going to be remarkable. In preference to fastest-finger-first to fill this workshop, we've got were given decided to mild attendees. Cheap Essay Writing

    ReplyDelete
  7. Call now 09336801280 call boy Play boy service provider all India ki all City mey services available full secret service provider koi Dhokha dhadhi nhi full maza Masti key saath pasaa kamye (50000 sey 90000 per month)call me 09336801280Call now 09336801280 call boy Play boy service provider all India ki all City mey services available full secret service provider koi Dhokha dhadhi nhi full maza Masti key saath pasaa kamye (50000 sey 90000 per month)call me 09336801280Call now 09336801280 call boy Play boy service provider all India ki all City mey services available full secret service provider koi Dhokha dhadhi nhi full maza Masti key saath pasaa kamye (50000 sey 90000 per month)call me 09336801280Call now 09336801280 call boy Play boy service provider all India ki all City mey services available full secret service provider koi Dhokha dhadhi nhi full maza Masti key saath pasaa kamye (50000 sey 90000 per month)call me 09336801280Call now 09336801280 call boy Play boy service provider all India ki all City mey services available full secret service provider koi Dhokha dhadhi nhi full maza Masti key saath pasaa kamye (50000 sey 90000 per month)call me 09336801280Call now 09336801280 call boy Play boy service provider all India ki all City mey services available full secret service provider koi Dhokha dhadhi nhi full maza Masti key saath pasaa kamye (50000 sey 90000 per month)call me 09336801280Call now 09336801280 call boy Play boy service provider all India ki all City mey services available full secret service provider koi Dhokha dhadhi nhi full maza Masti key saath pasaa kamye (50000 sey 90000 per month)call me 09336801280

    ReplyDelete
  8. If you professor ordered you to write argumentative essay on school uniforms you need to check out this blog post. I think that this kind of info could be useful

    ReplyDelete
  9. The present inquiry would you be able to rest through the first round of a 9-round discussion competition, similar to the World or European titles, without harming your conclusive outcome? You may think this inquiry is just fascinating to debaters, however the numerical method for noting it is possibly pertinent to any competition with some arbitrariness in pairings in which the victors play the champs and the failures play the washouts; this likewise occurs in chess and heaps of different rivalries.

    Best Content Writing Services

    ReplyDelete
  10. I'm looking senior project examples. I do not write the essay myself. I use additional information. I order an essay on any subject. I can save my time.

    ReplyDelete
  11. My Silver Service hopes that by giving all customers the best service possible, and My silver Service can encourage repeat business as well as referral to friends and family and good reviews.
    taxi melbourne baby seat

    ReplyDelete
  12. We do your online class as we offer quality online class help and we are best if you’re searching to take my online class for me or Pay To Take My Online Class For Me!

    ReplyDelete
  13. Our Academic papers may be done as a part of a class, in a program of your study. We design our academic writing service in literary analysis, research paper and dissertation.
    Essay

    ReplyDelete
  14. At last my enthusiasm lies in inventive articulation through composition and showing others how its done to help other people along their way of self-revelation.
    whiteboard animation video

    ReplyDelete
  15. For those who want Government Jobs Latest Update and news, this is the website to come. You can check here for the latest updates about govt job. On this website you can found All India Govt Jobs Employment News of Central and State Government Jobs, Govt Undertaking, Public Sector, Railway and Bank Jobs In India.
    Army Recruitment
    Railway Jobs
    Jobs in India
    Teaching Jobs
    Engineering Jobs
    Bank Jobs in India
    State Government Jobs

    ReplyDelete
  16. Do you want online exam help? All you have got to say is do my online exam for me. We provide the best take my online exam help.

    ReplyDelete
  17. Web Design Services in USA offers custom web design for home service companies. Our contractor websites generate more leads and sales, guaranteed!

    ReplyDelete
  18. Get Custom Dissertation Writing Services UK by professional dissertation writers. UK Diss Get discount on all your dissertation writing help orders today!

    ReplyDelete
  19. Premium Brand Designs is the best https://premiumbranddesigns.com/logo-design.php that serves you at cheap prices while maintaining top quality.

    ReplyDelete
  20. Don’t worry and just subscribe to our Best Logo Design Company in USA who is eager to help you in all the aspects of designing.

    ReplyDelete
  21. I want to testify on how I got cured from Herpes. I have been living with this disease for the past 11 months, i have done all i can to cure this disease but all my efforts proved abortive until i met an old friend of mine who told me about a herbal doctor called Dr Oniha, she told me that Dr Oniha have cure for all kinds of diseases, though i never believed in it because my believe was that there is no cure for Herpes disease. But I decided to give it a try when I contacted Dr Oniha, he told me he has a cure for herpes that he cured with herbal medicine. I order for the herbal medicine, which Dr Oniha sent to me through a courier service which i make use of and now behold the herpes is gone and i now have my life back,if you are out there living with this disease i will like you to also contact Dr Oniha and get this disease cured out of your body. I am a living testimony of Dr Oniha herbal cure. Thanks once again Dr Oniha for you are God sent. contact Dr oniha through his contact information.
    Email:driyaseherblahoe@gmail.com
    Whatsapp number: +2347057052206

    ReplyDelete
  22. Hire Someone To Take My Class? Reach out to take my online class for me. We have the best online class Helper to your Take Your Online class for my needs.

    ReplyDelete
  23. We assure you that you will get impressive scores in your course!

    ReplyDelete
  24. Need take my online class services? Reach out to Online Class Help Now. We have the best online class takers to help your Do My Course Online For Me needs.

    ReplyDelete
  25. Our online course experts will assist you topics in all areas like hire someone to take a test.

    ReplyDelete
  26. Your article is very useful, the content is great, I have read a lot of articles, but for your article, it left me a deep impression, thank you for sharing. for More Information Click Here:- HP Support Assistant

    ReplyDelete
  27. The reel n realtor well leading real estate company in south Florida. They have highly trained Real Estate Experts that help you to purchase and buy property.

    ReplyDelete
  28. ambbet เว็บไซต์ที่รวบรวมข่าวสารวงการเกมเดิมพันออนไลน์ เเชร์สูตรสล็อต ทรคิ เเบะเทคนิคในการทำเงินของเกเมดิมพันออนไลน์ สล็อตออนไลน์ไว้ในเว็บไซต์เดียว คลิกตรงนี้ สูตรสล็อตออนไลน์ แฮ็กสล็อต ใช้ได้ทุกเกม ใช้ได้ทุกเว็บไซต์ ให้อ่าน เเจกเเบบฟรีๆ ไม่มีค่าช้จ่ายใดๆ ทั้งสิ้น รับรองว่าสามารถทำเงินได้อย่างต่อเนื่อง เเละปังอย่างเเน่นอน !!

    ReplyDelete

  29. Emblix Academy – Digital marketing institute in KPHB, we address all major and minor aspects required for any student’s advancement in digital marketing. Clutch USA named our Digital Marketing Institute the best SEO firm.

    The future of digital marketing is promising and full of possibilities.

    As a result, skilled digital marketers who can keep up with the rising demand are in high order.
    In the Emblix Academy Digital marketing institute in KPHB, you will learn about all the major and minor modules of digital marketing, from Search engine marketing to Social Media Marketing and almost all Tools used for Digital Marketing.
    One stop place for all Digital Marketing courses! Emblix Academy is a Team of dedicated Professionals with 12years of experience in various Digital Platforms. We assure to provide the best Digital Marketing courses to enhance your Career.
    Certifications

    • Search Advertising
    • Display Advertising
    • Analytics Certification
    • Hubspot Certification
    • Bing Certification
    • Twitter Certification
    • Facebook Certification

    https://emblixacademy.com/

    ReplyDelete
  30. Do you need a Pest Control KL ? Mypestcontrol provides the most effective pest control available in KL. Our solutions are highly effective, kid and pet-friendly and completely eco-friendly.

    ReplyDelete
  31. If you are looking for Monmouth Car service that will relocate your car from one destination to another in a safe and secured way, then Welimos will help you to get the right car services in Monmouth for your car shifting.

    ReplyDelete
  32. We offer stylish nautical lamps,Pelican lamps,
    ,Turtle Lamp,Coastal Lamps,& Mermaid Lamps a wide variety of styles, colors, and shapes to suit your budget and needs.

    ReplyDelete
  33. Thank you for your post, I look for such article along time, today i find it finally. this post give me lots of advise it is very useful for me. nice i like it.
    안전놀이터

    ReplyDelete
  34. 메이저놀이터 If more people that write articles really concerned themselves with writing great content like you, more readers would be interested in their writings. Thank you for caring about your content.


    ReplyDelete
  35. 섯다족보



    It is said that in life you always get to learn and I am very happy to see your post, I think I will get to learn a lot from your post and I will take inspiration from your post to make my website and post more beautiful Will try

    ReplyDelete
  36. Great survey, I'm sure you're getting a great response.สล็อตวอเลท

    ReplyDelete
  37. Awesome and interesting article. Great things you've always shared with us. Thanks. Just continue composing this kind of post.บา คา ร่า วอ เลท

    ReplyDelete
  38. Your content is nothing short of brilliant in many ways. I think this is engaging and eye-opening material. Thank you so much for caring about your content and your readers.เว็บ ตรง

    ReplyDelete
  39. We offer product description services in USA. We help convey your product story.

    ReplyDelete
  40. Some appear to be executed on the screen of the written text of the content. Who did you do to see if you could deal with this?

    토토사이트링크
    토토사이트
    바카라사이트
    온라인카지노

    ReplyDelete
  41. Are you looking for Best Gutter Installation in Nashville. Gutter Solution provides a proven, tested and permanent solution to the damage caused by clogged gutters. With no holes, gaps, or large openings, Gutter Solution keeps out all types of debris including pine needles, shingle grit, seed pods, and more! Gutter Solution installs directly on your existing gutters to bring you permanent peace of mind and help prevent damages related to clogged gutters.

    ReplyDelete
  42. Finding Car Accident Lawyer in Canada? So VSR Law can assist you in recovering damages and ensure that your Motor Vehicle Accident claim is as strong as possible. Car Accident Lawyer in Cambridge

    ReplyDelete
  43. Losing the first round of a tournament can significantly impact a player's chances of winning the tournament. Below are three important pieces of information about how losing the first round can impact a player's chances of winning the tournament:

    Read more: Best Law Schools in Maryland – A Comprehensive Guide

    ReplyDelete
  44. If you're a student in the US, Australia, or the UK and looking for reliable online homework solutions, My Homework Help is here to help. Our platform provides 24/7 support to students in these regions and beyond.

    At My Homework Help, we understand that students face a myriad of challenges when it comes to completing their homework assignments. From time constraints to a lack of understanding of subject matter, these challenges can hinder your academic progress. That's why we offer a wide range of homework help services to cater to your specific needs.

    Our team of expert tutors is available 24/7 to provide you with the support you need, no matter what subject you need help with. Our tutors are knowledgeable, experienced, and passionate about helping students achieve their academic goals. They will work with you to ensure that you understand the subject matter and are able to complete your assignments with ease.

    ReplyDelete