Jun 20, 1996 [ANALYSIS] Tiebreakers: Head-to-Head vs Goal Difference
(Alan Douglas, Marcelo Weinberger, Karel Stokkermans, et al)
Jun 25, 1996 [Analysis] Goal Difference vs Time of Possession (Jeff Ely)
================================================================
Subject: [ANALYSIS] Tiebreakers: Head-to-Head vs Goal Difference
From: Alan_Douglas@mindlink.bc.ca (Alan Douglas)
Date: June 20, 1996
To some extent whether or not head-to-head results makes for a
better tiebreaker than overall goal difference is really
subjective. Both are used to pick the team that *performed*
better, but each using different criteria for judging a team's
performance. Which criteria is preferable is purely a matter of
opinion, and I won't be drawn into that argument one way or the
other.
However one can assess the tiebreakers in terms of how well they
do in picking the *better* team. This may be considered unfair
though, as tournaments do not necessarily reward teams that *are*
better, but rather those that happened to *play* better.
However, tournaments exist for the purpose of determining the
"best" team, so it makes sense that one should want to use a
tiebreaker that does tend to favour the team that really is
better.
The problem is that in real life it's not always possible to know
for certain which team is better. It is possible though to run
computer simulations using a model in which the relative
strengths of the teams are known, and then compare the two
tiebreakers to see how often they favoured the team that really
is better. This is what I've done.
I used a fairly simple model for simulating game scores between
teams of varying strengths (well, simple for those with a
mathematics background, incomprehensible nonsense otherwise). I
set it up like group play from Euro96, with four teams each of
clearly different strengths. For simplicity sake, I only
considered two-way ties, as three-way tiebreaking is tricky to
compare (and in four-way ties there's no difference between the
two tiebreakers anyway). I considered all two-way ties, not just
those for 1st or 2nd place.
I ran the model until I reached 10000 ties, and did this several
times using different distributions of team strengths. In all
cases, the Goal Difference tiebreaker favoured the better team
far more often than did Head-to-Head. Typical results are:
Of the 10000 ties:
Head-to-Head broke the tie 43% of the time
Goal Difference broke the tie 73% of the time
When they did break the tie:
Head-to-Head favoured the better team 48% of the time
Goal Difference favoured the better team 63% of the time
When they both broke the same tie, but in different ways:
Head-to-Head favoured the better team 34% of the time
Goal Difference favoured the better team 66% of the time
Goal Difference clearly wins out here. And in runs in which I
increased the difference in strength between the teams, Goal
Difference performed even better, while Head-to-Head did even
worse. In runs in which the teams were more closely matched, the
two tiebreakers performed similarly, but Goal Difference still
had a slight edge.
On the basis of these simulations, Goal Difference clearly
favoured the better team, while Head-to-Head actually slightly
favoured the weaker team. IMHO, the Italians have good reason to
complain.
However, if people wish to believe that Head-to-Head is a better
way of comparing how two teams *played*, then there's really no
arguing with it. That's a matter of opinion. What I've shown is
that Goal Difference is a better tiebreaker if you want to favour
the teams that really *are* better, but not necessarily those
that *played* better.
====================================================================
Subject: Re: [ANALYSIS] Tiebreakers: Head-to-Head vs Goal Difference
From: kalivoda@sb.dcs.uga.edu (Ted Kalivoda)
Date: June 21, 1996
steinw@itk.unit.no (Stein Ommund Wasboe) wrote:
> Thank you, for a *very* interesting and enlightening article.
> I agree with you 100%, and IMHO GD should always be used before
> head-to-head as a tiebreaker. You should send this to UEFA!
Don't be so quick. What was the "criteria" for "better" team? The point of
head to head is that during that game the winning team was the better
team.
Head to head solves ridiculous advancements like Japan and Korea in WC 94.
The below is not exact but is close to what happened:
W D L GF GA
Japan 1 1 1 6 3
Korea 1 1 1 7 3
Japan defeated Korea. But Korea advances even though Japan took them down?
That is stupid, and I can't imagine the sense of futbol fans thinking
otherwise. That's why FIFA changed the rules.
Actually though, the Penalty Kick rule is the most horrid rule, so
complain about that instead.
--------------------------------------------------------------------
Subject: Re: [ANALYSIS] Tiebreakers: Head-to-Head vs Goal Difference
From: Alan_Douglas@mindlink.bc.ca (Alan Douglas)
Date: June 21, 1996
> Don't be so quick. What was the "criteria" for "better" team?
I imagine that was rhetorical, but to just to reiterate... in the
simulation Team A was known as better than Team B because that's
how the model was created. This means that if the two teams were made
to play against each other repeatedly, then (in long run) Team A would
win the bulk of the games.
> The point of head to head is that during that game the winning
> team was the better team.
Yes, but the team with the better goal difference did better over
the course of *three* games, not just one. In fact when
comparing two teams on goal difference, the results of *five*
games are being considered. Anything can happen in just one
game. The more games you consider, the better indicator you get
as to which team really is stronger.
> Head to head solves ridiculous advancements like Japan and
> Korea in WC 94. The below is not exact but is close to what
> happened:
[example deleted]
Ok, let's try a hypothetical situation with 4 teams called GREAT,
LUCKY, BAD, and CRAP in group play with the top team advancing.
GREAT slaughters BAD and CRAP 4-0 each. LUCKY manages a 1-0 win
against CRAP, but runs out of luck against BAD and loses 3-0.
In the big showdown against LUCKY, GREAT totally dominates but
can't score. LUCKY poaches a fluke goal at the other end and
steals the win 1-0. The top two teams finish:
W D L GF GA Pts
GREAT 2 0 1 8 1 6
LUCKY 2 0 1 2 3 6
Head-to-head would have LUCKY advance. Goal difference would
pick GREAT. The point is that a weak team can often pull off the
upset in a *single* game. However it's much much harder for a
weak team to pull off a string of high scoring upsets and end up
with a good goal difference.
Both goal difference and head-to-head judge teams based on their
performance. But goal difference is the much better criteria for
assessing which team really is stronger, precisely because it
looks at more games. And if determining the strongest team based
on their performance isn't the point of a tournament like Euro96,
then what is?
================================================
From: marcelo@apollo.HP.COM (Marcelo Weinberger)
Subject: Re: [ANALYSIS] Tiebreakers: Head-to-Head vs Goal Difference
Date: June 21, 1996
Alan: your model was built in a way that actually guarrantees the
outcome of your simulation! In fact, even though I don't know exactly
what your model is, my intuition of information theorist tells me that
with a little more mathematical thinking, and less readiness to rely on
simulation, you could actually predict the outcome without performing
the simulation.
The fluke of your reasoning is that, from your description, I can guess
that your model is such that the "goodness" of a team possesses the
transitivity property, namely, if A is better than B and B is better
than C, then A is better than C. Without too much thinking, I can
conjecture that such property is sufficient to postulate the advantage
of "goal difference." But the whole issue is that football does not
necessarily have this transitive property! Also, you may have a problem
with your metric, and your example of GREAT, LUCKY, BAD, and CRAP, is a
clear indication of what I'm saying. Some teams are great when they play
crap teams, but they don't have what it takes to beat tough teams. On
the other hand, other teams will score against crap teams only what they
need for a win, but then against tough teams they still are able to
perform at the same level.
In other words, you are already assuming what you want to prove.
Cheers from an information theorist who loves soccer even more than
information theory.
---------------------------------------------------------------------
Subject: Re: [ANALYSIS] Tiebreakers: Head-to-Head vs Goal Difference
From: Alan_Douglas@mindlink.bc.ca (Alan Douglas)
Date: June 22, 1996
marcelo@apollo.HP.COM (Marcelo Weinberger) writes:
> Alan: your model was built in a way that actually guarrantees
> the outcome of your simulation!
I selected my model on the basis of six years experience modelling and
forecasting the outcome of sporting events. I consider it to be a
simplistic, but perfectly adequate simulation of reality. I was
actually hoping my simulation would justify head-to-head just to shut
up the Italians.
> [...]you could actually predict the outcome without performing the
> simulation.
Perhaps. I could also write up a formal proof if I worked at it. But
the simulation remains a far more effective demonstration for those
who are less comfortable with mathematics.
> The fluke of your reasoning is that, from your description, I
> can guess that your model is such that the "goodness" of a team
> possesses the transitivity property
Yes, I did (and generally do) assume transitivity. I would argue that
it's an essential first order approximation in sports modelling. I
have yet to see any convincing real-world evidence of the need for any
sort of model which exhibits non-transitive behaviour when all factors
external to the teams are held constant. If such subtleties exist,
then they tend to be well masked by random variability.
> But the whole issue is that football does not necessarily have
> this transitive property!
Very well. I can run another simulation using a similar model, but
for which transitivity is not assumed. My original system used a
systematic method based on each team's intrinsic "strength" to
calculate the expected numbers of goals scored and then used a pair of
independent Poisson distributions for the actual number of goals.
Instead I could scrap the intrinsic strengths and assign the expected
number of goals for all six matches randomly. For each game I could
then calculate the probability of a win, loss, or draw and then find
the expected number of points to be earned by each of the four teams
in that trial. Expected points could then be used to rank the teams
from best to worst. This entire process would be repeated for each of
the thousands of iterations in a run.
Under this model then, Team A being "better" than Team B (for that
group of 4 teams) would mean that Team A would average (in the long
run) more points in that group than Team B if all the group games were
constantly repeated.
Now would this model be acceptable?
It would, I believe, be an example of the far end of the transitivity
spectrum -- total chaos compared to my first simulation of total
order. Reality is probably somewhere in between.
One could examine this more closely by looking at the variation in the
distribution of the points of the teams in the group. My earlier
transitive model should have group point standings tending towards the
extreme:
Pts
Team A 9
Team B 6
Team C 3
Team D 0
The non-transitive one should have less spread, and tend towards the
opposite extreme 4,4,4,4. At this (very) late hour I can't think of a
reasonable way to characterise these "distributions of distributions"
so that a simple comparison with real-world group tables could be
made. Anyone have any ideas? Anyone still reading this?
Didn't think so. :-)
----------------------------------------------------
From: marcelo@hplgser.hpl.hp.com (Marcelo Weinberger)
Subject: Re: [ANALYSIS] Tiebreakers: Head-to-Head vs Goal Difference
Date: June 22, 1996
> If such subtleties exist, then they tend to be well masked by random
> variability.
Of course these subtleties exist in soccer! I gave a typical example in
my previous post. Look also at the all-time statistics of games between
the strongest soccer nations - Argentina, Brasil, Germany, and Italy.
I really don't think it is possible to come up with a sensible metric of
"goodness" in soccer, and if that metric exists, then you shouldn't be
able to derive an order relation on it. Now, you can insist and impose
transitivity, but then, I conjecture that transitivity itself ensures
that "goal difference" is preferable to "head-to-head."
Therefore, my whole point is: don't make simulations. Just tell us: "I
think that "goodness" in soccer has the transitivity property and, for
that reason, I prefer "goal difference." But I reckon that this is just
a conjecture. I didn't put any thinking on a formal proof.
> Now would this model be acceptable?
Not at all, precisely for the reasons you state below.
> It would, I believe, be an example of the far end of the transitivity
> spectrum -- total chaos compared to my first simulation of total
> order. Reality is probably somewhere in between.
You could assign different kinds of strengths, and assume transitivity
for each strength. This seems reasonable. But you cannot give fixed
weights to these strengths, because otherwise you would get an overall
score that yields an order relation. Instead, the weight of each
strength should be a function of the teams actually playing. In an
extreme case, say, having a phenomenal goalie won't help you much
against a team that plays without forwards; or having big defenders with
excellent performance on air-balls might be crucial if you play England,
but won't help you if you play, say, Portugal, to take teams from Euro
'96. The key issue is to weight a given strength differently, depending
on the teams that are playing. You could even do it adaptively, by
letting the model "learn" the weights from past experience, but I guess
that at this point I stopped being a futbol fan and remained with the
information theorist...
======================================================
From: kstokker@risc.uni-linz.ac.at (Karel Stokkermans)
Subject: Re: [ANALYSIS] Tiebreakers: Head-to-Head vs Goal Difference
Date: June 21, 1996
First of all, the premiss that one is looking for which team *is* better
is clearly false. The point of competition is not being better (how to
measure that, anyway) but playing better (and even that doesn't guarantee
winning on the pitch, but never mind). Between teams that finish equal
on points, that is obviously hard to call.
It really is only a matter of taste. Many countries use head-to-head
comparison rather than goal difference (or, in the past, goal average).
The main advantage of head-to-head is that it eliminates freak or even
bought results against weak teams or teams that have nothing left to
play for. (This is more a concern in a long league season than in a
four team group, sure, but it's still valid - cf. Croatia's lacklustre
performance once qualified.)
My main gripe with the head-to-head system is that it still allows teams
to calculate a favourable result for both in the last round even if a
third team which has chances to achieve equality on points has a last
match to play (this doesn't apply so much when calculating goal difference
since then team 3 can take things in its own hand by scoring enough goals).
Moreover, it's silly to complain after the fact. Had both Italy and the
Czechs won their last match, the Italians would have profited from the
rule even if the Czechs had won 4-0 or so (a score they could and should
have had at halftime against Russia).
The best solution would be a playoff, but given that this is not possible
in current tournament schedules, I'd prefer goal difference too in such
small groups. In qualifying tournaments with up to two minnows in a group,
I'd go for head-to-head as a 10-0 against San Marino doesn't really say
much more than a 6-0 (for instance); moreover, it might prevent suspicious
12-1's in the last round.
================================================
From: Alan_Douglas@mindlink.bc.ca (Alan Douglas)
Subject: Re: [ANALYSIS] Tiebreakers: Head-to-Head vs Goal Difference
Date: June 21, 1996
tjandik@vms.cis.pitt.edu (Tom) writes:
> Supporters of goal difference in this NG seem to be particularly
> concerned with the article of some math guy who ran simulations
> showing that GD leads to more frequent advancing of "better" team.
Hey, according to my sig, I'm a "Soccer Guy". I must object to the
term "math guy" -- I prefer to be called a math geek :-)
> Apart from problems with simulations (having earned math degree
> myself, I am always quite skeptical when I see results like
> that - e.g. what exactly does a "better" team mean?)
For the purpose of the simulation, Team A was considered to be better
than Team B in the sense that if they played each other repeatedly,
then Team A would (in the long run), win the sizeable majority of the
games. In the simulation run I quoted, the best team would outscore
the worst by an average of about 1.5 to 0.7, while the two middle
teams against each other would average a score of 1.2 to 0.8 (in the
long run).
As for problems with simulations, I will concede that mine did not
take into account subtleties such as differences in lineups, fields,
playing conditions, emotions, and motivations. For example, it's been
pointed out that when Germany played Italy, all it needed was a draw,
while when it played the Czechs it was looking for (and got) the win.
However, I believe these are minor effects and that their exclusion
does not bias my simulation one way or the other. In the long run,
these intangibles would tend to cancel each other out as far as
impacting on goal difference is concerned. They are, after all, a
product of the order in which games are played, and that I believe is
determined randomly.
> To his credit, the author picked the correct reason why head-
> to-head performance is the best tiebreaker himself - actual
> *performance*, not any imaginatory *team potential* - that's
> what counts in any sport!
I said that it was a matter of opinion as to what constituted a better
performance. Upon further reflection I have come to the conclusion
that:
1) Competition is designed to select the best team based on
performance.
2) Goal difference is a superior method of judging performance
in order to select the best team.
3) Head-to-head is an inferior method because it only considers
the result of one game, while goal difference (in the case
of Italy-Czech) looks at five games.
> Head to head performance is the core of 99% sport comparisons.
The Champions of most European Leagues are determined by the results
of their play over the whole season, not just one game. The Euro96
group play is just that, *group* play. It's play against the group
that should be important. A single game is just too small a sample.
> Tournaments (and yes, Europe is still *a* tournament, no matter
> how important) enable us to compare how teams are capable to
> convert skills into *real performance*, and thus, once again,
> results in group C show one thing - in June96, Italy did not
> perform as good as Czechs.
In that one game, yes they did. Over the five games that the two
teams played though, Italy did better. Ask yourself: if the Czechs
are so much better than Italy because they beat them, then how come
the two teams ended up level on points with Italy having the higher
goal difference? That single win taken by itself gave the Czechs a
three point advantage and a goal difference that was 2 goals better
than Italy's (+1 to -1)! So what happened? I'll tell ya... Italy had
a far better *real performance* in their other two games than did the
Czechs. Over the five games, Italy was better.
(BTW, just so you know I'm also unbiased... I was very happy to see
Italy knocked out. I was cheering for the Czechs.)
---------------------------------------------
From: mazzare@primenet.com (Ariel Mazzarelli)
Subject: Re: [ANALYSIS] Tiebreakers: Head-to-Head vs Goal Difference
Date: June 21, 1996
So, in your simulation, "better team" meant "team whose goal differential was
greater".
Hence, the conclusion that
> 2) Goal difference is a superior method of judging performance
> in order to select the best team.
is straightforward. There was no need to do everything else that you did.
You're not the guy in charge of the COCACOLA ranking are you? There is a
familiar scent to the methodologies.
------------------------------------------------
From: Alan_Douglas@mindlink.bc.ca (Alan Douglas)
Subject: Re: [ANALYSIS] Tiebreakers: Head-to-Head vs Goal Difference
Date: June 22, 1996
One of us is missing the point here Ariel, and given your reputation
in this group, I'm not about to assume that it's you. So allow me to
just clarify my argument in light of your observation, and see on
which side of internet the penny drops.
Under my model, a stronger team is more likely to have a better goal
difference than a weaker team. However, a stronger team is *also*
more likely to defeat a weaker team head-to-head. The question I was
trying to answer was which is the greater correlation, and hence which
tiebreaker gives a better indication of the stronger team.
The answer my simulation gave is that goal difference was the better
indicator. I do not see that this was a needless exercise or a
self-fulfilling prophesy as you seem to suggest.
> You're not the guy in charge of the COCACOLA ranking are you?
No. That was a couple of Swiss (alleged) mathematicians who should be
taken out into a field and made to more closely resemble a piece of
their country's cheese. I used to believe that it might be possible
to create a proper mathematically sound ranking system for FIFA that
could be accepted by the masses. But that was before I got into this
tiebreaker thread, back when I was still a young and idealistic
dreamer.
---------------------------------------------
From: mazzare@primenet.com (Ariel Mazzarelli)
Subject: Re: [ANALYSIS] Tiebreakers: Head-to-Head vs Goal Difference
Date: June 23, 1996
Given such a proper and respectful response, I think the best thing for us is
to agree to disagree.
>No. That was a couple of Swiss (alleged) mathematicians
My investigation into the matter in 1993 revealed that it was a couple of
Swiss bartenders who did all sorts of favors for Sepp.
=========================================================
Subject: [Analysis] Goal Difference vs Time of Possession
From: Jeff Ely
Date: June 25, 1996
The debate over head-to-head tiebreakers vs. goal difference
is very interesting and obviously hotly contested. Yet I
would suggest that there is a much more fundamental debate
to be settled. What is the best way to determine the winner
of a single match?
Since the game's inception, the winner of a match has been
the team which scores the most goals. Is this the
right way to decide it? Many would argue that there are much
better ways to determine the better team. For example, although
in their qualifying round match Italy scored fewer goals than
the Czech republic, many feel that Italy is the better team.
I ran some simulations to determine whether time of possession,
a highly underrated statistic, is a better indicator of the
true strength of a team than goal difference. You may think
this is unfair since the winner of the game is not necessarily
supposed to be the "better" team, just the team playing better
that given day. But in keeping with the spirit embodied in
that age old saying "may the best team win," it might be
worth considering time of possession as the criterion for determining
the winner, if in fact it performs better as an indicator of
true strength.
I used a simple model to simulate scores and times of possessions
between teams of varying objective strengths. (well, simple for
someone with a mathematics training, an utter waste of time
otherwise.)
I ran the model 11,328 times using varying distributions of
team strengths, and also included a variable for the weather
and nationality of referree. Typical results
criterion how often it picked the better team
Goal difference 65%
Time of possession 85%
Time of possession clearly wins out here. Interestingly, I ran
another simulation in which I shortened the game to 60 minutes.
In that simulation goal difference slightly outperformed time
of possession as the better team had less of an opportunity to
gain an advantage in possession time. This suggests that if FIFA
is wedded to goal difference in determining the winner of a match,
it should consider shortening the game to improve its accuracy.
IMHO the Italians have good reason to complain.
Cheers,
Jeff Ely
Statistics guy