Measured Performance Handicapping
The Three Continuums
There are many, many different ideas on Keel handicapping. Further, many of the proponents of a particular system have not considered nor are aware of alternate systems used successfully elsewhere.
There are two major schools of thought re handicapping with a few “half way” between the two extremes.
Boat Potential Performance Handicaps <<——————–>> Measured Competitor Performance Handicaps
A Boat’s Potential Performance Handicap is the handicap that reflects the performance that a boat should potentially be capable of, given a competent skipper and crew, and given that the boat is in top condition with good sails etc. This form of handicapping strives to make the “boats equal” and provide results as per one class racing where the results are primarily dependent on the sailor’s abilities.
Usually this type of handicapping relies on some form of boat measurement system. These measurements are then put into a formula to derive the potential handicap for the boat. This is usually a set handicap and not adjusted except for maybe an annual minor adjustment after the boat is re-measured.
Measured Competitor Performance is the performance of a boat including the normal crew on board. It is usually measured for a number of races and via some agreed maths a final “average” value is produced. Some areas rely on human judgement to provide the “average” while others rely strictly on a mathematical formula to provide it. This “average” is the handicap for the future race(s).
In all that follows the discussion centres around automatically updated handicaps based purely on mathematics and run on a computer. Further it is assumed that the handicap value is automatically re-assessed/updated after each valid race. (With human updating scenarios this is often done on a monthly or even yearly basis.)
And in what follows the handicaps will be treated as Time Correct Factors or “Time on Time” handicaps rather than “Time on Distance” handicaps although the same basic systems can be applied to either TOT or TOD.
While the discussion revolves around keel boats, the same principles apply equally to dinghies and trailables.
The aim of Measured Competitor Performance is usually to try and give all competitors within a fleet an equal chance of winning any race so that all maintain interest in the racing. That is to provide a handicap that endeavours to overcome the measured performance differences between all the competitors in the group and thus to endeavour to make the “equal”.
Within Measured Performance systems there are two major schools of thought. The formulas used by various clubs might look very different, but virtually all of these come down to one of the following or a blend somewhere along the continuum.
Running Average<<———————————–>> Exponential Average
With the Running Average concept, the performance for the last say 5 races is recorded and averaged to provide a new handicap for the next race. This “measured performance” is a scale factor that shows the relative performance of each competitor when compared to a reference time derived from each race.
The Exponential Average takes the competitor’s current handicap and adjusts it by either
- Some predetermined number of points where the number of points depends on the competitor’s placing. Those who came first, second, third etc have their handicap increased while those who fared poorly have their handicap reduced.
- OR by an amount that is proportional to their handicap corrected time when compared to a reference time derived from that race i.e. by their “measured performance” as immediately above.
Both these system usually employ several clamps or limits or other mechanisms to ensure that handicaps do not move too dramatically after just a few races.
Before we proceed we need to mention 6 observations that have become apparent after looking at many performance graphs from a number of Series from several keel boat clubs.
- Virtually all competitors’ performances vary from race to race. Some vary considerably while others vary just a little from race to race.
- These variations tend to oscillate around a mean (average) value. That mean value may remain stable or may its self slowly move up or down over a number of races. This gives rise to the belief amongst some folks that each competitor has an “permanent” handicap value that is all but fixed. While observation suggest that over the very long term most competitor’s will have an average handicap value, it none the less tends to meander up and down for race to race and season to season.
- The significant majority of these performances are within a window of +/-3% of the mean (average) of that competitor’s performances.
- It is very rare for a competitor’s consecutive performances to continue to move away from their average for more than 4 consecutive races. Almost inevitably their performances move back towards their mean after 4 or less races.
- In a 10 to 20 races Series many boats will have one race when they perform well above their average.
- In a 10 to 20 races Series quite a few competitors will have several races where they perform well below their average.
- For any race the winner is the competitor who performed most above their handicap. Those who performed at around their handicap are usually placed mid fleet in the results.
Exponential system tends to alter the handicaps after every race. But that alteration is very often just the race by race performance differences mentioned in point 1 above particularly for those boats that just naturally tend to perform quite differently from week to week. This is the major weakness of this system. If the “adjust because of the placing” method is used, then this methodology can provide handicaps that are a rather poor reflection of actual relative performances. For example, the winner may have won by just one second or by 10 minutes and yet “the rules say his handicap will be increased by 5 points”. This problem is further aggravated by observation number 5 above. Exponential averaging can provide usable handicaps but not necessarily handicaps that reflect a competitor’s “average” relative performance.
But Exponential averaging does have at least two advantages.
- As it alters each competitor’s handicap towards their most recent performance it is perceived as “sensible” by many sailors.
- Provide an appropriate initial handicap is given and adjustments are made by reference to the relative performances, then you can get reasonable handicaps and these can be adjusted after just one race rather than needing a history of several races as traditionally used in the running average system
So that leaves us with the running average system to generate handicaps.
Once again there are two schools of thought that fit on the two ends of a continuum.
Average Over Lots of Races <<———————————>> Average Over Just A Few Races
Averaging over Lots of Races provides a very stable handicap for each competitor but as it takes many races to react to any changes this can lead to a runaway winner if one competitor is rapidly improving.
Average of Few Races adjusts very quickly so you are less likely to have a runaway winner because constantly adjusting handicaps. But handicaps tend to fluctuate from week to week . It is also more open to “sand bagging” unless limits etc employed.
In either case the new handicap = Running Average of last ‘X’ BCHs. Where BCH is the “Back Calculated Handicap” often known as the” Race Time Correction Factor”. This is the handicap needed for each competitor to have had the same handicap Corrected times as all others in the group (e.g. that Division).
So how many races are optimum? Well sadly there is no perfect solution, but from observation number 4 above, it seems a good number is 4 or 5 races. In practice this seems to be a good compromise between handicaps that are very slow in reacting and those that fluctuate up and down rather too much.
What to Carry Forward???
Many Clubs divide their year into a Summer Series (or set of Series) and a Winter Series (or set of Series).
So a constantly asked question is “What do I carry forward into the next Series?”
Again this is answered differently by different clubs but here are some points to consider before answering that question at your club.
- Many competitors sail with a different intent during the winter series. Many use winter to train a new helms person etc. During winter many of the crew have gone off to watch the football, etc, etc. So the measured performance handicap that was valid at the end of the summer season may be rather different to the actual performances during winter. So some clubs run a winter handicap and a Summer handicap for each competitor and use the handicap from the previous winter or summer as the initial handicap when setting up a new series.
- The running average approach does reflect any improving or un-improving trend in performance, in calculating the next handicap. Consequently some clubs feel that they must take the data from say the last 4 races into the new series because this data is used in calculating the new handicap after race one (this assumes a running average of 5 races). If this series is continuing on directly after the previous series and the competitors are sailing with the same crew and intent, then this is a good idea as it reflects an ongoing performance trend for the competitor. But, if you are setting up the winter season after the summer season then the carry forward of actual race data may provide no advantage and in fact may be quite misleading for the reasons mentioned in the previous point. Likewise the carrying forward of the last 4 race from the previous winter series, offers no value at all as any trend that was occurring some 9 months ago will be totally irrelevant now.
So What to do???
- Start the winter series with the final handicap from last winter series. This handicap is the average performance of the last say 5 races of last winter.
- If you use TopYacht set to average over say 5 races then it will provide a quasi-exponential average for the first 4 races. This uses the initial handicap and ‘averages’ this with the successive BCHs, with each BCH providing 1/5 of the new average and the initial handicap providing the rest. Once 5 races have been run then it just uses the 5 BCHs.
- Some clubs use the facility in TopYacht to more quickly adjust handicaps for the first few races of a new series so that anyone who is now performing a long way from their previous handicap is quickly adjusted to a handicap that is more in tune with their current performance.
Choosing the Reference Time:
- For measured performance handicaps you must provide a reference time for each race. Historically this may have been the handicap Corrected time of the 5th boat. This was easy for everyone to work out but it will lead to a gradual altering/drifting of all the handicaps in this group unless the group size happens to be around 12 boats.
- A better system is to use the Handicap Corrected time of the 45% boat. I..e. the 4th boat in a fleet of 10 ; the 9th boat in a fleet of 20 etc. This has been found to be the “middle” or pivot time for most fleets and therefore all handicaps are adjusted around the handicap of that competitor and hence little or no handicap drift occurs for the group (division or fleet).
- Arguably a slightly better reference time is the average time of say the 10% boat to the 70% boat. This then provides an averaging process that can overcome some unusual situations where the 40% boat is not representative of the middle of the fleet.
- As noted above the measured performance for a race is the BCH where BCH = Ref Time / Elapsed Time of that competitor
Evaluating Handicapping Methods/Maths/Setups.
It can be said that the aim of measured performance handicapping is that all competitors should have an equal handicap corrected time for each race. But is the handicapping maths you are running getting anywhere near this aim? How can you test your results?
TopYacht provides a number of methods of checking the ongoing success of the handicapping maths. These include:
- The standard deviation of (normalised) finish times should be around 3 minutes or less for a normalising factor of 100 minutes for the average handicap corrected time.
- The number of different competitors who recorded a first place should be greater than half the number of competitors in the series. Likewise for second place, third place etc.
- There should be no ongoing handicap drift for the overall group.
- The spread of aggregate scores should be small for all competitors who are not loaded down with DNCs etc.
- The race graphs of say handicap vs “place” should show little preference to any one range of handicaps.
- The competitor’s performance graphs reveal how well the system is working. Most competitors should have a reasonably steady graph for their handicaps with their performances oscillating around their handicaps.
If you don’t like what you see then save your data then alter the parameters and tell TopYacht to re-calculate all races in this series.
Problems and complaints against handicaps :
Sailors are rarely happy with the handicaps in their fleet.
- When using a running average, the handicap of the winner may actually decrease even though the competitor won the race. This can occur if the winners previous performances were trending down and this race was an exceptionally good performance. The maths explains this, but competitors often feel that if someone wins then that competitor’s handicap should be increased. But to penalise a competitor for one good result is not helping provide a sensible ongoing handicaps. See observation number 5. Maybe a competitor education program at the local club level would help?
- From the handicapper’s point of view a fast improver can be a headache as can someone who has improved dramatically since last season. The later can be partially solved by the “fast adjustment” of handicap in the first few races. The former is helped by fast adjusting handicaps and/or by deliberate inclusions in the mathematics that tends to quickly adjust just the single competitor who is consistently performing well above his/her handicap.
- With the running average as the oldest BCH is dropped and that latest added then the new handicap can appear to “jump”. This is particularly true of an “improver”. Again this just how the maths works.
- A further complaint is that the consistent competitors never seem to “win” a race. This is another area currently under investigation by TopYacht.
- A consistent performer with small variation in their week to week performance will rarely win a race. But their consistent mid first half of the fleet scores may win a series. See observation 7.
- The inconsistent competitor may well win a race but in the next race score very badly. See observation 7
- Under this type of handicapping the series winner is usually either someone who has slowly improved over the series or someone who performed well above the Initial handicap they were given at the start of the series and the handicap maths took too long to catch up with them.
- Several different types of boat, e.g. sports boats & multihulls perform significantly differently under different conditions so they are very hard to handicap with just one “general purpose” handicap.
- Measured performance handicapping requires data that records the relative performance of competitors. It is therefore a difficult task to maintain handicaps in small fleets or fleets where few sail in any one race and where consequently there is little data for meaningful comparisons.
- Maintaining inter clubs handicaps should be achievable if there are regular “all in races” and a spirit of co-operation across the clubs involved. This is made easier by several facilities provided in TopYacht.
General notes re handicapping.
- If all the competitors sailed each week.
- If the course was the same each week.
- If the wind and water conditions were the same each week
Then a very viable set of handicaps could be generated.
- BUT, it is most unusual for any of the 3 conditions above to be fulfilled let alone all 3 conditions. So the handicaps that are generated will vary from race to race depending on the course, the other competitors racing and the conditions. This is par for the course, and short of going to the full IMS system, this is a weakness that just has to be understood by the competitors and accepted by them.
- most sailors know that certain conditions or certain courses favour certain boats. This is something that is generally understood and accepted as normal. In larger fleets the “real” competition is often between the boats that are of very similar design, leading to very similar performance. In smaller fleets this rarely occurs and again is accepted by most sailors as normality.
- If you have regular races round laid courses and also have “round the islands” races then, for the above reasons, it is necessary to have two sets of handicaps each to suit the particular style of race.
- Remember that Measured Performance handicapping provides a relative not absolute