Logical validity is not a guarantee of truth.
David Foster Wallace
At the outset of this series of posts, I indicated that many discussions involving assessments of the competitiveness of running races and, particularly, the competitiveness of ultramarathons, lack any sort of analytical context upon which one might rely to assert whether or not a race was “competitive”. In the prior three posts on this subject a methodology has been developed and successfully applied to known competitive races in the road marathon and road 10 km distances. The methodology utilizes the finishing time and finishing rank order as input data to define the normalized variables of percentage back from the winning time (from the finishing time data) and the cumulative probability/percentile rank (from the finishing rank order). These variables are plotted against one another for each race and this plot results in a graphical representation of the performance distribution for the race. Because the data are normalized, robust comparisons can be made with other races.
Analysis of the functionality of this performance distribution typically leads to a simple exponential function of the form:
y = a • exp (b • x)
x = percentage back from winning time for the cohort
y = cumulative probability of the result in the cohort
a = a pre-exponential factor inversely proportional to the excellence of the winning time relative to the cohort
b = the exponential factor directly proportional to the competitiveness of the cohort
It has been found that, with the exception of the Falmouth Road Race, an exponential performance distribution is extant in all marathon and 10 km road races analyzed. An exponential functionality is expected as the analysis is parameterizing the high performance tail of a normal distribution of competitors. The “steepness” of the exponential function (controlled by the magnitude of “b” in the equation above) is directly proportional to the competitiveness. This allows for the definition of a competitive index (CI) as equal to “b” in the functional form outlined above. Calculation of “b” and comparisons with other races and other race types allows for an analytical basis for assessing the competitiveness of a given race. An extensive comparison of marathon races is provided in Part II
of this series and comparison of two well known 10 km races is provided in Part III
The Falmouth Road Race typically exhibits a linear performance distribution and it is suggested that this is due to the presence of a “stacked field” of competitors assembled by the race directors. As a result, the performance distribution is not exponential since the field of competitors in the high performance tail is not representative of the tail of a normal distribution. Rather, this “non-normal” group of high performance competitors out-perform the expected exponential distribution. This is because the race has artificially assembled the high performance end of the population of competitors and “stacked” the field. Such races may represent the practical ultimate in competitiveness of a given race.
In this post an assessment of the competitiveness of ultramarathons is presented.
A selection of ultramarathon events have been chosen that represent the wide variety of such races. Presented here are analyses of:
- Western States Endurance Run- a mountainous, primarily trail 100 mile race
- Wasatch 100- a very mountainous 100 mile trail race
- JFK 50- a trail, towpath, and road 50 mile race
- Leadville 100- a high altitude, mountainous trail and dirt road 100 mile race
- Ultra Trail du Mont Blanc (UTMB)- a very mountainous, primarily trail 100 mile race
- Comrades Marathon- a road ultramarathon
- Pikes Peak Marathon- a marathon-length, mountainous trail “ultramarathon”
- The North Face Endurance Challenge 50 Mile Championship- a late season (December) trail 50 mile race that typically draws a large proportion of sponsored full-time athletes
The Comrades Marathon and Pikes Peak Marathon will be addressed separately as Comrades is a road ultramarathon (56 miles/ 90 km) and Pikes Peak marathon is a marathon-length “ultramarathon” (for the purposes of the analysis presented here, the Pikes Peak Marathon race is considered an ultramarathon because of the 7000 ft (2100 m) of both climbing and descending).
A great majority of the ultramarathon data fit well to the exponential performance distribution as is observed in an overwhelming majority of the other races analyzed in this study. However, in contrast to all of the road Marathon races analyzed, there are numerous ultramarathon races that exhibit a linear functionality in some years. Presented below is a summary of the data for the eight trail, hybrid, and road ultramarathons analyzed.
Among these data are quite a few events that are best described by a linear relationship. As shown in Part III, such linear functionality can be the result of a “stacked field” of competitors where the expected normal distribution is perturbed at the high performance tail. The data for each ultramarathon will be discussed separately below.
The Western States Endurance Run is a long-running (since 1973), very popular, and a generally accepted de-facto 100 mile trail ultrarunning championship race (although no “championship” award is given). The race is also one of the four races comprising the “grand slam of ultrarunning”. Due to the popularity of this race a lottery system for entry has been in place for quite some time. The field is limited to about 350 due to USFS Wilderness permit restrictions for a tiny portion of the course and, as of 2014, in excess of 2000 qualified applicants are in the lottery each year. Therefore entry into this race is highly unlikely from a probability perspective. As a result, the starting field of competitors is not necessarily representative of the ultramarathon population and may be skewed to some degree. One part of the ultramarathon population that can be compromised in such a system is the most competitive portion- the part of the population that is the subject of this series of posts. However, starting in 2007 Western States entered into an agreement with presenting sponsor Montrail that allows top finishers from a series of races known as the Montrail Ultra Cup to gain direct entry to Western States. This has greatly increased the probability for top competitors to get into the race.
Turning to the data on Western States presented above and singularly below, it is noted that there is a dramatic change in the functionality of the performance distribution that is directly aligned with the Montrail Ultra Cup entry process. Starting in 2007, the performance distribution of the 125% cohort reflects a linear relationship whereas prior to 2007 the expected exponential functionality is extant.
As is seen in the Falmouth Road Race fields, this linear relationship is indicative of a “stacked field”. The temporal correlation of this change in functionality with the direct entry process from the Montrail Ultra Cup for top competitors, although not necessarily causal, has arguably produced a consistent crop of competitors that effectively “stack” the field. As an example, presented below is a plot of the cumulative probability versus the percentage back from the winning time for the 2014 Western States race. Both linear and exponential fits are shown; clearly the data are best fit to a linear relationship. Note also that, just as has been seen in the Falmouth Road race results, the competitors in the 2014 Western States race are out-performing the equivalent exponential distribution, meaning that this field of competitors is of a higher caliber than what would occur with a more random selection process.
Presented below is a plot of the same data as above as well as the data from the 2004 Western States race (pre-Montrail Ultra Cup direct entry process). This comparison is exemplary of the entire dataset. Here it is seen that the performance distribution comparison also shows that the 2014 field is substantially out-performing the 2004 field, meaning that the 2014 race is more competitive than the 2004 race.
Comparison of results from the Western States 100 2004 (pre-Montrail Ultra Cup entry process) and 2014 (post Montrail Ultra Cup entry process). The different functionality is indicative of a “stacked field” in the 2014 event.
Finally, presented below is a comparison of the 2014 Western States data and the 2009 Falmouth Road Race data showing a very similar performance distribution with essentially the same slope. The slope of the linear fit to the data provides the same competitiveness metric as the exponential factor “b” described above, i.e. the slope is the competitiveness index for these fields where a steeper slope indicates a more competitive field. What this means is that the 2014 Western States race was, within a reasonable error estimation, as competitive as the 2009 Falmouth Road Race, a race which is one of the most competitive 10 km road events in the world.
The calculated slopes for the 2007-2014 Western States races are 0.039, 0.036, 0.037, 0.040, 0.041, 0.033, and 0.039, respectively. With the exception of the very hot 2013 race, there has been a general increasing trend in the competitiveness of the Western States race, something which has been discussed anecdotally within the ultramarathon community for the past few years. This is confirmed by the observed continued decrease of about 8% in the average finishing time of the 125% cohort studied here- once again with the exception of the hot 2013 race.
Finishing times for the Western States Endurance Run, 2007-2014 (race was cancelled in 2008) showing reduction of about 8% in the average finishing time of the 125% cohort over the period.
It is noted that although a linear finishing time distribution is indicative of a “stacked field” of high performance competitors, such a linear relationship could obtain in the less probable instance of a disproportionate number of comparatively lower-performing competitors being in the 125% cohort. This would lead to a significantly lower slope and should therefore be identifiable. There is no evidence that such a low-performance linear distribution is extant in any of the data analyzed in this 4-part series.
Prior to 2007 and the inclusion of competitors from the Montrail Ultra Cup designated slots, the Western States race exhibits a uniform adherence to the expected exponential functionality as seen in competitor populations that are not “manipulated”. Also prior to 2007, the Western States competitor slots (with the exception of those competitors who were in the top ten the prior year and choose to compete) are filled via a lottery. The results of these lotteries seem to represent a random cross section of the competitor population, otherwise a non-exponential functionality would likely be in evidence. The pre-2007 races analyzed here have CIs in the 0.111-0.125 range with an average of about 0.118. When compared to the “big 5″ marathons and the 10 km road races analyzed in parts II and III, we see that the pre-2007 Western States races were, on average, significantly less competitive (about 20%-30% less competitive).
The introduction of competitor entry via the Montrail Ultra Cup events has significantly increased the competitiveness of Western States to a level that is on par with one of the most competitive 10 km-type road races (Falmouth Road race). In addition, during this time period the course record has been broken and reset two times, so not only has the race been very competitive in the post-2007 period, it is also a very fast race, once again similar to what is found in the fast and highly competitive Falmouth Road Race. Western States sets a high standard for competitiveness in ultramarathons.
The Wasatch 100 mile trail race was chosen for this study because it represents one of the more difficult mountain trail races (with over 25,000 feet (7600 m) of climbing and a similar amount of descending). In addition, Wasatch is one of the very few 100 mile mountain trail races that has been run on the same course for an extended period. This allows for transparent and robust aggregation of data from numerous years. Such aggregated data will be analyzed and presented in Part V (Syntopicon). Wasatch 100 is also one of the 4 “grand slam of ultrarunning” races.
Presented below are the competitiveness parametrics for the Wasatch 100 race for the study period.
* the data for 2013 fit a linear and exponential functionality equally well with the exponential functionality giving a slightly better fit
The Wasatch 100 race has a course record time of 18:30:55 set by Geoff Roes in 2009. This compares to the record time for the Western States Endurance Run record of 14:46:44 set by Timothy Olson in 2012. An almost 4 hour difference in the record time for the nearly equivalent course distance reveals exactly how difficult the Wasatch race is in comparison to Western States. Wasatch is known to be much less of a “runners” race as there are a couple of miles more of vertical than at Western States. There are also substantial sections of steep power-hiking and equally difficult descents at Wasatch, both of which will slow the time of even the fastest competitors. Such a race will also have a different pool of competitors as the climbing, the attitude (+5500 feet (1700 m)), the heat (at times in excess of 100F (38C)), and the technical nature of portions of the course all filter out a good proportion of competitive runners who choose to participate in races more aligned with “runnable” courses. Wasatch still attracts many highly competitive ultrarunners given the underlying ethos of “challenge above all else” that is the fabric of ultrarunning as a sport.
The competitiveness of Wasatch is on par with pre-2007 Western States and includes a couple of more competitive “linear” years (2001 and 2008). The 2001 race winning time of 21:44:38 is the slowest time relative to other winning times in the study period. This time is about 5% slower than the next slowest time in the study period and over 17% slower than the record time of 18:30:55 from 2009. So although the 2001 race was competitive, it was not a particularly fast race. The 2008 winning time of 20:01:07 is a relatively fast time and, combined with the linear functionality of the percent back distribution, this race represents a fast and competitive example for the Wasatch 100. However, the Wasatch race is primarily a regional event and does not routinely draw a large group of known high-level competitors, so a highly competitive, linear finishing time distribution race is not necessarily a fast race as has been noted above. Of course, the weather can (and does) have significant negative effects on the finishing time, so that should be considered as well.
For the 12 “exponential” years in the study period, Wasatch has an average CI of about 0.110 compared to an average of about 0.118 in the pre-2007 (“exponential”) period for Western States. This makes Western States about 6% more competitive than Wasatch in these years. Neither the Wasatch nor the pre-2007 Western States races are as competitive as the “Big 5″ road marathons or other sub-elite road marathons analyzed where the “Big 5″ and the 2 sub-elite road marathons show a minimum average CI of 0.132 (New York) and 0.131 (Columbus). These road marathons are, on average, about 11% more competitive than Western States and about 16% more competitive than Wasatch. The two linear years 2001 and 2008 exhibit slopes of 0.035 and 0.039, respectively. These values are similar to, but generally lower than, those found with Western States although given the much smaller number of competitors in the 125% cohort in these years, the associated error is significantly greater, so it is best to keep this in mind when making comparisons.
Western States has an average of 19 finishers in the 125% cohort whereas Wasatch has an average of about 14. Western States has a field size of about 375. The Wasatch 100 field size has grown during the study period from about 200 to about 325. Although the starting field size will play some role, the smaller number of finishers in the 125% cohort of Wasatch is at least partly due to the difficulty of the Wasatch course where the slower pace naturally leads to larger multiplicative time differentials.
The JFK 50 is a long-running 50 mile ultramarathon race first run in 1963 with 4 finishers and now with typically over 1000 finishers. The course is a hybrid of trail, gravel road (towpath), and road and is very “runnable”. This race was also chosen because of the longevity of the race on a stable course route thereby enabling aggregation of data.
Presented below are the competitiveness parametrics for the JFK 50 race for the study period.
As can be gleaned from the table, the JFK 50 has a similar level of competitiveness as Western States and Wasatch in the “exponential” years. The slopes in the linear years are 0.040, 0.040, 0.038, 0.038, and 0.041 for 2001, 2002, 2008, 2009, and 2011, respectively. All of these values are similar to those in the linear years of Western States. It is noted that although the JFK 50 exhibits similar competitiveness to Western States, the number of competitors in the 125% cohort is similar as well (particularly in the last few years) even though the total field size is about 3 times larger at the JFK 50. This indicates that Western States has a proportionally deeper field in the 125% cohort, likely as a result of the Montrail Cup entry process. However, the competitive depth in both races, in an absolute sense, is essentially the same.
Ultra Trail du Mont Blanc (UTMB) is a 100 mile, very mountainous (31,000+ feet (9600 m) of climbing), primarily trail race and is thought to be one of the most difficult 100 mile races. The race is also viewed as being very competitive as it attracts a large group of sponsored professional athletes from around the globe. UTMB, like Wasatch, is different from Western States and the JFK 50 as it is not considered to be a “runnable” race, meaning that there are significant portions of speed hiking, technical climbs, and slow, technical descents.
The competitiveness data for UTMB is presented below where the years of shortened races are excluded.
UTMB exhibits a wide range of competitiveness from very low (0.077) to values on par with some of the most competitive “Big 5″ marathons (0.137). No linear years are observed. Why there is such a range in competitiveness and there are no highly competitive linear years, even with a high quality starting contingent, may have to do with the ruggedness of the course (lack of “runability”) and the highly variable weather playing havoc with even the fastest, most prepared athletes. The Alps is known for rapid weather changes that can test the mettle of anyone. Therefore the dynamics of the number of variables that play into a competitive time at UTMB is large enough to be seen in the results, independent of the quality of the field. The same might very well be the case for Wasatch and any other “difficult”, non-“runable” 100 mile race.
The Leadville 100 is a high altitude (10,000+ feet (3200 m)), mountainous (about 11,000 feet (3300 m) of climbing and descending), trail and dirt road race. This race has become very popular and the race promoters have recently established a lottery for entrance into the race (starting with the 2015 event).
The parametric data for Leadville for each year the study period are presented below.
The Leadville 100 race is unique among the events studied here in that prior to about 2008 the number of finishers in the 125% cohort is very small. This means that the analysis for these years will have a high quantitative uncertainty, however, as will be explained below, some very solid conclusions can be made about the competitiveness of this race.
First we examine the study period of 2001-2007 as, with the exception of 2003, these races have similarly low CIs and very shallow fields in the 125% cohort. It should be pointed out that this period includes both Mat Carpenter’s record time of 15:42:59 (2005) and Anton Krupicka’s two attempts in 2006 and 2007 (17:01:56 and 16:14:35, respectively) to take down this record. At the time, these finishing times were much faster than those preceding and lead to very few competitors in the 125% cohort. In fact, in 2005 the second place finisher was over 20% back. This shows how fast Carpenter’s time was. Similarly for 2006 and 2007, the second place finishers were 10% and 20% back from Krupicka’s time. No other events studied here show winning times that are so much faster than the remainder of the field. This is indicative of the presence of a singular talent that super dominates the field which can be the result of the winner having a “perfect” day or that the winner is just that much better than everyone else. It is probably a mixture of these things in this case, however, Carpenter was clearly a “super” talent much like Jornet is today. Presented below are the percentage back versus cumulative probability plots for the 2005 and 2007 races and the 2004 race as a representative race from the 2001-2008 period. This plot shows just how superior these winning performances by Carpenter and Krupicka were.
Finishing time distributions for the Leadville 100 2005 (blue), 2007 (red), and 2004 (green) races showing how extraordinary the winning performances were in 2005 and 2007. Note, the cumulative probability values for the 0% percentage back performances (winners) for 2005 and 2007 are coincident.
Since the 2001-2007 period there have now been seven sub-17 hour finishes starting with Ryan Sandes 16:46:54 in 2011. In 2014 there were three finishers in sub-17 hours. The race is drawing a deeper field of high caliber competitors and is therefore becoming significantly more competitive. This is substantiated by the appearance of more competitive “linear” years. However, the CIs for the linear years are relatively low- 0.036, 0.037, and 0.029 for 2010, 2011, and 2014, respectively. Should the trends over the past few years continue it is likely that Leadville will ciontinue to become more competitive.
North Face Endurance Challenge Championship
The North Face Endurance Challenge Championship (NFECC) race is a late season 50 mile primarily trail race with significant prize money that started in 2007. This race also experiences little to no competition for runners from other similarly scheduled races and therefore this race regularly draws a high caliber, international field of professional runners and many upcoming elite runners.
The parametric data for the finishing time distributions for the 2008-2014 NFECC is presented below (I was unable to find the data for 2007).
All of the years studied here exhibit linear finishing time distributions and the 125% cohort is as large as for any other trail ultramarathon in this study. In addition, the finishing times for this race are very fast for a trail race- for instance, Sage Canaday averaged a 7:12 mile pace for the 2014 race in muddy conditions.
The linear finishing time distributions are indicative that this race is very competitive. This combined with the consistently fast times and the deep fields means that the NFECC is arguably the most competitive ultramarathon studied here. A summary of the CIs for this event is presented in the following table along with the coefficient of and the size of the 125% cohort. Note that in this case the CI is the slope of the linear fit to the data as explained above.
The most competitive year for this race is 2012 but in that year the race course was changed and shortened due to torrential rainfall and a number of the top competitors got lost, inadvertently “shorted” the course, and were therefore DQ’d. So this race has an “asterisk”. However, a clear trend in increasing competitiveness is seen in the data and the magnitude of this competitiveness is on par with the highest values observed for Western States. The field is, however, much deeper than that of Western States. This is partly due to the fact that the race is 50 miles and late-race attrition is at a lower magnitude, but this is also due to the fact that entry into NFECC is essentially barrier-free for established elite runners.
Two “other” ultramarathons have been analyzed to provide additional context for comparisons:
- Pikes Peak Marathon- a trail marathon with 7000 feet (2100 m) of climbing and descending
- Comrades- a road ultramarathon
Pikes Peak Marathon
This race has a long history and has been run on the same course for many years. The 7000 foot (2100 m) climb followed by a return route down the same trail makes this marathon an “ultramarathon”. The race regularly draws top talent from the mountain and ultramarathon running world as well as a few washed-up elite marathoners looking for a new challenge. In 2014 the “ascent” race (run the day before the marathon) was the designated World Mountain Running Championship.
The parametric data for the Pikes Peak marathon during the study period is presented below.
The competitiveness of this race exhibits a very wide range and includes some more competitive “linear” years as well. This presents a very mixed bag with the depth of the field showing a decreasing trend. Comparisons of this race with others is best done on a year-by-year basis. The “linerar” years have CIs of 0.037, 0.036, 0.038, and 0.040 for 2005, 2010, 2012, and 2013, respectively. These values are all very similar to those found in the “linear” years of Western States and JFK 50, indicating that in these years Pikes Peak marathon is similarly competitive. I plan to do a more extensive post on the Pikes Peak Marathon in the near future as there are numerous interesting results when the races from the 1990’s (i.e the “Matt Carpenter era”) are included in the analysis.
Comrades is a long running, very well known, highly popular 56 mile (93 km) road ultramarathon that draws an international field of high caliber athletes and is a good choice to make comparisons with trail and mountainous trail ultramarathons. The race is run in opposing directions in alternate years- one year “up” and the next year “down”.
The parametric data for the Comrades race during the study period is presented below.
As expected the Comrades race has very similar results to that of the “Big 5″ marathons, both with respect to the competitiveness and to the depth of the field in the 125% cohort. The competitiveness is on par with all of the “Big 5″ marathons and encompasses a range of CI values that are essentially the same.
There is no apparent difference in competitiveness on the alternate, opposing direction, years.
Comrades represents an ultramarathon that is as competitive as any standard marathon.
A substantial quantity of data and analysis has been presented here. Although the results are clear, additional insight can be gained with a few graphical comparative examples.
First we compare a very competitive road marathon (Berlin) to a very competitive road ultramarathon (Comrades). Presented below is a competitiveness plot with results from the 2008 Berlin Marathon and the 2010 Comrades Ultramarahon. The CIs are 0.158 and 0.155, respectively and therefore represent very similar races from a competitiveness perspective. They also have similarly deep fields of 106 and 179, respectively, in the 125% cohort and the finishing times for both races are very fast. In fact even the pre-exponentials are essentially the same which indicates that the finishing times for each of these races is similarly “fast” in relation to the 125% cohort.
Both races show an exponential relationship indicating that the fields are not “stacked” and represent a normal distribution of competitors. This comparison illustrates that the most competitive road ultramarathons are just as competitive as the most competitive road marathons. Part II provides more data on road marathons, all of which support the observations made here.
A question that arises is why there are no “linear” years in either the road marathons or the road ultramarathon studied here, yet the 10 km road, hybrid ultramarathon, and trail ultramarathons all show some “linear” years. One reason may be that, particularly in the “Big 5″ road marathons, the top competitors typically choose one or two marathons to run each year with the expectation that a win will have a big payoff from a remuneration perspective. So the top athletes are picking and choosing which marathons to enter, perhaps to best increase their odds of winning. As a result there are no truly “stacked” fields because the top competitors are being distributed among the numerous prestigious marathons rather than all showing up to just one event. In the case of trail and hybrid marathons, similar economics do not prevail so the competitors may not be primarily picking races in a way that would increase their odds of winning rather they are just engaging with the best competition that they can find and end up “stacking” the fields of certain races (like NFECC). In the case of the Falmouth Road Race, 10 km races do not require the same kind of recovery that marathons and ultramarathons do and therefore a competitor can race many more 10 km events and not be as concerned with recovery and injury. Certainly there are other possibilities that may explain the lack of “linear” years for the road marathons and ultramarathon.
Moving on to comparisons of trail ultramarathons with road marathons we will utilize the 2008 Berlin Marathon as an example of a highly competitive, deep, and fast exponential finishing time distribution as a basis. Presented below is a comparison plot of the 2014 Western States, the 2002 Western States, and the 2008 Berlin Marathon. Here it is clear how much more competitive a linear distribution is when one compares the 2014 Western States (linear) to the 2002 Western States (exponential). Even though the depth of the 125% cohort is the same (21 for both 2014 and 2002), only 30% of the of the cohort is less than 15% back in 2002 whereas this value rises to 60% for the 2014 race. That is a big difference in competitiveness.
It is also clear from this comparative graph that the depth in the Berlin Marathon is much higher and the competitiveness is very high as well (exp = 0.158) when compared to the Western States 2002 race (exp = 0.118) a difference of over 25%. The 2008 Berlin Marathon is significantly more competitive than the 2002 Western States. Note: when comparing these finishing time distributions one must take into account the pre-exponential values as in this case they are very different (0.0195 for 2008 Berlin Marathon and 0.0523 for the 2002 Western States race) and this leads to a displacement of the Western Sates 2002 graph to a position “above” 2008 Berlin Marathon in the plot. This is a result of the winning time for the 2002 Western States being comparatively slow in relation to the winning time for the 2008 Berlin Marathon, for the respective 125% cohorts. However, the 2002 Western States graph is clearly a shallower function, and therefore less competitive, compared to the 2008 Berlin Marathon. Prior to 2007 and the introduction of entry into Western States via the Montrail Ultracup, and with just four exceptions in the 67 “Big 5″ marathon races analyzed, none of the Western States races in this period were as competitive as the “Big 5″ road marathons. Certainly, on average the pre-2007 Western States races were much less competitive than the “Big 5″ road marathons in the same period.
It is difficult to analytically compare the 2014 Western States race to the 2008 Berlin Marathon as the finishing time distributions are functionally different and therefore yield different parametrics for assessment of competitiveness (i.e. slope of the linear fit for linear functions and the exponential factor for exponential functions). So, in the absence of any sort of normailzation approach, it is best to make comparisons of exponential finishing time distributions as a group and likewise for linear finishing time distributions. In the case of the 2008 Berlin Marathon, the CI is one of the highest measured in this study (only the 2013 and 2014 Boston Marathon exhibits a higher value of CI) and this value is higher than any of the ultramarthons with exponential finishing time distributions studied here.
A representative example for linearly distributed races presented below is a comparison of 2014 Western States with the 2009 Falmouth Road Race and the 2014 NFECC.
In this case the slope of the linear fit is the CI and we find about a 6% variation in CI for this comparison which is not statistically significant, meaning that, within the error of the fit, all of these races have about the same competitiveness. This is a significant finding as the comparison is between two “championship” ultramarathons and a known highly competitive, international road race. Based on this comparison it is clear that the most competitive ultramarathons are as competitive as the most competitive road races.
A summary of the data in tabular form is presented below in ranked format. For each event the associated competitive index (CI) (either the exponential factor for exponential functional years or the slope for linear functional years) is tabulated along with the standard deviation. Also provided is the size of the 125% cohort analyzed, the standard deviation of this value, and the number of event years that have been analyzed.
Among the races that exhibit exponential functionality the Boulder Boulder 10 km road race is the most competitive on average. However, the 2014 Boston Marathon was the most competitive race analyzed in the exponential functionality group. We see a distinct and significant drop in competitiveness for the trail ultramarathons where Western States 2001-2006 has the highest competitiveness and Leadville is the least competitive race among the ultramarathons studied. Also clear is a significant reduction in the size of the 125% cohort with the trail ultramarathons when compared to the road races. This is certainly due to the much smaller fields in the ultramarathons but may also be a reflection of a generally less deep competitive field in the 125% cohort.
In the linear functionality group, the North Face Endurance Challenge Championship is the most competitive with JFK 50, Western States 2007-2014, Falmouth, and Pikes Peak Marathon in close succession. Given the limited data, both the Wasatch and Leadville races that exhibit linear functionality should be considered individually and not as means, although the means are provided for completeness. Of the linear group it is best to limit any conclusive comments to the NFECC, Falmouth, and Western States where from a statistical perspective each have about the same level of competitiveness. Once again, as expected, the road races are much deeper in the 125% cohort.
At the outset of this series of articles, it was observed that much of any discussion of competitiveness lacked a fundamental analytical basis for comparisons and that such discussions were of limited value as a result. Provided here is a proposed methodology for development of competitiveness metrics including assessment of field depth and relative speed for the winning time. Such metrics can serve to quantify and “calibrate” competitiveness in a way that facilitates comparisons and can lead to constructive discussion of competitiveness in distance, marathon, and ultramarathon running races.
The primary conclusions from this work are:
- Generally speaking, finishing time distributions for distance, marathon, and ultramarathon races exhibit exponential functionality. Such functionality is expected from a normal distribution of competitors in the event.
- In the case of “manipulated” or “stacked” fields, a linear finishing time distribution can obtain. Such events are typically (although not always) very competitive due to the non-normal quantity of highly competitive runners in the 125% cohort. These linear functionality races are arguably more competitive than the exponential functionality races for the 125% cohort.
- The competitive road ultramarathon analyzed here (Comrades) is just as competitive as the most competitive road marathons and distance road races.
- The most competitive trail ultramarathons (Western States 2007-2014 and NFECC) are just as competitive as the most competitive distance road race analyzed here (Falmouth).
- 3 and 4 indicate that ultramarathons (either trail or road) can be just as competitive as any of the most competitive road races.
Addendum – 7 January 2015
There has been some discussion of the relative ranking of the most competitive 2014 ultramarathons over here. Although other races might also be considered, Lake Sonoma 50, Western States Endurance Run, and North Face Endurance Challenge Championship 50 are clearly among the most competitive trail ultras in the US for 2014. Presented below is a competitiveness plot comparing these three 2014 races all of which have a linear performance distribution for the 125% cohort.
Inset on the graph is a chart of the competitiveness index (CI), the coefficient of determination (R^2), and n (the number of finishers in the 125% cohort). From both a CI and depth perspective, NFECC is the most competitive race of this group with the highest CI and the deepest field. Western States 2014 is next, and Lake Sonoma is the least competitive of the group. One might argue that the NFECC race should be given some sort of proportional weighting to account for the highly competitive nature of this year’s race and so on with other races being considered when choosing an award like the UROY.
I will encourage others to conduct such analyses to found any position on the competitiveness of a given race. But, as it concerns UROY, in the end an award is an award and such will always have a degree of subjectivity that will, hopefully, nucleate some interesting and fruitful discussions.