I do not have an answer, what I can say is that I find more stable test results testing the same function multiple times in the same test.
Ciao.
Mine is using the same method as Micheal's is. His ensures that the fastest one runs at least for a second by repeating the runs until more than 1sec elapsed. Then it repeats the rest for the same number of times (so if the 2nd takes twice as long it will be slightly more than 2sec).
Mine simply performs the same as Micheal's does for the fastest - i.e. run each until 1 sec elapsed. Then uses math to compare the various values. The Increment is the number of times each item was run, then the normalized column adjusts the time by multiplying it with the factor to match the fastest.
What I was actually referring to - is the value under the Relative column as compared to Michaels relative speed value. If I take my old version it takes around 10 seconds to do this:
Benchmarking ........ done for 128 iterations. Sorted from fastest.
Statement Increment Time(ms) Normalize Rating
--------------------------------------------------------------------------------
(ALE_LISTCOUNTITEM SEARCH ALIST) 128 1075 1075 8.59
(DG:COUNT-FOUND ALIST SEARCH) 32 1169 4676 1.97
(ALE_LISTCOUNTITEM2 SEARCH ALIST) 32 1294 5176 1.78
(ALE_LISTCOUNTITEMFUZZ2 SEARCH ALIST...) 32 1388 5552 1.66
(ALE_LISTCOUNTITEMFUZZ SEARCH ALIST ...) 32 1388 5552 1.66
(RELTRO:COUNT-FOUND1 ALIST SEARCH) 32 1452 5808 1.59
(RELTRO:COUNT-FOUND ALIST SEARCH) 32 1466 5864 1.57
(LM:COUNTITEMFUZZ SEARCH ALIST 1.0e-008) 16 1154 9232 1.00
--------------------------------------------------------------------------------
Then running the same with Micheal's it takes about 1 minute to complete
Benchmarking ...........Elapsed milliseconds / relative speed for 256 iteration(s):
(ALE_LISTCOUNTITEM SEARCH ALIST).............1108 / 8.71 <fastest>
(DG:COUNT-FOUND ALIST SEARCH)................4758 / 2.03
(ALE_LISTCOUNTITEM2 SEARCH ALIST)............5538 / 1.74
(ALE_LISTCOUNTITEMFUZZ2 SEARCH ALIST...).....5600 / 1.72
(ALE_LISTCOUNTITEMFUZZ SEARCH ALIST ...).....5709 / 1.69
(RELTRO:COUNT-FOUND1 ALIST SEARCH)...........5991 / 1.61
(RELTRO:COUNT-FOUND ALIST SEARCH)............6021 / 1.6
(LM:COUNTITEMFUZZ SEARCH ALIST 1.0e-...).....9656 / 1 <slowest>
Benchmarking ...........Elapsed milliseconds / relative speed for 256 iteration(s):
(ALE_LISTCOUNTITEM SEARCH ALIST).............1092 / 8.83 <fastest>
(DG:COUNT-FOUND ALIST SEARCH)................4758 / 2.03
(ALE_LISTCOUNTITEM2 SEARCH ALIST)............5413 / 1.78
(ALE_LISTCOUNTITEMFUZZ2 SEARCH ALIST...).....5601 / 1.72
(ALE_LISTCOUNTITEMFUZZ SEARCH ALIST ...).....5725 / 1.68
(RELTRO:COUNT-FOUND1 ALIST SEARCH)...........5990 / 1.61
(RELTRO:COUNT-FOUND ALIST SEARCH)............6006 / 1.61
(LM:COUNTITEMFUZZ SEARCH ALIST 1.0e-...).....9641 / 1 <slowest>
Not too different in the values.
But now with the new one those relative figures are reversed:
Benchmarking ........ done for 128 iterations. Sorted from fastest.
Statement Increment Time(ms) Normalize Relative
--------------------------------------------------------------------------------
(ALE_LISTCOUNTITEM SEARCH ALIST) 128 1061 1061 1.00
(DG:COUNT-FOUND ALIST SEARCH) 32 1170 4680 4.41
(ALE_LISTCOUNTITEM2 SEARCH ALIST) 32 1295 5180 4.88
(ALE_LISTCOUNTITEMFUZZ2 SEARCH ALIST...) 32 1373 5492 5.18
(ALE_LISTCOUNTITEMFUZZ SEARCH ALIST ...) 32 1389 5556 5.24
(RELTRO:COUNT-FOUND ALIST SEARCH) 32 1437 5748 5.42
(RELTRO:COUNT-FOUND1 ALIST SEARCH) 32 1467 5868 5.53
(LM:COUNTITEMFUZZ SEARCH ALIST 1.0e-008) 16 1155 9240 8.71
--------------------------------------------------------------------------------