Skip to content

Results with optimized hyperparameters

Jump to bottom
Ricardo Montañana Gómez edited this page Oct 1, 2024 · 5 revisions

Accuracy results with optimized hyperparameters

# Dataset OdteT TBRRoFT TBRaFT TBRoFT
1 balance-scale 0.9750±0.013 0.9044±0.021 0.9023±0.024 0.7787±0.188
2 balloons 0.7250±0.286 0.6288±0.238 0.6000±0.232 0.6188±0.243
3 breast-cancer 0.7517±0.033 0.7347±0.053 0.7331±0.045 0.7227±0.043
4 breast-cancer-wisc 0.9707±0.011 0.9688±0.012 0.9690±0.012 0.9618±0.015
5 breast-cancer-wisc-diag 0.9761±0.012 0.9741±0.013 0.9714±0.013 0.9734±0.013
6 breast-cancer-wisc-prog 0.8031±0.042 0.8115±0.053 0.7999±0.054 0.8043±0.057
7 cardiotocography-10clases 0.8405±0.017 0.8327±0.017 0.8324±0.016 0.7827±0.019
8 cardiotocography-3clases 0.9264±0.012 0.9222±0.012 0.9197±0.011 0.8954±0.016
9 conn-bench-sonar-mines-rocks 0.8587±0.048 0.8357±0.049 0.8216±0.054 0.7905±0.054
10 cylinder-bands 0.7842±0.041 0.7728±0.036 0.7510±0.038 0.7148±0.038
11 dermatology 0.9806±0.014 0.9752±0.015 0.9741±0.016 0.9732±0.015
12 echocardiogram 0.8550±0.050 0.8507±0.053 0.8496±0.052 0.8292±0.056
13 fertility 0.8806±0.021 0.8810±0.065 0.8804±0.063 0.8800±0.060
14 haberman-survival 0.7350±0.022 0.7374±0.041 0.7356±0.045 0.7326±0.044
15 heart-hungarian 0.8232±0.039 0.8183±0.037 0.8203±0.041 0.8068±0.039
16 hepatitis 0.8425±0.052 0.8180±0.054 0.8195±0.055 0.8099±0.056
17 ilpd-indian-liver 0.7316±0.017 0.7133±0.033 0.7143±0.033 0.7072±0.034
18 ionosphere 0.9498±0.023 0.9471±0.021 0.9321±0.025 0.9069±0.030
19 iris 0.9653±0.036 0.9653±0.025 0.9630±0.025 0.9772±0.020
20 led-display 0.7275±0.030 0.6862±0.053 0.7126±0.023 0.7029±0.026
21 libras 0.8727±0.044 0.8851±0.034 0.8527±0.037 0.7556±0.047
22 low-res-spect 0.9115±0.021 0.8994±0.024 0.9007±0.025 0.8686±0.030
23 lymphography 0.8636±0.058 0.8559±0.058 0.8485±0.058 0.7734±0.066
24 mammographic 0.8353±0.028 0.8261±0.023 0.8296±0.021 0.8130±0.020
25 molec-biol-promoter 0.9014±0.067 0.8146±0.064 0.8230±0.071 0.7480±0.079
26 musk-1 0.9332±0.031 0.8939±0.028 0.8807±0.030 0.8629±0.030
27 oocytes_merluccius_nucleus_4d 0.8411±0.026 0.8389±0.020 0.8391±0.020 0.8256±0.023
28 oocytes_merluccius_states_2f 0.9326±0.015 0.9270±0.015 0.9238±0.016 0.9195±0.016
29 oocytes_trisopterus_nucleus_2f 0.8557±0.021 0.8365±0.024 0.8345±0.023 0.8049±0.025
30 oocytes_trisopterus_states_5b 0.9309±0.019 0.9323±0.016 0.9279±0.017 0.9304±0.016
31 parkinsons 0.9344±0.032 0.9310±0.036 0.9143±0.041 0.8948±0.046
32 pima 0.7727±0.024 0.7685±0.030 0.7633±0.028 0.7675±0.028
33 pittsburg-bridges-MATERIAL 0.8732±0.052 0.8602±0.054 0.8667±0.057 0.8014±0.077
34 pittsburg-bridges-REL-L 0.7061±0.090 0.6904±0.076 0.6956±0.083 0.6185±0.094
35 pittsburg-bridges-SPAN 0.6942±0.083 0.6924±0.085 0.6865±0.085 0.6222±0.095
36 pittsburg-bridges-T-OR-D 0.8686±0.042 0.8702±0.056 0.8650±0.058 0.8714±0.063
37 planning 0.7309±0.018 0.7202±0.058 0.7146±0.054 0.7131±0.059
38 post-operative 0.7121±0.028 0.7109±0.084 0.7074±0.087 0.7115±0.085
39 seeds 0.9605±0.025 0.9468±0.027 0.9439±0.030 0.9507±0.042
40 statlog-australian-credit 0.6783±0.003 0.6783±0.031 0.6783±0.029 0.6783±0.031
41 statlog-german-credit 0.7682±0.020 0.7578±0.027 0.7548±0.025 0.7561±0.025
42 statlog-heart 0.8481±0.041 0.8367±0.039 0.8374±0.040 0.8250±0.043
43 statlog-image 0.9711±0.007 0.9760±0.007 0.9698±0.008 0.9568±0.033
44 statlog-vehicle 0.8133±0.026 0.7863±0.025 0.7765±0.027 0.8033±0.044
45 synthetic-control 0.9910±0.008 0.9920±0.007 0.9913±0.007 0.9807±0.014
46 tic-tac-toe 0.9939±0.008 0.9857±0.007 0.9799±0.012 0.9833±0.008
47 vertebral-column-2clases 0.8635±0.040 0.8505±0.036 0.8501±0.034 0.8505±0.039
48 wine 0.9844±0.019 0.9826±0.020 0.9817±0.019 0.9879±0.017
49 zoo 0.9707±0.041 0.9466±0.045 0.9411±0.051 0.9462±0.047
Average Score 0.8595±0.093 0.8463±0.097 0.8425±0.097 0.8243±0.103

Friedman test

H0: There is no significant differences between all the classifiers.

  • Friedman statistic: 73,5306
  • Critical χ2 Value for df=3 and alpha=0,05: 7,8147279e+00
  • p-value: 7,4815371e-16 is less than 0,05
  • The null hypothesis H0 is rejected.

Post-hoc test

H0: There is no significant differences between the control model and the other models.

Post-hoc Holm test: H0: There is no significant differences between the control model and the other models.

classifier pvalue rank win tie loss H0
ODTET - 1,35 - - -
TBRRoFT 7,6660e-04 2,22 40 0 9 rejected
TBRaFT 2,7947e-10 3,02 46 0 3 rejected
TBRoFT 7,9936e-15 3,41 44 0 5 rejected
Clone this wiki locally