Performance
Performance test performed on Windows on an Intel i7-6500U CPU @ 2.50GHz, 2 Cores, 4 Logical Processors. Almost all results were averaged over 10 runs (Except runs taking longer than 4 hours).
Due to faster I/O, runs on linux tend to be 20-25% faster.
Each test was run with the following properties:
UNSEEN_NEGATIVE_EXAMPLES = 5
TOP_K_OUTPUT = 10
WORKER_THREADS = 3
| Abbreviation | Property |
|---|---|
| D_U | DISCRIMINATION_UNIQUE |
| I_D | INTERMEDIATE_DISCRIMINATION |
| D_B | DISCRIMINATION_BOUND |
Runtime file reading and preparation: 11,805 ms
| D_U | I_D | D_B | Rule application | hits@1 | hits@3 | hits@10 |
|---|---|---|---|---|---|---|
| 0 | 0 | 1000 | 18.95 s | 0.7933 | 0.8292 | 0.8640 |
| 0 | 1 | 1000 | 18.53 s | 0.7934 | 0.8292 | 0.8639 |
| 1 | 0 | 1000 | 25.28 s | 0.7933 | 0.8292 | 0.8643 |
| 1 | 1 | 1000 | 24.7 s | 0.7933 | 0.8292 | 0.8641 |
| 1 | 0 | 4000 | 27.28 s | 0.8090 | 0.8445 | 0.8783 |
Runtime file reading and preparation: 11,752 ms
| D_U | I_D | D_B | Rule application | hits@1 | hits@3 | hits@10 |
|---|---|---|---|---|---|---|
| 0 | 0 | 1000 | 9.84 s | 0.7933 | 0.8292 | 0.8640 |
| 0 | 1 | 1000 | 9.71 s | 0.7934 | 0.8292 | 0.8639 |
| 1 | 0 | 1000 | 10.45 s | 0.7933 | 0.9292 | 0.8643 |
| 1 | 1 | 1000 | 10.37 s | 0.7933 | 0.8292 | 0.8641 |
| 0 | 1 | 4000 | 10.44 s | 0.8090 | 0.8445 | 0.8782 |
The AnyBURL original Algorithm is equivalent to the IRIFAB re-implemented original Algorithm with D_U = 1 and I_D = 0.
Runtime file reading and preparation: 7,097 ms
| D_U | I_D | D_B | Rule application | hits@1 | hits@3 | hits@10 |
|---|---|---|---|---|---|---|
| 1 | 0 | 1000 | 148.87 s | 0.8094 | 0.8443 | 0.8785 |
| 1 | 0 | 4000 | 203.53 s | 0.8094 | 0.8443 | 0.8785 |
Runtime file reading and preparation: 43.58 s
| D_U | I_D | D_B | Rule application | hits@1 | hits@3 | hits@10 |
|---|---|---|---|---|---|---|
| 1 | 0 | 1000 | 276 mins (4.6 h) | 0.1160 | 0.2107 | 0.3514 |
Runtime file reading and preparation: 42.9 s
| D_U | I_D | D_B | Rule application | hits@1 | hits@3 | hits@10 |
|---|---|---|---|---|---|---|
| 1 | 0 | 1000 | 19 min | 0.1160 | 0.2107 | 0.3514 |
| 0 | 1 | 1000 | 56.73 s | 0.1191 | 0.2135 | 0.3560 |
The AnyBURL original Algorithm is equivalent to the IRIFAB re-implemented original Algorithm with D_U = 1 and I_D = 0.
Runtime file reading and preparation: 79.19 s
| D_U | I_D | D_B | Rule application | hits@1 | hits@3 | hits@10 |
|---|---|---|---|---|---|---|
| 1 | 0 | 1000 | 789 min (13.2 h) | 0.1160 | 0.2107 | 0.3514 |
Runtime file reading and preparation: 47.27 s
| D_U | I_D | D_B | Rule application | hits@1 | hits@3 | hits@10 |
|---|---|---|---|---|---|---|
| 0 | 1 | 1000 | 52.1 min | 0.1646 | 0.2798 | 0.4375 |
Runtime file reading and preparation: 46.96 s Confidence level: 95% Margin of error: 5%
Means over 100 runs:
| D_U | I_D | D_B | Rule application | hits@1 | hits@3 | hits@10 |
|---|---|---|---|---|---|---|
| 0 | 1 | 1000 | 45.43 s | 0.1672 | 0.2866 | 0.4372 |
Boxplot of 100 runs:
