ONN vs Transformer — Experiment Plan

Open source document →

1. #1

   $$\text{CSR} = \frac{\text{\# constraints satisfied in } \hat{G}}{\text{\# total constraints}}$$
2. #2

   $$\text{Contra} = \text{\# constraints violated in } \hat{G}$$
3. #3

   $$\text{RepairCost} = \text{Hamming distance}(\hat{G}, G^*) / |E^*|$$
4. #4

   $$\text{GA} = \frac{\text{\# locally-predicted edges consistent with global solution}}{\text{\# total edges}}$$
5. #5

   $$\text{ASR} = \frac{\text{\# episodes reaching goal without constraint violation}}{\text{\# total episodes}}$$
6. #6

   $$\text{SVR} = \frac{\text{\# episodes with violated constraints}}{\text{\# total episodes}}$$
7. #7

   $$\text{RecoveryRate} = \frac{\text{\# episodes recovering after perturbation}}{\text{\# perturbed episodes}}$$
8. #8

   $$\text{Latency} = \text{time per forward pass (ms)}$$
9. #9

   $$\text{PeakMem} = \text{max GPU/CPU memory (MB)}$$
10. #10

    $$\text{IterCount} = \text{\# iterations until tolerance threshold}$$
11. #11

    $$\text{ASR}(n) = f(n_{\text{objects}})$$
12. #12

    $$\Delta \text{ASR} = \text{ASR}_{\text{clean}} - \text{ASR}_{\text{noisy}}$$
13. #13

    $$\text{OOD ASR} = \text{ASR on unseen constraint types or object combinations}$$
14. #14

    $$\text{TPR} = \frac{\text{\# detected shifts}}{\text{\# actual shifts}}$$
15. #15

    $$\text{FPR} = \frac{\text{\# false alarms}}{\text{\# non-shift windows}}$$
16. #16

    $$\text{Delay} = |\hat{t}_{\text{shift}} - t_{\text{shift}}|$$