ONN vs Transformer — Experiment Plan

Open source document →

1. #1

   $$\text{CSR}=\frac{\text{\# constraints satisfied in }\hat{G}}{\text{\# total constraints}}$$
2. #2

   $$\text{Contra}=\text{\# constraints violated in }\hat{G}$$
3. #3

   $$\text{RepairCost}=\text{Hamming distance}(\hat{G},G^{\ast })/|E^{\ast }|$$
4. #4

   $$\text{GA}=\frac{\text{\# locally-predicted edges consistent with global solution}}{\text{\# total edges}}$$
5. #5

   $$\text{ASR}=\frac{\text{\# episodes reaching goal without constraint violation}}{\text{\# total episodes}}$$
6. #6

   $$\text{SVR}=\frac{\text{\# episodes with violated constraints}}{\text{\# total episodes}}$$
7. #7

   $$\text{RecoveryRate}=\frac{\text{\# episodes recovering after perturbation}}{\text{\# perturbed episodes}}$$
8. #8

   $$\text{Latency}=\text{time per forward pass (ms)}$$
9. #9

   $$\text{PeakMem}=\text{max GPU/CPU memory (MB)}$$
10. #10

    $$\text{IterCount}=\text{\# iterations until tolerance threshold}$$
11. #11

    $$\text{ASR}(n)=f(n_{\text{objects}})$$
12. #12

    $$\Delta \text{ASR}={\text{ASR}}_{\text{clean}}-{\text{ASR}}_{\text{noisy}}$$
13. #13

    $$\text{OOD ASR}=\text{ASR on unseen constraint types or object combinations}$$
14. #14

    $$\text{TPR}=\frac{\text{\# detected shifts}}{\text{\# actual shifts}}$$
15. #15

    $$\text{FPR}=\frac{\text{\# false alarms}}{\text{\# non-shift windows}}$$
16. #16

    $$\text{Delay}=|{\hat{t}}_{\text{shift}}-t_{\text{shift}}|$$