-
The gap between SOTA paper results and practical deployment
Reading top conference papers, model metrics keep climbing. But when moving these models toward real‑world inference, latency, cost and robustness often become show‑stoppers. Do you think conference evaluation frameworks should put more weight on deployment‑oriented metrics?
Ling Liang, Qin Tang and 38 others5 Comments-
This is such a real pain point. Many SOTA systems are not feasible outside well‑resourced lab environments.
21 -
Some venues are starting to add efficiency tracks, but those papers get far less attention than pure accuracy‑focused submissions.
104 -
This is such a real pain point. Many SOTA systems are not feasible outside well‑resourced lab environments.
1 -
Some venues are starting to add efficiency tracks, but those papers get far less attention than pure accuracy‑focused submissions.
1 -
Even model quantization behavior is rarely reported in original papers. That information is critical for anyone wanting to build on the work.
1
-