Just finished a deep dive into the sparse attention variants from the “Longformer” paper. The implementation seems promising for long sequences, but I’m wondering if it’s practical for low-resource NLP tasks. Has anyone experimented with it on small datasets? Curious about your experience.