Preprint2020
LamBERT
Unknown
LAMBERT enhances RoBERTa with layout embeddings and relative bias for layout-aware document understanding without raw images.
0Feb 1, 2020
arXiv
AI Research
AI & ML Research Papers
A comprehensive index of artificial intelligence and machine-learning research with AI-generated summaries, citation metrics, and direct links to papers and code.
All Papers
4 papersPreprint2020
DeBERTa
Unknown
DeBERTa enhances BERT and RoBERTa through disentangled attention, an enhanced mask decoder, and virtual adversarial training.
0Jun 1, 2020Attention Mechanisms
Preprint2019
XLM Roberta
Unknown
XLM-RoBERTa scales multilingual masked language modeling to 100 languages, achieving strong cross-lingual transfer without translation objectives.
0Nov 1, 2019
arXiv
Preprint2019
RoBERTa
Unknown
RoBERTa shows BERT was undertrained and achieves state-of-the-art by optimizing hyperparameters and training on more data.
0Jul 1, 2019
arXiv