[논문 리뷰] REALM: Retrieval-Augmented Language Model Pre-Training

이번 게시물에서는 최초로 retrieval와 language model을 같이 pre-training을 진행한 REALM을 제안한 논문인 REALM: Retrieval-Augmented Language Model Pre-Training에 대해 다뤄보겠다. 원문 링크는 아래와 같다.

REALM: Retrieval-Augmented Language Model Pre-Training Language model pre-training has been shown to capture a surprising amount of world knowledge, crucial for NLP tasks such as question answering. However, this knowledge is stored implici.....