Emergent Communication Pretraining.
2023
Description
Investigations entailed using existing methods to develop emergent languages to be utilized as a pretraining dataset for multimodal-LLMs. Tests included downstream tasks such as Visual Grounding and VQA.
By training a Speaker agent that outputs an emergent language describing images, we used the resulting language to pretrain a multimodal LLM ,OFA, with corresponding images for various multimodal learning tasks. 