DeepMind's Journey to Enhanced Language Models via Machine Translation

DeepMind researchers have introduced a new method called Reinforced Self-Training (ReST) to improve the quality of large language models (LLMs) by aligning them with human preferences. They tested ReST in the domain of machine translation (MT) and found that it significantly improves translation quality. ReST generates synthetic training data offline and fine-tunes the LLM using a reward model based on performance feedback. The researchers believe ReST has potential in various generative learning settings and can advance reinforcement learning from human feedback (RLHF) across a broad range of language-related tasks.