How it works: why use KL divergence as policy constrait? An information theory perspective.
Published:
KL divergence has been long used as a policy constrait in the field of reinforcement learning (RL). For example, in online RL, where agents interacts with the environment to update its policy, KL divergence is adopted to limit the search steps. Actually, KL divergence are so widely in the RL that it has become the golden standard. However, it sounds magical to me: why we adopt KL divergence as the constrait of policies?