You May Also Enjoy
How it works: why use KL divergence as policy constrait? An information theory perspective.
less than 1 minute read
Published:
KL divergence has been long used as a policy constrait in the field of reinforcement learning (RL). For example, in online RL, where agents interacts with the environment to update its policy, KL divergence is adopted to limit the search steps. Actually, KL divergence are so widely in the RL that it has become the golden standard. However, it sounds magical to me: why we adopt KL divergence as the constrait of policies?