Entropy bounds on Bayesian Learning

O. Gossner, T. TOMALA

Journal of Mathematical Economics

January 2008, vol. 44, n°1, pp.24-32

Departments: Economics & Decision Sciences, GREGHEC (CNRS)

Keywords: Bayesian learning; Repeated decision problem; Value of information; Entropy

An observer of a process View the MathML source(xt) believes the process is governed by Q whereas the true law is P . We bound the expected average distance between P(xt|x1,…,xt-1)P(xt|x1,…,xt-1) and Q(xt|x1,…,xt-1)Q(xt|x1,…,xt-1) for t=1,…,nt=1,…,n by a function of the relative entropy between the marginals of P and Q on the n first realizations. We apply this bound to the cost of learning in sequential decision problems and to the merging of Q to P