Relazione su invito
Stochastic gradient descent-like relaxation is equivalent to Glauber dynamics in discrete inference problems.
Angelini M.C., Cavaliere A., Marino R., Ricci-Tersenghi F.
The temperature in a Monte Carlo algorithm governs the randomness that allows exploring a complex energy landscape. In contrast, in a Stochastic Gradient Descent (SGD) algorithm there is no temperature, and the parameter that controls the degree of randomness is given by the size of the mini-batch used. Despite SGD is widely used, a careful analysis of the optimal mini-batch size is missing. I will discuss the performances of these two well-known algorithms for discrete optimization and inference problems: despite their deep differences (SGD-like algorithm does not satisfy Detailed Balance, while Monte Carlo algorithm does), I will show that there exists an equivalence between their dynamics both at equilibrium and out of equilibrium. This is of particularly importance in understanding how to choose the optimal mini-batch size.