Contextual bandit algorithms are extremely popular and widely used in recommendation systems to provide online personalized recommendations. A recurrent assumption is the stationarity of the reward function, which is rather unrealistic in most of the real-world applications. In the music…