Time-Dependent Blackwell Approachability and Application to Absorbing Games

Abstract

Blackwell’s approachability is a general online learning framework where a Decision Maker obtains vector-valued outcomes, and aims at the convergence of the average outcome to a given “target” set. Blackwell gave a sufficient condition for the decision maker having a strategy guaranteeing such a convergence against an adversarial environment, which ensures convergence. Blackwell’s approachability has since been applied to numerous problems, in regret minimization and game theory, in particular. We extend this framework by allowing the outcome function and the inner product to be time-dependent. We establish a general guarantee for the natural extension to this framework of Blackwell’s algorithm. In the case where the target set is an orthant, we present a family of time-dependent inner products which yields different convergence speeds for each coordinate of the average outcome. We apply this framework to absorbing games (an important class of stochastic games) for which we construct ε-uniformly optimal strategies using Blackwell’s algorithm in a well-chosen auxiliary approachability problem, thereby giving a novel illustration of the relevance of online learning tools for solving games.