Decentralized Stochastic Control with Many Decision Makers and their Mean-Field Limit

Abstract

In this thesis, we study large stochastic team problems (known also as decentralized stochastic control problems) with finite as well as countably infinite number of decision makers.

In the first part of the thesis, we introduce sufficient conditions of optimality and characterize existence and structural properties for globally optimal policies. We first establish sufficient conditions of optimality (in terms of regularity and convexity conditions on the cost function) for static teams with a countably infinite number of decision makers. Then, we focus on static and dynamic team problems and their mean-field limits, where the cost function and dynamics satisfy an exchangeability condition. For this class of team problems, by first imposing convexity, we show that an optimal policy exhibits symmetry and then we establish the convergence of optimal policies for teams with $N$ decision makers to the corresponding optimal policies of mean-field teams. Then, we establish an existence and structural result for convex mean-field teams with a decentralized information structure. Next, we relax the convexity assumption, and we characterize existence and structural properties of an optimal decentralized policy.

Through our analysis of the aforementioned results, it has been shown that static reduction of dynamic stochastic team problems is an effective method for establishing existence and approximation results for optimal policies. With this relation, in the second part, we classify static reductions into three categories: (i) policy-independent, (ii) policy-dependent, and (iii) static measurement with control-sharing reduction. For the first type, we show that there is a bijection between person-by-person optimal (globally optimal) policies of dynamic teams and their policy-independent static reductions. For the second type, although there is a bijection between globally optimal policies of dynamic teams with partially nested information structures and their static reductions, in general, there is no bijection between person-by-person optimal policies of dynamic teams and their policy-dependent static reductions. We present, however, sufficient conditions under which bijection relationships hold. Under static measurement with control-sharing reduction, the connections between optimality concepts can be established under relaxed conditions. An implication is a convexity characterization of dynamic team problems under static measurement with control-sharing reduction.