Optimal Provision of Regulation Service Reserves Under Dynamic Energy Service Preferences

We propose and solve a stochastic dynamic programming (DP) problem addressing the optimal provision of regulation service reserves (RSR) by controlling dynamic demand preferences in smart buildings. A major contribution over past dynamic pricing work is that we pioneer the relaxation of static, uniformly distributed utility of demand. In this paper we model explicitly the dynamics of energy service preferences leading to a non-uniform and time varying probability distribution of demand utility. More explicitly, we model active and idle duty cycle appliances in a smart building as a closed queuing system with price-controlled arrival rates into the active appliance queue. Focusing on cooling appliances, we model the utility associated with the transition from idle to active as a non-uniform time varying function. We (i) derive an analytic characterization of the optimal policy and the differential cost function, and (ii) prove optimal policy monotonicity and value function convexity. These properties enable us to propose and implement a smart assisted value iteration (AVI) algorithm and an approximate DP (ADP) that exploits related functional approximations. Numerical results demonstrate the validity of the solution techniques and the computational advantage of the proposed ADP on realistic, large-state-space problems.