INTELLIGENT POWER MANAGEMENT FOR BASE STATION IN CELLULAR NETWORKS USING MODEL-BASED REINFORCEMENT LEARNING

ABSTRACT

This study creates base station as an intelligent agent for power management in wireless communication using reinforcement learning (RL). Base station (BS) operations in a cellular network were modelled as a Markov Decision Process with five tuples (states or S, actions or A, transition or T, discount factor or Y, horizon or H) represented as BS (S, A, T, Y, H) for the purpose of minimizing energy waste while maintaining quality of service (QoS). From the initial state to the goal state as in any typical RL setup, the base station was trained using value iteration algorithm (VIA) to progressively learn to make optimal decision from experience gained in the wireless environment. The outcome of the implementation of the MDP model and VIA in Python programming showed that at convergence, two set of values were generated: the optimal policy was (0: 0, 1: 0, 2: 1) and the value function was (0: 52.59086557841341, 1: 47.331779020572064, 2: 42.59860111851486). With this optimal policy, the base station as a well-trained intelligent agent for power management, knows the set of actions to take at any given state for attaining cumulative reward of energy waste minimization while simultaneously sustaining QoS in the cellular network.

Keywords: Artificial intelligence, Base station, Markov decision process, Power management, Reinforcement learning, Wireless communication