A Critical Review of Reinforcement Learning for Optimal Coordination and Control of Modern Power Systems Under Uncertainties

The increasing penetration of distributed energy resources (DERs), electric vehicles (EVs), dynamic line ratings (DLRs), and flexible loads is reshaping modern power systems while introducing significant operational uncertainties. Reinforcement learning (RL) has gained attention as a data-driven solution for optimal coordination and control under uncertainty. However, existing studies that used RL for optimal coordination reviewed in this research primarily address uncertainties from DERs and load variability, largely neglecting DLRs and EVs as a time-varying network constraint. Moreover, long training times and limited interpretability hinder the practical deployment of RL-based controllers. This paper presents a comprehensive review of RL applications in power system operational control, categorizing approaches based on uncertainty sources, control objectives, and learning architectures. The review highlights the operational advantages of incorporating DLR uncertainty, including improved line utilization, congestion mitigation, enhanced renewable hosting capacity, and increased system flexibility. A critical research gap is identified in the absence of integrated RL frameworks that jointly consider DLRs and learning efficiency. To address this gap, a future research direction integrating a Belief–Desire–Intention (BDI) framework within RL is proposed, enabling faster convergence, constraint-aware decision-making, improved transparency, and enhanced resilience in modern power system coordination and control.