How is the reward function defined for the DQN agent in the microgrid secondary control case study, and what is the agent's objective?
In the microgrid case study, the DQN agent's reward is r = -(delta-omega) - (delta-v), where delta-omega is the sum over the three DGs of (omega_ss - omega_i) and delta-v is the sum over the three DGs of (v_ss - v_i). The agent's objective is to maximize this reward, which equivalently minimizes each DG's frequency and voltage deviation from the steady-state reference values under stochastic conditions.
The source defines the reward as r = -Δω - Δv. The terms are Δω = Σ_{i=1}^{3}(ωss - ωi) and Δv = Σ_{i=1}^{3}(vss - vi), where ωss and vss are the steady-state frequency and voltage references. Because the reward takes the negative of these sums, larger deviations produce more negative reward. The DQN agent acts in the secondary control layer, observing a nine-element state containing each DG's frequency, voltage, and intrusion detection system outputs, and selecting one of the three secondary-control compute/communication action sets. Its stated goal is to maximize r, thereby minimizing the frequency and voltage deviation from their nominal steady state under stochastic conditions such as load changes, outages, and cyberattacks.
Key points
- Reward is r = -Δω - Δv, with Δω and Δv summed across all three distributed generators.
- Δω measures total frequency deviation from steady state as Σ(ωss - ωi).
- Δv measures total voltage deviation from steady state as Σ(vss - vi).
- The negative sign turns frequency and voltage errors into negative rewards.
- The agent maximizes r, which is equivalent to driving the microgrid back to nominal frequency and voltage.
Related questions
AI for Cybersecurity_ Research and Practice
Unknown
John Wiley & Sons, Inc.