Abstract

To extend the operation window of batteries, active cell balancing has been studied in the literature. However, such an advancement presents significant computational challenges on real-time optimal control, especially when the number of cells in a battery increases. This article investigates the use of reinforcement learning (RL) and model predictive control (MPC) to effectively balance battery cells while at the same time keeping the computational load at a minimum. Specifically, event-triggered MPC is introduced as a way to reduce real-time computation. Different from the existing literature where rule-based or threshold-based event-trigger policies are used to determine the event instances, deep RL is explored to learn and optimize the event-trigger policy. Simulation results demonstrate that the proposed framework can keep the cell state-of-charge variation under 1% while using less than 1% computational resources compared to conventional MPC.

References

1.
Chen
,
J.
,
Liang
,
M.
, and
Ma
,
X.
,
2021
, “
Probabilistic Analysis of Electric Vehicle Energy Consumption Using MPC Speed Control and Nonlinear Battery Model
,”
2021 IEEE Green Technologies Conference
,
Denver, CO
,
Apr. 7–9
, pp.
181
186
.
2.
Singh
,
A.
,
Rustagi
,
R.
, and
Hegde
,
R. M.
,
2024
, “
Lifetime Improvement in Rechargeable Mobile IoT Networks Using Deep Reinforcement Learning
,”
IEEE Trans. Circuits Syst. II: Express Briefs
,
71
(
7
), pp.
4005
4009
. http://dx.doiorg/10.1109/TCSII.2024.3370686
3.
Hoekstra
,
F. S. J.
,
Ribelles
,
L. A. W.
,
Bergveld
,
H. J.
, and
Donkers
,
M. C. F.
,
2020
, “
Real-Time Range Maximisation of Electric Vehicles Through Active Cell Balancing Using Model-Predictive Control
,” 2020 American Control Conference, July 1–3, Denver, CO, pp.
2219
2224
.
4.
Chen
,
J.
,
Zhou
,
Z.
,
Zhou
,
Z.
,
Wang
,
X.
, and
Liaw
,
B.
,
2022
, “
Impact of Battery Cell Imbalance on Electric Vehicle Range
,”
Green Energy Intell. Transp.
,
1
(
3
), pp.
1
8
.
5.
Dubarry
,
M.
,
Vuillaume
,
N.
, and
Liaw
,
B. Y.
,
2010
, “
Origins and Accommodation of Cell Variations in Li-Ion Battery Pack Modeling
,”
Int. J. Energy Res.
,
34
(
2
), pp.
216
231
.
6.
Plett
,
G. L.
,
2015
,
Battery Management Systems, Volume II: Equivalent-Circuit Methods
,
Artech House
,
Norwood, MA
.
7.
Omariba
,
Z. B.
,
Zhang
,
L.
, and
Sun
,
D.
,
2019
, “
Review of Battery Cell Balancing Methodologies for Optimizing Battery Pack Performance in Electric Vehicles
,”
IEEE Access
,
7
(
1
), pp.
129335
129352
.
8.
Einhorn
,
M.
,
Roessler
,
W.
, and
Fleig
,
J.
,
2011
, “
Improved Performance of Serially Connected Li-Ion Batteries With Active Cell Balancing in Electric Vehicles
,”
IEEE Trans. Veh. Technol.
,
60
(
6
), pp.
2448
2457
.
9.
Chen
,
J.
,
Behal
,
A.
, and
Li
,
C.
,
2024
, “
Active Battery Cell Balancing by Real Time Model Predictive Control for Extending Electric Vehicle Driving Range
,”
IEEE Trans. Autom. Sci. Eng.
,
21
(
3
), pp.
4003
4015
.
10.
Gong
,
Z.
,
van de Ven
,
B. A. C.
,
Gupta
,
K. M.
,
da Silva
,
C.
,
Amon
,
C. H.
,
Bergveld
,
H. J.
,
Donkers
,
M. C. F. T.
, and
Trescases
,
O.
,
2019
, “
Distributed Control of Active Cell Balancing and Low-Voltage Bus Regulation in Electric Vehicles Using Hierarchical Model-Predictive Control
,”
IEEE Trans. Ind. Electron.
,
67
(
12
), pp.
10464
10473
.
11.
Alessio
,
A.
, and
Bemporad
,
A.
,
2009
,
A Survey on Explicit Model Predictive Control
(
Nonlinear Model Predictive Control
),
Springer
,
New York
, pp.
345
369
.
12.
Li
,
H.
, and
Shi
,
Y.
,
2014
, “
Event-Triggered Robust Model Predictive Control of Continuous-Time Nonlinear Systems
,”
Automatica
,
50
(
5
), pp.
1507
1513
.
13.
Luo
,
Y.
,
Xia
,
Y.
, and
Sun
,
Z.
,
2019
, “
Robust Eventtriggered Model Predictive Control for Constrained Linear Continuous System
,”
Int. J. Robust Nonlinear Control
,
29
(
5
), pp.
1216
1229
.
14.
Flessner
,
D.
,
Chen
,
J.
, and
Xiong
,
G.
,
2024
, “
Reinforcement Learning-Based Event-Triggered Active Battery Cell Balancing Control for Electric Vehicle Range Extension
,”
Electronics
,
13
(
5
), pp.
1
22
.
15.
He
,
H.
,
Xiong
,
R.
,
Zhang
,
X.
,
Sun
,
F.
, and
Fan
,
J.
,
2011
, “
State-of-Charge Estimation of the Lithiumion Battery Using an Adaptive Extended Kalman Filter Based on an Improved Thevenin Model
,”
IEEE Trans. Veh. Technol.
,
60
(
4
), pp.
1461
1469
.
16.
Sutton
,
R. S.
, and
Barto
,
A. G.
,
2018
,
Reinforcement Learning: An Introduction
,
MIT Press
,
Cambridge, MA
.
17.
Mnih
,
V.
,
Kavukcuoglu
,
K.
,
Silver
,
D.
,
Graves
,
A.
,
Antonoglou
,
I.
,
Wierstra
,
D.
, and
Riedmiller
,
M.
,
2013
, “Playing Atari With Deep Reinforcement Learning,” URL 1312.5602.
18.
Van Hasselt
,
H.
,
Guez
,
A.
, and
Silver
,
D.
,
2016
, “
Deep Reinforcement Learning With Double Q-Learning
,”
Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 30.1
,
Phoenix, AZ
,
Feb. 12–17
, pp.
2094
2100
.
You do not currently have access to this content.