Abstract

This article presents a dynamic mathematical model for a robot-enabled manufacturing system, where mobile robots independently manage workstation tasks. Each robot possesses one or multiple skills, enabling collaborative work at workstations. A real-time robot assignment problem is formulated to maximize production of the system, and a novel control strategy is developed to address this problem. Leveraging system properties derived from the model and moving window downtime prediction, the problem of maximizing system production is transformed into a more tractable control problem focused on identifying and achieving ideal clean configurations. The proposed solution significantly outperforms various benchmarks, including a pure reinforcement learning-based strategy, underscoring the importance of system understanding and its crucial role in enhancing flexibility and productivity in manufacturing systems.

References

1.
Waseem
,
M.
, and
Chang
,
Q.
,
2023
, “
Adaptive Mobile Robot Scheduling in Multiproduct Flexible Manufacturing Systems Using Reinforcement Learning
,”
ASME J. Manuf. Sci. Eng.
,
145
(
12
), p.
121005
.
2.
Bhatta
,
K.
,
Li
,
C.
, and
Chang
,
Q.
,
2022
, “
Production Loss Analysis in Mobile Multi-Skilled Robot Operated Flexible Serial Production Systems
,”
Manuf. Lett.
,
33
, pp.
835
842
, 50th SME North American Manufacturing Research Conference (NMRC 50,2022).
3.
Agrawal
,
A.
,
Won
,
S. J.
,
Sharma
,
T.
,
Deshpande
,
M.
, and
McComb
,
C.
,
2021
, “
A Multi-Agent Reinforcement Learning Framework for Intelligent Manufacturing With Autonomous Mobile Robots
,”
Proc. Des. Soc.
,
1
, pp.
161
170
.
4.
Dang
,
Q.-V.
,
Nguyen
,
C. T.
, and
Rudová
,
H.
,
2019
, “
Scheduling of Mobile Robots for Transportation and Manufacturing Tasks
,”
J. Heuristics
,
25
(
2
), pp.
175
213
.
5.
Ghodsian
,
N.
,
Benfriha
,
K.
,
Olabi
,
A.
,
Gopinath
,
V.
, and
Arnou
,
A.
,
2023
, “
Mobile Manipulators in Industry 4.0: A Review of Developments for Industrial Applications
,”
Sensors
,
23
(
19
).
6.
Monostori
,
L.
,
Váncza
,
J.
, and
Kumara
,
S. R. T.
,
2006
, “
Agent-Based Systems for Manufacturing
,”
CIRP Ann.
,
55
(
2
), pp.
697
720
.
7.
Lee
,
M.-L.
,
Behdad
,
S.
,
Liang
,
X.
, and
Zheng
,
M.
,
2022
, “
Task Allocation and Planning for Product Disassembly With Human-Robot Collaboration
,”
Robot. Comput.-Integr. Manuf.
,
76
, p.
102306
.
8.
Chen
,
Z.
,
Alonso-Mora
,
J.
,
Bai
,
X.
,
Harabor
,
D. D.
, and
Stuckey
,
P. J.
,
2021
, “
Integrated Task Assignment and Path Planning for Capacitated Multi-Agent Pickup and Delivery
,”
IEEE Robot. Autom. Lett.
,
6
(
3
), pp.
5816
5823
.
9.
Liu
,
M.
,
Ma
,
H.
,
Li
,
J.
, and
Koenig
,
S.
,
2019
, “
Task and Path Planning for Multi-Agent Pickup and Delivery
,”
Proceedings of the 18th International Conference on Autonomous Agents and MultiAgent Systems, AAMAS ’19
,
Montreal, QC, Canada
,
May 13–17
, pp.
1152
1160
, International Foundation for Autonomous Agents and Multiagent Systems.
10.
Dang
,
Q.-V.
,
Nguyen
,
C. T.
, and
Rudová
,
H.
,
2018
, “
Scheduling of Mobile Robots for Transportation and Manufacturing Tasks
,”
J. Heuristics
,
25
(
2
), pp.
175
213
.
11.
Casalino
,
A.
,
Zanchettin
,
A. M.
,
Piroddi
,
L.
, and
Rocco
,
P.
,
2021
, “
Optimal Scheduling of Human–Robot Collaborative Assembly Operations With Time Petri Nets
,”
IEEE Trans. Autom. Sci. Eng.
,
18
(
1
), pp.
70
84
.
12.
Yao
,
F.
,
Alkan
,
B.
,
Ahmad
,
B.
, and
Harrison
,
R.
,
2020
, “
Improving Just-in-Time Delivery Performance of Iot-Enabled Flexible Manufacturing Systems With AGV Based Material Transportation
,”
Sensors
,
20
(
21
).
13.
Zhang
,
L.
,
Yan
,
Y.
,
Hu
,
Y.
, and
Ren
,
W.
,
2021
, “
A Dynamic Scheduling Method for Self-Organized AGVs in Production Logistics Systems
,”
Proc. CIRP
,
104
, pp.
381
386
, 54th CIRP CMS 2021—Towards Digitalized Manufacturing 4.0.
14.
Zou
,
W.-Q.
,
Pan
,
Q.-K.
, and
Fatih Tasgetiren
,
M.
,
2021
, “
An Effective Iterated Greedy Algorithm for Solving a Multi-Compartment AGV Scheduling Problem in a Matrix Manufacturing Workshop
,”
Appl. Soft. Comput.
,
99
, p.
106945
.
15.
Fazlollahtabar
,
H.
,
Rezaie
,
B.
, and
Kalantari
,
H.
,
2010
, “
Mathematical Programming Approach to Optimize Material Flow in an AGV-Based Flexible Jobshop Manufacturing System With Performance Analysis
,”
Int. J. Adv. Manuf. Technol.
,
51
(
9–12
), pp.
1149
1158
.
16.
Li
,
Z.
,
Sang
,
H.
,
Pan
,
Q.
,
Gao
,
K.
,
Han
,
Y.
, and
Li
,
J.
,
2023
, “
Dynamic AGV Scheduling Model With Special Cases in Matrix Production Workshop
,”
IEEE Trans. Ind. Inform.
,
19
(
6
), pp.
7762
7770
.
17.
Hu
,
H.
,
Jia
,
X.
,
He
,
Q.
,
Fu
,
S.
, and
Liu
,
K.
,
2020
, “
Deep Reinforcement Learning Based AGVS Real-Time Scheduling With Mixed Rule for Flexible Shop Floor in Industry 4.0
,”
Comput. Ind. Eng.
,
149
, p.
106749
.
18.
Lämmle
,
A.
,
König
,
T.
,
El-Shamouty
,
M.
, and
Huber
,
M. F.
,
2020
, “
Skill-Based Programming of Force-Controlled Assembly Tasks Using Deep Reinforcement Learning
,”
Proc. CIRP
,
93
, pp.
1061
1066
.
19.
Yu
,
T.
,
Huang
,
J.
, and
Chang
,
Q.
,
2021
, “
Optimizing Task Scheduling in Human-Robot Collaboration With Deep Multi-Agent Reinforcement Learning
,”
J. Manuf. Syst.
,
60
, pp.
487
499
.
20.
Zheng
,
C.
,
An
,
Y.
,
Wang
,
Z.
,
Wu
,
H.
,
Qin
,
X.
,
Eynard
,
B.
, and
Zhang
,
Y.
,
2022
, “
Hybrid Offline Programming Method for Robotic Welding Systems
,”
Robot. Comput. Integr. Manuf.
,
73
, p.
102238
.
21.
Bai
,
X.
,
Fielbaum
,
A.
,
Kronmüller
,
M.
,
Knoedler
,
L.
, and
Alonso-Mora
,
J.
,
2023
, “
Group-Based Distributed Auction Algorithms for Multi-Robot Task Assignment
,”
IEEE Trans. Autom. Sci. Eng.
,
20
(
2
), pp.
1292
1303
.
22.
Li
,
J.
, and
Yang
,
F.
,
2020
, “
Task Assignment Strategy for Multi-Robot Based on Improved Grey Wolf Optimizer
,”
J. Ambient Intell. Humaniz. Comput.
,
11
, pp.
6319
6335
.
23.
Nie
,
Z.
, and
Chen
,
K.-C.
,
2022
, “
Hypergraphical Real-Time Multirobot Task Allocation in a Smart Factory
,”
IEEE Trans. Ind. Inform.
,
18
(
9
), pp.
6047
6056
.
24.
Bänziger
,
T.
,
Kunz
,
A.
, and
Wegener
,
K.
,
2020
, “
Optimizing Human-Robot Task Allocation Using a Simulation Tool Based on Standardized Work Descriptions
,”
J. Intell. Manuf.
,
31
, pp.
1635
1648
.
25.
Touzani
,
H.
,
Hadj-Abdelkader
,
H.
,
Séguy
,
N.
, and
Bouchafa
,
S.
,
2021
, “
Multi-Robot Task Sequencing and Automatic Path Planning for Cycle Time Optimization: Application for Car Production Line
,”
IEEE Robot. Autom. Lett.
,
6
(
2
), pp.
1335
1342
.
26.
Bhatta
,
K.
,
Huang
,
J.
, and
Chang
,
Q.
,
2022
, “
Dynamic Robot Assignment for Flexible Serial Production Systems
,”
IEEE Robot. Autom. Lett.
,
7
(
3
), pp.
7303
7310
.
27.
Li
,
J.
, and
Meerkov
,
S. M.
,
2009
,
Production Systems Engineering
,
Springer
,
New York
.
28.
Świechowski
,
M.
,
Godlewski
,
K.
,
Sawicki
,
B.
, and
Mańdziuk
,
J.
,
2022
, “
Monte Carlo Tree Search: A Review of Recent Modifications and Applications
,”
Artif. Intell. Rev.
,
56
(
3
), pp.
2497
2562
.
29.
Kelleher
,
J.
, and
O’Sullivan
,
B.
,
2009
, “
Generating All Partitions: A Comparison of Two Encodings
,”
preprint arXiv:0909.2331
. https://arxiv.org/abs/0909.2331
30.
Bhatta
,
K.
, and
Chang
,
Q.
,
2023
, “
Integrating Robot Assignment and Maintenance Management: A Multi-Agent Reinforcement Learning Approach for Holistic Control
,”
IEEE Robot. Autom. Lett.
,
8
(
9
), pp.
5338
5344
.
You do not currently have access to this content.