Online Cyber-Attack Detection in the Industrial Control System: A Deep Reinforcement Learning Approach

In the open network environment, industrial control systems face huge security risks and are often subject to network attacks. The existing abnormal detection methods of industrial control networks have the problem of a low intelligence degree of adaptive detection and recognition. To overcome this...

Full description

Saved in:
Bibliographic Details
Published inMathematical problems in engineering Vol. 2022; pp. 1 - 9
Main Authors Liu, Zhenze, Wang, Chunyang, Wang, Weiping
Format Journal Article
LanguageEnglish
Published New York Hindawi 06.07.2022
John Wiley & Sons, Inc
Subjects
Online AccessGet full text

Cover

Loading…
Abstract In the open network environment, industrial control systems face huge security risks and are often subject to network attacks. The existing abnormal detection methods of industrial control networks have the problem of a low intelligence degree of adaptive detection and recognition. To overcome this problem, this article makes full use of the advantages of deep reinforcement learning in decision-making and builds a learning system with continuous learning ability. Specifically, industrial control network and deep reinforcement learning characteristics are applied to design a unique reward and learning mechanism. Moreover, an industrial control anomaly detection system based on deep reinforcement learning is constructed. Finally, we verify the algorithm on the gas pipeline industrial control dataset of Mississippi State University. The experimental results show that the convergence rate of this model is significantly higher than that of traditional deep learning methods. More importantly, this model can get a higher F1 score.
AbstractList In the open network environment, industrial control systems face huge security risks and are often subject to network attacks. The existing abnormal detection methods of industrial control networks have the problem of a low intelligence degree of adaptive detection and recognition. To overcome this problem, this article makes full use of the advantages of deep reinforcement learning in decision-making and builds a learning system with continuous learning ability. Specifically, industrial control network and deep reinforcement learning characteristics are applied to design a unique reward and learning mechanism. Moreover, an industrial control anomaly detection system based on deep reinforcement learning is constructed. Finally, we verify the algorithm on the gas pipeline industrial control dataset of Mississippi State University. The experimental results show that the convergence rate of this model is significantly higher than that of traditional deep learning methods. More importantly, this model can get a higher F1 score.
Author Wang, Chunyang
Liu, Zhenze
Wang, Weiping
Author_xml – sequence: 1
  givenname: Zhenze
  orcidid: 0000-0002-0708-6084
  surname: Liu
  fullname: Liu, Zhenze
  organization: College of Communication EngineeringJilin UniversityChangchunJilin 130025Chinajlu.edu.cn
– sequence: 2
  givenname: Chunyang
  orcidid: 0000-0002-9565-1257
  surname: Wang
  fullname: Wang, Chunyang
  organization: School of Computer and Communication EngineeringUniversity of Science and Technology BeijingBeijing 100083Chinaustb.edu.cn
– sequence: 3
  givenname: Weiping
  orcidid: 0000-0002-6796-7596
  surname: Wang
  fullname: Wang, Weiping
  organization: School of Computer and Communication EngineeringUniversity of Science and Technology BeijingBeijing 100083Chinaustb.edu.cn
BookMark eNp9kE1Lw0AQhhepYFu9-QMWPGrsziabbLyF-gmFgh_gLWw3s3ZruombLdJ_b0p7EvQ0c3jeeZlnRAaucUjIObBrACEmnHE-4VwymcERGYJI40hAkg36nfEkAh6_n5BR160Y4yBADomZu9o6pNPtAn1UhKD0J73FgDrYxlHraFgifXLVpgveqppOGxd8U9OXbRdwfUOLnsaWPqN1pvEa1-gCnaHyzroPWrStb5RenpJjo-oOzw5zTN7u716nj9Fs_vA0LWaRjuMsRHEsciOAK5BKSMwZZjoDSLPECLaohJCqQp6zKpE85Ys4yTBPUaWMM23AsHhMLvZ3-9qvDXahXDUb7_rKkqc5kzxPAXqK7yntm67zaEptg9o9HLyydQms3Pksdz7Lg88-dPUr1Hq7Vn77F365x5fWVerb_k__AJSog4U
CitedBy_id crossref_primary_10_1016_j_aej_2023_10_061
Cites_doi 10.1007/978-3-030-12330-7_1
10.3389/fenrg.2021.666130
10.1016/j.cose.2019.03.009
10.1145/3318464.3389779
10.1109/cagre.2019.8713289
10.23919/softcom.2019.8903886
10.1109/jiot.2019.2944632
10.1109/access.2020.2965259
10.15837/ijccc.2019.3.3548
10.1109/tpds.2021.3135412
10.1109/access.2020.2982057
10.1007/978-1-4842-5914-6_14
10.1109/rweek.2016.7573322
10.1109/comst.2021.3094360
10.1109/tii.2019.2891261
10.1016/j.cose.2019.101677
10.1109/dsn.2017.34
10.1109/eurospw.2017.62
10.1016/j.ijcip.2018.10.008
10.1145/3357613.3357637
10.1007/s10845-017-1315-5
10.1109/tste.2021.3105529
10.1109/access.2020.2992249
10.23919/softcom.2019.8903672
10.1016/j.cose.2020.101935
10.1109/jiot.2019.2919635
10.1109/dcoss.2019.00059
ContentType Journal Article
Copyright Copyright © 2022 Zhenze Liu et al.
Copyright © 2022 Zhenze Liu et al. This is an open access article distributed under the Creative Commons Attribution License (the “License”), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License. https://creativecommons.org/licenses/by/4.0
Copyright_xml – notice: Copyright © 2022 Zhenze Liu et al.
– notice: Copyright © 2022 Zhenze Liu et al. This is an open access article distributed under the Creative Commons Attribution License (the “License”), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License. https://creativecommons.org/licenses/by/4.0
DBID RHU
RHW
RHX
AAYXX
CITATION
7TB
8FD
8FE
8FG
ABJCF
ABUWG
AFKRA
ARAPS
AZQEC
BENPR
BGLVJ
CCPQU
CWDGH
DWQXO
FR3
GNUQQ
HCIFZ
JQ2
K7-
KR7
L6V
M7S
P5Z
P62
PHGZM
PHGZT
PIMPY
PKEHL
PQEST
PQGLB
PQQKQ
PQUKI
PRINS
PTHSS
DOI 10.1155/2022/2280871
DatabaseName Hindawi Publishing Complete
Hindawi Publishing Subscription Journals
Hindawi Publishing Open Access
CrossRef
Mechanical & Transportation Engineering Abstracts
Technology Research Database
ProQuest SciTech Collection
ProQuest Technology Collection
ProQuest Materials Science & Engineering
ProQuest Central (Alumni)
ProQuest Central UK/Ireland
Advanced Technologies & Aerospace Collection
ProQuest Central Essentials
ProQuest Central
Technology Collection
ProQuest One Community College
Middle East & Africa Database
ProQuest Central
Engineering Research Database
ProQuest Central Student
SciTech Premium Collection
ProQuest Computer Science Collection
Computer Science Database
Civil Engineering Abstracts
ProQuest Engineering Collection
Engineering Database
Advanced Technologies & Aerospace Database
ProQuest Advanced Technologies & Aerospace Collection
ProQuest Central Premium
ProQuest One Academic
ProQuest Publicly Available Content
ProQuest One Academic Middle East (New)
ProQuest One Academic Eastern Edition (DO NOT USE)
ProQuest One Applied & Life Sciences
ProQuest One Academic
ProQuest One Academic UKI Edition
ProQuest Central China
Engineering Collection
DatabaseTitle CrossRef
Publicly Available Content Database
Computer Science Database
ProQuest Central Student
Technology Collection
Technology Research Database
ProQuest One Academic Middle East (New)
Mechanical & Transportation Engineering Abstracts
ProQuest Advanced Technologies & Aerospace Collection
ProQuest Central Essentials
ProQuest Computer Science Collection
ProQuest Central (Alumni Edition)
SciTech Premium Collection
ProQuest One Community College
ProQuest Central China
ProQuest Central
ProQuest One Applied & Life Sciences
ProQuest Engineering Collection
Middle East & Africa Database
ProQuest Central Korea
ProQuest Central (New)
Engineering Collection
Advanced Technologies & Aerospace Collection
Civil Engineering Abstracts
Engineering Database
ProQuest One Academic Eastern Edition
ProQuest Technology Collection
ProQuest SciTech Collection
Advanced Technologies & Aerospace Database
ProQuest One Academic UKI Edition
Materials Science & Engineering Collection
Engineering Research Database
ProQuest One Academic
ProQuest One Academic (New)
DatabaseTitleList Publicly Available Content Database

CrossRef
Database_xml – sequence: 1
  dbid: RHX
  name: Hindawi Publishing Open Access
  url: http://www.hindawi.com/journals/
  sourceTypes: Publisher
– sequence: 2
  dbid: 8FG
  name: ProQuest Technology Collection
  url: https://search.proquest.com/technologycollection1
  sourceTypes: Aggregation Database
DeliveryMethod fulltext_linktorsrc
Discipline Applied Sciences
EISSN 1563-5147
Editor Li, Yang
Editor_xml – sequence: 1
  givenname: Yang
  surname: Li
  fullname: Li, Yang
EndPage 9
ExternalDocumentID 10_1155_2022_2280871
GeographicLocations India
GeographicLocations_xml – name: India
GrantInformation_xml – fundername: Peking University Third Hospital
  grantid: BYSYZHKC2021107
– fundername: Science and Technology Department of Jilin Province
  grantid: 20200401144GX
– fundername: Foshan Municipal Government
  grantid: BK21BF001
– fundername: National Natural Science Foundation of China
  grantid: 81961138010
– fundername: Fundamental Research Funds for the Central Universities
  grantid: QNXM20210036
– fundername: Technological Innovation Foundation of Shunde Graduate School, USTB
  grantid: BK19BF006
GroupedDBID 29M
2WC
3V.
4.4
5GY
5VS
8FE
8FG
8R4
8R5
AAFWJ
AAJEY
ABDBF
ABJCF
ABUWG
ACIPV
ACIWK
ADBBV
AENEX
AFKRA
AINHJ
ALMA_UNASSIGNED_HOLDINGS
ARAPS
BCNDV
BENPR
BGLVJ
BPHCQ
CCPQU
CS3
CWDGH
E3Z
EBS
ESX
GROUPED_DOAJ
HCIFZ
I-F
IAO
IEA
IOF
ISR
ITC
K6V
K7-
KQ8
L6V
M7S
MK~
M~E
OK1
P2P
P62
PIMPY
PQQKQ
PROAC
PTHSS
Q2X
REM
RHU
RHW
RHX
RNS
TR2
TUS
XSB
YQT
~8M
0R~
24P
AAYXX
ACCMX
CITATION
H13
OVT
PHGZM
PHGZT
7TB
8FD
AAMMB
AEFGJ
AGXDD
AIDQK
AIDYY
AZQEC
DWQXO
FR3
GNUQQ
JQ2
KR7
PKEHL
PQEST
PQGLB
PQUKI
PRINS
ID FETCH-LOGICAL-c337t-3359f512a18a58e90e7c711674f50bd558ade290d48262b347e96ea6020cf1f03
IEDL.DBID BENPR
ISSN 1024-123X
IngestDate Fri Jul 25 10:13:24 EDT 2025
Thu Apr 24 23:16:03 EDT 2025
Tue Jul 01 04:24:29 EDT 2025
Sun Jun 02 18:51:58 EDT 2024
IsDoiOpenAccess true
IsOpenAccess true
IsPeerReviewed true
IsScholarly true
Language English
License This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
https://creativecommons.org/licenses/by/4.0
LinkModel DirectLink
MergedId FETCHMERGED-LOGICAL-c337t-3359f512a18a58e90e7c711674f50bd558ade290d48262b347e96ea6020cf1f03
Notes ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ORCID 0000-0002-0708-6084
0000-0002-6796-7596
0000-0002-9565-1257
OpenAccessLink https://www.proquest.com/docview/2690829611?pq-origsite=%requestingapplication%
PQID 2690829611
PQPubID 237775
PageCount 9
ParticipantIDs proquest_journals_2690829611
crossref_citationtrail_10_1155_2022_2280871
crossref_primary_10_1155_2022_2280871
hindawi_primary_10_1155_2022_2280871
ProviderPackageCode CITATION
AAYXX
PublicationCentury 2000
PublicationDate 2022-07-06
PublicationDateYYYYMMDD 2022-07-06
PublicationDate_xml – month: 07
  year: 2022
  text: 2022-07-06
  day: 06
PublicationDecade 2020
PublicationPlace New York
PublicationPlace_xml – name: New York
PublicationTitle Mathematical problems in engineering
PublicationYear 2022
Publisher Hindawi
John Wiley & Sons, Inc
Publisher_xml – name: Hindawi
– name: John Wiley & Sons, Inc
References 22
23
25
26
27
28
J. F. Hernandez-Garcia (24) 2019
29
R. Setola (5) 2019; 77
M. Barrère (8) 2019
30
10
11
12
13
14
15
16
17
18
19
1
2
3
4
6
7
9
20
21
References_xml – ident: 16
  doi: 10.1007/978-3-030-12330-7_1
– ident: 11
  doi: 10.3389/fenrg.2021.666130
– ident: 15
  doi: 10.1016/j.cose.2019.03.009
– ident: 23
  doi: 10.1145/3318464.3389779
– ident: 4
  doi: 10.1109/cagre.2019.8713289
– ident: 30
  doi: 10.23919/softcom.2019.8903886
– ident: 21
  doi: 10.1109/jiot.2019.2944632
– ident: 2
  doi: 10.1109/access.2020.2965259
– ident: 17
  doi: 10.15837/ijccc.2019.3.3548
– ident: 14
  doi: 10.1109/tpds.2021.3135412
– volume: 77
  start-page: 907
  year: 2019
  ident: 5
  article-title: An overview of cyber attack to industrial control system
  publication-title: Chemical Engineering Transactions
– ident: 12
  doi: 10.1109/access.2020.2982057
– ident: 19
  doi: 10.1007/978-1-4842-5914-6_14
– ident: 27
  doi: 10.1109/rweek.2016.7573322
– ident: 7
  doi: 10.1109/comst.2021.3094360
– ident: 3
  doi: 10.1109/tii.2019.2891261
– ident: 20
  doi: 10.1016/j.cose.2019.101677
– ident: 28
  doi: 10.1109/dsn.2017.34
– year: 2019
  ident: 24
  article-title: Understanding multi-step deep reinforcement learning: a systematic study of the dqn target
– ident: 18
  doi: 10.1109/eurospw.2017.62
– ident: 6
  doi: 10.1016/j.ijcip.2018.10.008
– ident: 1
  doi: 10.1145/3357613.3357637
– ident: 22
  doi: 10.1007/s10845-017-1315-5
– ident: 25
  doi: 10.1109/tste.2021.3105529
– ident: 26
  doi: 10.1109/access.2020.2992249
– year: 2019
  ident: 8
  article-title: Identifying Security-Critical Cyber-Physical Components in Industrial Control Systems
– ident: 29
  doi: 10.23919/softcom.2019.8903672
– ident: 9
  doi: 10.1016/j.cose.2020.101935
– ident: 10
  doi: 10.1109/jiot.2019.2919635
– ident: 13
  doi: 10.1109/dcoss.2019.00059
SSID ssj0021518
Score 2.3153906
Snippet In the open network environment, industrial control systems face huge security risks and are often subject to network attacks. The existing abnormal detection...
SourceID proquest
crossref
hindawi
SourceType Aggregation Database
Enrichment Source
Index Database
Publisher
StartPage 1
SubjectTerms Adaptive control
Algorithms
Anomalies
Cybersecurity
Datasets
Decision making
Deep learning
Design
Gas pipelines
Industrial electronics
Machine learning
Natural gas
Neural networks
Nuclear power plants
SummonAdditionalLinks – databaseName: Hindawi Publishing Open Access
  dbid: RHX
  link: http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwjV1NS8QwEA26IHjxW1xdJYf1JMW2-WjirazKIuhBXOitpOlEF6UuuxXx35u02RVdRI-FaQ8zSee9ZOYNQn0pJDex6-gxGixBkTpQFjYHLJFMu4ROS3fecXvHhyN6k7HMiyTNlq_wbbZz9Dw-d6otwvWKr9oF5kj5MFvwKpu02o632EnwkWxe3_7j3W-ZZ-3JUd738dIvuMkr11towwNCnLYR3EYrUO2gTQ8Osd96s11kWlFQPPgoYBqkda30M76EuimlqvC4whbK4a9JHHjQFqHjVpP8AqfWGib4HhqtVN0cC2Ivr_qIU68tvodG11cPg2HghyQEmpCkDghh0tisrSKhmAAZQqITd7lCDQuLkjGhSohlWFJLJOKC0AQkB8UtTNQmMiHZR53qtYIDhAkxYPEKFKECKkiklODEIjimiFGKxl10Nndgrr2CuBtk8ZI3TIKx3Lk79-7uotOF9aRVzvjFru9j8YdZbx6o3G-zWR5zN7Fd8ig6_N9XjtC6e2zqbXkPderpGxxbVFEXJ82a-gQxxcFT
  priority: 102
  providerName: Hindawi Publishing
Title Online Cyber-Attack Detection in the Industrial Control System: A Deep Reinforcement Learning Approach
URI https://dx.doi.org/10.1155/2022/2280871
https://www.proquest.com/docview/2690829611
Volume 2022
hasFullText 1
inHoldings 1
isFullTextHit
isPrint
link http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwhV3fS8MwEA5uQ_DF3-J0jjzMJylrm6ZNfJE6N4fi0OFgbyVNEx1KN7eK-N-btOmGiPpSKD36cJfcfXe5fAdAixLqS1ff6JFcqASFcosp2GzhgGKuA7qX6HrH3cDvj7ybMR6bgtvCtFWWPjF31MmU6xp52_X1cG7qO87F7M3SU6P06aoZoVEBNeWCCamC2mV3cD9cplwqnhWX4VzNzofGZes7xjrrd9uaDIYEzregtP6ss-GPyQ_vnIec3jbYNFgRhoVxd8CaSHfBlsGN0OzKxR6QBV8o7HzGYm6FWcb4C7wSWd5llcJJChXKg6shHbBT9KfDgq78HIZKWszgUOQ0qjyvGELDvPoEQ0M7vg9Gve5jp2-Z-QkWRyjILIQwlSqgM4cwTAS1RcADfe7iSWzHCcaEJcKlduKpHMONkRcI6gvmKwTJpSNtdACq6TQVhwAiJIWCMiK2mfAIchgjPlLgDjMkGfPcOjgrFRhxQy6uZ1y8RnmSgXGk1R0ZddfB6VJ6VpBq_CLXMrb4R6xRGioyO3ARrdbL0d-fj8GG_lnegus3QDWbv4sTBTSyuAkqpHfdNGtKvd0-EPUc9sdfUcDP9g
linkProvider ProQuest
linkToHtml http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwtV1NT9wwEB1RKlQupaVF0C6tD3CqIhI7TuJKCK0Wtkv5OCCQ9pY6zhgQKGzZVGj_VH9jPYkDQqjtiXNGPownM2_GM28ANlSmEstposcadAmKMoF2sDmQqZKGAnpcUr3j6DgZncXfx3I8B7-7WRhqq-x8YuOoyxtDNfItntBybpVE0c7kZ0Bbo-h1tVuh0ZrFAc7uXMo23d7fdfe7yflw73QwCvxWgcAIkdaBEFJZF-Z0lGmZoQoxNSm9RsRWhkUpZaZL5CosY4e8eSHiFFWCOnG4ytjIhsKd-wJexsJFcppMH367T_Bc9GxH7zhxAYpx12gvJdUY-BZRz2Rp9CgELlxQ7n13-SQWNAFu-AZee2TK-q0pvYU5rJZhyaNU5n3A9B3Ylp2UDWYF3gb9utbmiu1i3fR0VeyyYg5TsoeVIGzQdsOzlhz9K-s7aZywE2xIW01Tn2Se5_Wc9T3J-Xs4exa9rsB8dVPhKjAhLDrghEWoMc5EpHWWCAclpRZW65ivwZdOgbnxVOa0UeM6b1IaKXNSd-7VvQab99KTlsLjL3Ib_i7-I9brLir3__s0f7DOD__-_BlejU6PDvPD_eODj7BIBzfNv0kP5uvbX7juIE5dfGrsisGP5zbkPxYLBt8
linkToPdf http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwtV3fT9wwDLbYoU17AfYDjcFYHuBpqq5NmrZBmqbjjhOM7YTQkO6tS1MH0KZy44oQ_9r-usVtCpomxhPPtfJgu_Znx_kMsKUylVhOL3qsQVegKBNoB5sDmSppKKHHJfU7vk6S_ZP481ROF-B39xaGxiq7mNgE6vLCUI-8zxNazq2SKOpbPxZxNBp_mv0KaIMU3bR26zRaFznEm2tXvs0_Hoycrbc5H-99G-4HfsNAYIRI60AIqaxLeTrKtMxQhZialG4mYivDopQy0yVyFZaxQ-G8EHGKKkGdOIxlbGRD4c59AospVUU9WNzdmxwd35Z7Lpe2D_E4MQOKaTd2LyV1HHifiGiyNPorIT49o0r8-vyfzNCku_EKLHmcygatY72ABaxewrLHrMxHhPkrsC1XKRveFHgZDOpamx9shHUz4VWx84o5hMnuFoSwYTsbz1qq9B02cNI4Y8fYULiaplvJPOvrKRt4yvPXcPIoml2FXnVR4RtgQlh0MAqLUGOciUjrLBEOWEotrNYxX4MPnQJz44nNab_Gz7wpcKTMSd25V_cabN9Kz1pCj3vktrwtHhDb6AyV-79_nt_56tv_f34Pz5wT518OJofr8JzObSaBkw3o1ZdX-M7hnbrY9I7F4Ptj-_If92QMcQ
openUrl ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Online+Cyber-Attack+Detection+in+the+Industrial+Control+System%3A+A+Deep+Reinforcement+Learning+Approach&rft.jtitle=Mathematical+problems+in+engineering&rft.au=Liu%2C+Zhenze&rft.au=Wang%2C+Chunyang&rft.au=Wang%2C+Weiping&rft.date=2022-07-06&rft.pub=John+Wiley+%26+Sons%2C+Inc&rft.issn=1024-123X&rft.eissn=1563-5147&rft.volume=2022&rft_id=info:doi/10.1155%2F2022%2F2280871&rft.externalDBID=HAS_PDF_LINK
thumbnail_l http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=1024-123X&client=summon
thumbnail_m http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=1024-123X&client=summon
thumbnail_s http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=1024-123X&client=summon