Steelmaking Process Optimised through a Decision Support System Aided by Self-Learning Machine Learning
This paper presents the application of a reinforcement learning (RL) algorithm, concretely Q-Learning, as the core of a decision support system (DSS) for a steelmaking subprocess, the Composition Adjustment by Sealed Argon-bubbling with Oxygen Blowing (CAS-OB) from the SSAB Raahe steel plant. Since...
Saved in:
Published in | Processes Vol. 10; no. 3; p. 434 |
---|---|
Main Authors | , , , , , , |
Format | Journal Article |
Language | English |
Published |
Basel
MDPI AG
01.03.2022
|
Subjects | |
Online Access | Get full text |
Cover
Loading…
Abstract | This paper presents the application of a reinforcement learning (RL) algorithm, concretely Q-Learning, as the core of a decision support system (DSS) for a steelmaking subprocess, the Composition Adjustment by Sealed Argon-bubbling with Oxygen Blowing (CAS-OB) from the SSAB Raahe steel plant. Since many CAS-OB actions are selected based on operator experience, this research aims to develop a DSS to assist the operator in taking the proper decisions during the process, especially less experienced operators. The DSS is intended to supports the operators in real-time during the process to facilitate their work and optimise the process, improving material and energy efficiency, thus increasing the operation’s sustainability. The objective is that the algorithm learns the process based only on raw data from the CAS-OB historical database, and on rewards set according to the objectives. Finally, the DSS was tested and validated by a developer engineer from the CAS-OB steelmaking plant. The results show that the algorithm successfully learns the process, recommending the same actions as those taken by the operator 69.23% of the time. The algorithm also suggests a better option in 30.76% of the remaining cases. Thanks to the DSS, the heat rejection due to wrong composition is reduced by 4%, and temperature accuracy is increased to 83.33%. These improvements resulted in an estimated reduction of 2% in CO2 emissions, 0.5% in energy consumption and 1.5% in costs. Additionally, actions taken based on the operator’s experience are incorporated into the DSS knowledge, facilitating the integration of operators with lower experience in the process. |
---|---|
AbstractList | This paper presents the application of a reinforcement learning (RL) algorithm, concretely Q-Learning, as the core of a decision support system (DSS) for a steelmaking subprocess, the Composition Adjustment by Sealed Argon-bubbling with Oxygen Blowing (CAS-OB) from the SSAB Raahe steel plant. Since many CAS-OB actions are selected based on operator experience, this research aims to develop a DSS to assist the operator in taking the proper decisions during the process, especially less experienced operators. The DSS is intended to supports the operators in real-time during the process to facilitate their work and optimise the process, improving material and energy efficiency, thus increasing the operation’s sustainability. The objective is that the algorithm learns the process based only on raw data from the CAS-OB historical database, and on rewards set according to the objectives. Finally, the DSS was tested and validated by a developer engineer from the CAS-OB steelmaking plant. The results show that the algorithm successfully learns the process, recommending the same actions as those taken by the operator 69.23% of the time. The algorithm also suggests a better option in 30.76% of the remaining cases. Thanks to the DSS, the heat rejection due to wrong composition is reduced by 4%, and temperature accuracy is increased to 83.33%. These improvements resulted in an estimated reduction of 2% in CO2 emissions, 0.5% in energy consumption and 1.5% in costs. Additionally, actions taken based on the operator’s experience are incorporated into the DSS knowledge, facilitating the integration of operators with lower experience in the process. |
Author | Ollila, Seppo Dorado Navas, Fernando del Real Torres, Alejandro Ojeda Roldán, Álvaro Andreiana, Doru Stefan Acevedo Galicia, Luis Enrique Leyva Guerrero, Carlos |
Author_xml | – sequence: 1 givenname: Doru Stefan orcidid: 0000-0002-7806-0316 surname: Andreiana fullname: Andreiana, Doru Stefan – sequence: 2 givenname: Luis Enrique surname: Acevedo Galicia fullname: Acevedo Galicia, Luis Enrique – sequence: 3 givenname: Seppo surname: Ollila fullname: Ollila, Seppo – sequence: 4 givenname: Carlos surname: Leyva Guerrero fullname: Leyva Guerrero, Carlos – sequence: 5 givenname: Álvaro surname: Ojeda Roldán fullname: Ojeda Roldán, Álvaro – sequence: 6 givenname: Fernando surname: Dorado Navas fullname: Dorado Navas, Fernando – sequence: 7 givenname: Alejandro surname: del Real Torres fullname: del Real Torres, Alejandro |
BookMark | eNpNkF1LwzAYhYMoOOdu_AUB74Rqmo-2uRzzEyYTqtclTd9umWtSk_Ri_96OKXpu3sPh4bxwLtCpdRYQukrJLWOS3PU-JYQRzvgJmlBK80TmaX76z5-jWQhbMkqmrBDZBK3LCLDr1Kexa_zmnYYQ8KqPpjMBGhw33g3rDVb4HrQJxllcDn3vfMTlPkTo8Nw0I1fvcQm7NlmC8vZQ9ar0xljAv8ElOmvVLsDs507Rx-PD--I5Wa6eXhbzZaKpFDGhLaWkoFAUTUsE5YroNGOFAlUzpnmTayWg1kUmckEEl1xoTmUmaq1YLilhU3R97O29-xogxGrrBm_HlxXNOOWCZ1k-UjdHSnsXgoe26r3plN9XKakOW1Z_W7JvMUVnyA |
CitedBy_id | crossref_primary_10_3390_pr10112461 crossref_primary_10_3390_pr10071378 crossref_primary_10_3390_app122312377 crossref_primary_10_1007_s11663_024_03035_z |
Cites_doi | 10.1016/j.ifacol.2015.10.068 10.3390/sym13061057 10.1016/j.watres.2020.116576 10.1109/ACCESS.2019.2941229 10.1016/j.chemosphere.2019.06.103 10.1007/s10845-018-1433-8 10.1016/B978-0-12-821348-3.00006-9 10.1002/pamm.201900390 10.3390/su13147703 10.1007/978-3-319-63913-0 10.3390/su12208718 10.1093/imammb/dqz004 10.1016/j.ifacol.2019.11.355 10.1038/nature16961 10.3390/app10207171 10.1007/BF00992698 10.3390/pr7100739 10.1201/9781420042313 10.1007/s10994-012-5280-0 10.1049/trit.2018.1007 10.1109/JAS.2017.7510310 10.3390/s20247297 10.3390/pr9071203 |
ContentType | Journal Article |
Copyright | 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License. |
Copyright_xml | – notice: 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License. |
DBID | AAYXX CITATION 7SR 8FD 8FE 8FG 8FH ABJCF ABUWG AFKRA AZQEC BBNVY BENPR BGLVJ BHPHI CCPQU D1I DWQXO GNUQQ HCIFZ JG9 KB. LK8 M7P PDBOC PIMPY PQEST PQQKQ PQUKI PRINS |
DOI | 10.3390/pr10030434 |
DatabaseName | CrossRef Engineered Materials Abstracts Technology Research Database ProQuest SciTech Collection ProQuest Technology Collection ProQuest Natural Science Collection Materials Science & Engineering Collection ProQuest Central (Alumni) ProQuest Central ProQuest Central Essentials Biological Science Collection AUTh Library subscriptions: ProQuest Central Technology Collection Natural Science Collection ProQuest One Community College ProQuest Materials Science Collection ProQuest Central Korea ProQuest Central Student SciTech Premium Collection Materials Research Database Materials Science Database Biological Sciences Biological Science Database Materials Science Collection Publicly Available Content Database ProQuest One Academic Eastern Edition (DO NOT USE) ProQuest One Academic ProQuest One Academic UKI Edition ProQuest Central China |
DatabaseTitle | CrossRef Publicly Available Content Database Materials Research Database ProQuest Central Student Technology Collection Technology Research Database ProQuest Central Essentials Materials Science Collection ProQuest Central (Alumni Edition) SciTech Premium Collection ProQuest One Community College ProQuest Natural Science Collection ProQuest Central China ProQuest Central Engineered Materials Abstracts Natural Science Collection ProQuest Central Korea Biological Science Collection Materials Science Database ProQuest Materials Science Collection ProQuest Biological Science Collection ProQuest One Academic Eastern Edition ProQuest Technology Collection Biological Science Database ProQuest SciTech Collection ProQuest One Academic UKI Edition Materials Science & Engineering Collection ProQuest One Academic |
DatabaseTitleList | Publicly Available Content Database CrossRef |
Database_xml | – sequence: 1 dbid: 8FG name: ProQuest Technology Collection url: https://search.proquest.com/technologycollection1 sourceTypes: Aggregation Database |
DeliveryMethod | fulltext_linktorsrc |
Discipline | Engineering Sciences (General) |
EISSN | 2227-9717 |
ExternalDocumentID | 10_3390_pr10030434 |
GeographicLocations | Europe |
GeographicLocations_xml | – name: Europe |
GroupedDBID | 5VS 8FE 8FG 8FH AADQD AAFWJ AAYXX ABJCF ACIWK ACPRK ADBBV AFKRA AFZYC ALMA_UNASSIGNED_HOLDINGS BBNVY BCNDV BENPR BGLVJ BHPHI CCPQU CITATION D1I GROUPED_DOAJ HCIFZ IAO ITC KB. KQ8 LK8 M7P MODMG M~E OK1 PDBOC PIMPY PROAC RNS 7SR 8FD ABUWG AZQEC DWQXO GNUQQ JG9 PQEST PQQKQ PQUKI PRINS |
ID | FETCH-LOGICAL-c295t-2f22082e88df0524a0c1638aeab33c4d7ca5ebc86575054945c42965bca379203 |
IEDL.DBID | BENPR |
ISSN | 2227-9717 |
IngestDate | Thu Oct 10 15:46:14 EDT 2024 Thu Sep 26 20:59:53 EDT 2024 |
IsDoiOpenAccess | true |
IsOpenAccess | true |
IsPeerReviewed | true |
IsScholarly | true |
Issue | 3 |
Language | English |
LinkModel | DirectLink |
MergedId | FETCHMERGED-LOGICAL-c295t-2f22082e88df0524a0c1638aeab33c4d7ca5ebc86575054945c42965bca379203 |
ORCID | 0000-0002-7806-0316 |
OpenAccessLink | https://www.proquest.com/docview/2642454667?pq-origsite=%requestingapplication% |
PQID | 2642454667 |
PQPubID | 2032344 |
ParticipantIDs | proquest_journals_2642454667 crossref_primary_10_3390_pr10030434 |
PublicationCentury | 2000 |
PublicationDate | 2022-03-01 |
PublicationDateYYYYMMDD | 2022-03-01 |
PublicationDate_xml | – month: 03 year: 2022 text: 2022-03-01 day: 01 |
PublicationDecade | 2020 |
PublicationPlace | Basel |
PublicationPlace_xml | – name: Basel |
PublicationTitle | Processes |
PublicationYear | 2022 |
Publisher | MDPI AG |
Publisher_xml | – name: MDPI AG |
References | Watkins (ref_34) 1992; 8 Wang (ref_26) 2019; 2019 ref_36 ref_13 Wang (ref_29) 2017; 4 ref_35 ref_12 Jang (ref_32) 2019; 7 ref_31 ref_30 ref_19 ref_18 ref_17 ref_16 Backman (ref_15) 2019; 52 Maass (ref_27) 2019; 37 Silver (ref_33) 2012; 87 Oztemel (ref_11) 2018; 31 Yang (ref_20) 2020; 189 Rotevatn (ref_14) 2015; 48 Silver (ref_22) 2016; 529 Kusunoki (ref_1) 2013; 104 Ha (ref_3) 2008; 53 ref_23 Pang (ref_24) 2019; 234 ref_21 Kanninen (ref_5) 2020; 2020 ref_2 ref_28 ref_9 ref_8 Shi (ref_10) 2018; 3 ref_4 Gottschalk (ref_25) 2019; 19 ref_7 ref_6 |
References_xml | – ident: ref_7 – ident: ref_28 – ident: ref_9 – volume: 2019 start-page: 3684 year: 2019 ident: ref_26 article-title: Measuring Structural Similarities in Finite MDPs publication-title: IJCAI Int. Jt. Conf. Artif. Intell. contributor: fullname: Wang – volume: 48 start-page: 6 year: 2015 ident: ref_14 article-title: A Model of the CAS-OB Process for Online Applications publication-title: IFAC-PapersOnLine doi: 10.1016/j.ifacol.2015.10.068 contributor: fullname: Rotevatn – ident: ref_35 doi: 10.3390/sym13061057 – volume: 189 start-page: 116576 year: 2020 ident: ref_20 article-title: Simulating a combined lysis-cryptic and biological nitrogen removal system treating domestic wastewater at low C/N ratios using artificial neural network publication-title: Water Res. doi: 10.1016/j.watres.2020.116576 contributor: fullname: Yang – volume: 7 start-page: 133653 year: 2019 ident: ref_32 article-title: Q-Learning Algorithms: A Comprehensive Classification and Applications publication-title: IEEE Access doi: 10.1109/ACCESS.2019.2941229 contributor: fullname: Jang – ident: ref_18 – volume: 234 start-page: 893 year: 2019 ident: ref_24 article-title: An influent responsive control strategy with machine learning: Q-learning based optimization method for a biological phosphorus removal system publication-title: Chemosphere doi: 10.1016/j.chemosphere.2019.06.103 contributor: fullname: Pang – volume: 31 start-page: 127 year: 2018 ident: ref_11 article-title: Literature review of Industry 4.0 and related technologies publication-title: J. Intell. Manuf. doi: 10.1007/s10845-018-1433-8 contributor: fullname: Oztemel – ident: ref_23 – ident: ref_36 doi: 10.1016/B978-0-12-821348-3.00006-9 – ident: ref_21 – volume: 19 start-page: e201900390 year: 2019 ident: ref_25 article-title: Differences and similarities between reinforcement learning and the classical optimal control framework publication-title: PAMM doi: 10.1002/pamm.201900390 contributor: fullname: Gottschalk – ident: ref_13 doi: 10.3390/su13147703 – ident: ref_19 doi: 10.1007/978-3-319-63913-0 – ident: ref_6 – ident: ref_8 – volume: 104 start-page: 109 year: 2013 ident: ref_1 article-title: Development of steelmaking processes for producing various high-quality steel grades at yawata works publication-title: Nippon Steel Tech. Rep. contributor: fullname: Kusunoki – ident: ref_4 – ident: ref_16 doi: 10.3390/su12208718 – volume: 2020 start-page: 3 year: 2020 ident: ref_5 article-title: Cost optimization system for full cycle metallurgical plants publication-title: Chernye Met. contributor: fullname: Kanninen – volume: 37 start-page: 22 year: 2019 ident: ref_27 article-title: A Markov decision process approach to optimizing cancer therapy using multiple modalities publication-title: Math. Med. Biol. doi: 10.1093/imammb/dqz004 contributor: fullname: Maass – volume: 52 start-page: 1174 year: 2019 ident: ref_15 article-title: Methods and Tools of Improving Steel Manufacturing Processes: Current State and Future Methods publication-title: IFAC-PapersOnLine doi: 10.1016/j.ifacol.2019.11.355 contributor: fullname: Backman – volume: 529 start-page: 484 year: 2016 ident: ref_22 article-title: Mastering the game of Go with deep neural networks and tree search publication-title: Nature doi: 10.1038/nature16961 contributor: fullname: Silver – ident: ref_30 doi: 10.3390/app10207171 – volume: 8 start-page: 279 year: 1992 ident: ref_34 article-title: Technical Note: Q-Learning publication-title: Mach. Learn. doi: 10.1007/BF00992698 contributor: fullname: Watkins – ident: ref_17 doi: 10.3390/pr7100739 – ident: ref_2 doi: 10.1201/9781420042313 – volume: 87 start-page: 183 year: 2012 ident: ref_33 article-title: Temporal-difference search in computer Go publication-title: Mach. Learn. doi: 10.1007/s10994-012-5280-0 contributor: fullname: Silver – volume: 53 start-page: 2 year: 2008 ident: ref_3 article-title: Start-up and some experience of CAS-OB at POSCO publication-title: Arch. Metall. Mater. contributor: fullname: Ha – volume: 3 start-page: 235 year: 2018 ident: ref_10 article-title: Adaptive PID controller based on Q-learning algorithm publication-title: CAAI Trans. Intell. Technol. doi: 10.1049/trit.2018.1007 contributor: fullname: Shi – volume: 4 start-page: 1 year: 2017 ident: ref_29 article-title: PDP: Parallel dynamic programming publication-title: IEEE/CAA J. Autom. Sin. doi: 10.1109/JAS.2017.7510310 contributor: fullname: Wang – ident: ref_31 doi: 10.3390/s20247297 – ident: ref_12 doi: 10.3390/pr9071203 |
SSID | ssj0000913856 |
Score | 2.2713397 |
Snippet | This paper presents the application of a reinforcement learning (RL) algorithm, concretely Q-Learning, as the core of a decision support system (DSS) for a... |
SourceID | proquest crossref |
SourceType | Aggregation Database |
StartPage | 434 |
SubjectTerms | Algorithms Argon Carbon dioxide Composition Decision support systems Efficiency Emissions Energy consumption Feedback Iron and steel plants Machine learning Operators Steel making Steel production User interface |
Title | Steelmaking Process Optimised through a Decision Support System Aided by Self-Learning Machine Learning |
URI | https://www.proquest.com/docview/2642454667 |
Volume | 10 |
hasFullText | 1 |
inHoldings | 1 |
isFullTextHit | |
isPrint | |
link | http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwfV1NT8IwGG4ELnowghpRJE30oIeG0bXdejKoIDEBjUjCbdn6QUwQEObBi7_ddutELh73cXrf9H3erz4PAJdYUl9rwRDTCUaE69DEQU8jA16BVB7XVNiG_mDI-mPyOKET13Bbu7XKIiZmgVouhO2RtwxwY0IJY8HN8gNZ1Sg7XXUSGiVQwW1ix7SV2-7w-eW3y2JZL0PKcl5S39T3reWqnY0DfbKNRNuBOEOX3gHYd2kh7OR-rIIdNa-BvT9kgTVQdcdwDa8cV_T1IZiOUqVm75mkFHRL__DJhAHjPiWhU-GBMbx3WjrQynialBvmVOWw8ybNf8kXHKmZRo5tdQoH2Y6lgsWLIzDudV_v-shpJyCBOU0R1hgbdFdhKLVHMYk9YTOvWMWJ7wsiAxFTlYjQzl1M1sYJFQaZGE1E7Acce_4xKM8Xc3UCIGc60KYqaavs2inhOBSCG68mUsZYBnVwUdgxWuYUGZEpLay1o42166BRmDhyx2QdbZx6-v_nM7CL7b2DbPmrAcrp6lOdm2wgTZqgFPYems7x5mnw3f0Brzm3lQ |
link.rule.ids | 315,786,790,12792,21416,27955,27956,33406,33777,43633,43838,74390,74657 |
linkProvider | ProQuest |
linkToHtml | http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwfV3LTsJAFJ0oLtSFEdSIok6iC11MLPNqZ2WIiqiACyBh17TzICYICHXh3ztTpiIbt21X93buua85B4ArrBgxRnLETYoRFSaycTAwyIJXqHQgDJOuod_p8taAvgzZ0DfcFn6tsoiJeaBWU-l65LcWuDFllPPwbvaJnGqUm656CY1NsEUJJ-4_j5pPvz0Wx3kZMb5kJSW2ur-dzev5MJDQdRxaD8M5tjT3wZ5PCmFj6cUy2NCTCtj9QxVYAWV_CBfw2jNF3xyAUS_TevyRC0pBv_IP32wQsM7TCnoNHpjAB6-kA52Ip0244ZKoHDbelf0u_YY9PTbIc62OYCffsNSweHAIBs3H_n0LeeUEJLFgGcIGY4vtOoqUCRimSSBd3pXoJCVEUhXKhOlURm7qYnM2QZm0uMRZKhMSChyQI1CaTCf6GEDBTWhsTVLX-aVTKnAkpbA-TZVKsAqr4LKwYzxbEmTEtrBw1o5X1q6CWmHi2B-SRbxy6cn_ry_Adqvfacft5-7rKdjB7gZCvgZWA6Vs_qXPbF6Qpee5838Arzm3BQ |
linkToPdf | http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwfV3PT8IwFG4UE6MHI6gRRW2iBz0sjK7t1pMh4sQfoAmSeFu2_iAmCAjz4H9v23UiF6_rTu-l7_v6-vp9AFwgQQKlOPWoypCHmYp0HfSVp8ErFNJninDT0O_1aXeIH97Im5t_WrixyrIm2kItptz0yJsauBEmmNKwqdxYxEsnvp59esZByty0OjuNdbBhSLaxcYjiu99-i9G_jAgtFEoDvd6czVv2YjDAq5i0WpItzsS7YMcRRNguMloFa3JSA9t_ZANroOo25AJeOtXoqz0wGuRSjj-suRR04__wWRcEnUgpoPPjgSnsOFcdaAw9NfmGhWg5bL8L_V_2DQdyrDynuzqCPTttKWH5YR8M49vXm67nXBQ8jhjJPaQQ0jgvo0gonyCc-txwsFSmWRBwLEKeEpnxyNzAaP7GMOEaoyjJeBqEDPnBAahMphN5CCCjKlT6fNKS9gEqZijinOn8ZkKkSIR1cF7GMZkVYhmJPmSYaCfLaNdBowxx4jbMIlmm9-j_5TOwqfOePN33H4_BFjKPEexEWANU8vmXPNEUIc9Obe5_AAMFuzo |
openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Steelmaking+Process+Optimised+through+a+Decision+Support+System+Aided+by+Self-Learning+Machine+Learning&rft.jtitle=Processes&rft.au=Doru%2C+Stefan+Andreiana&rft.au=Acevedo+Galicia%2C+Luis+Enrique&rft.au=Ollila%2C+Seppo&rft.au=Carlos+Leyva+Guerrero&rft.date=2022-03-01&rft.pub=MDPI+AG&rft.eissn=2227-9717&rft.volume=10&rft.issue=3&rft.spage=434&rft_id=info:doi/10.3390%2Fpr10030434&rft.externalDBID=HAS_PDF_LINK |
thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=2227-9717&client=summon |
thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=2227-9717&client=summon |
thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=2227-9717&client=summon |