強化学習によるリニア式波力発電装置の電力量最大化
This paper presents a deep reinforcement learning control method to maximize output energy for a point absorber type wave energy converter (WEC) with a linear generator. Conventional control methods require the dynamic model of the WEC. Modeling errors of the dynamic model, however, make energy abso...
Saved in:
| Published in | 日本船舶海洋工学会論文集 Vol. 31; pp. 229 - 238 |
|---|---|
| Main Authors | , |
| Format | Journal Article |
| Language | Japanese |
| Published |
公益社団法人 日本船舶海洋工学会
2020
|
| Online Access | Get full text |
| ISSN | 1880-3717 1881-1760 |
| DOI | 10.2534/jjasnaoe.31.229 |
Cover
| Abstract | This paper presents a deep reinforcement learning control method to maximize output energy for a point absorber type wave energy converter (WEC) with a linear generator. Conventional control methods require the dynamic model of the WEC. Modeling errors of the dynamic model, however, make energy absorption smaller and cause incorrect control. The proposed method, which is a model-free control method learns the optimal damping and stiffness coefficients based on experiences. In the proposed control method, damping and stiffness coefficients are able to vary in time-domain depending on the incident waves by deep reinforcement learning. The performance of the proposed control method is investigated through numerical simulation in both regular and irregular waves. Compared with the conventional control method, averaged output power increased, and the power fluctuation decreased without the dynamic model. It is understood that the proposed method is more effective than the conventional control method. |
|---|---|
| AbstractList | This paper presents a deep reinforcement learning control method to maximize output energy for a point absorber type wave energy converter (WEC) with a linear generator. Conventional control methods require the dynamic model of the WEC. Modeling errors of the dynamic model, however, make energy absorption smaller and cause incorrect control. The proposed method, which is a model-free control method learns the optimal damping and stiffness coefficients based on experiences. In the proposed control method, damping and stiffness coefficients are able to vary in time-domain depending on the incident waves by deep reinforcement learning. The performance of the proposed control method is investigated through numerical simulation in both regular and irregular waves. Compared with the conventional control method, averaged output power increased, and the power fluctuation decreased without the dynamic model. It is understood that the proposed method is more effective than the conventional control method. |
| Author | 梅田, 隼 藤原, 敏文 |
| Author_xml | – sequence: 1 fullname: 藤原, 敏文 organization: 国立研究開発法人海上・港湾・航空技術研究所 海上技術安全研究所 – sequence: 1 fullname: 梅田, 隼 organization: 国立研究開発法人海上・港湾・航空技術研究所 海上技術安全研究所 |
| BookMark | eNo9kM1Kw0AcxBepYFs9-xSJ-99NspujFrVCwUvvYZNstKGmkvTizbaIYoWCB0HqqYqCYi8eBK34MEsa-xbGz8vMMANz-JVQIWpFEqFlwDoxqbEShiKJREvqFHRC7DlUBM5BA2bhwnfGGmXAFlApSUKMjXzkRVRNJ8_p2UX6eJu9n6vOg-qeqG5f9e5Vr6-6o3QymD6N0tNhdvkyG75-XB9lb2PVGec5L2fHg-nVYXpzlz8sovlANBO59OtlVN9Yr1eqWm17c6uyWtNCYoKhcQ9AMmbYkli-bZic-IJL4jPTpcCZQQKJLZv7YFFpA_M94kruWyxwAy5cQsto7ec2TNpiRzr7cWNPxAeOiNsNrymdPwoOBQd_SY7if_R2ReyEgn4Cor51FQ |
| ContentType | Journal Article |
| Copyright | 2020 社団法人 日本船舶海洋工学会 |
| Copyright_xml | – notice: 2020 社団法人 日本船舶海洋工学会 |
| DOI | 10.2534/jjasnaoe.31.229 |
| DatabaseTitleList | |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Military & Naval Science |
| EISSN | 1881-1760 |
| EndPage | 238 |
| ExternalDocumentID | article_jjasnaoe_31_0_31_229_article_char_ja |
| GroupedDBID | 2WC ABJNI ACGFS ALMA_UNASSIGNED_HOLDINGS JSF KQ8 OK1 RJT |
| ID | FETCH-LOGICAL-j2514-8c11e7749e26d94582da8e2d75b318742fe0698d163e917dc2be8d67fbf8ab23 |
| ISSN | 1880-3717 |
| IngestDate | Wed Sep 03 06:30:10 EDT 2025 |
| IsDoiOpenAccess | true |
| IsOpenAccess | true |
| IsPeerReviewed | false |
| IsScholarly | true |
| Language | Japanese |
| LinkModel | OpenURL |
| MergedId | FETCHMERGED-LOGICAL-j2514-8c11e7749e26d94582da8e2d75b318742fe0698d163e917dc2be8d67fbf8ab23 |
| OpenAccessLink | https://www.jstage.jst.go.jp/article/jjasnaoe/31/0/31_229/_article/-char/ja |
| PageCount | 10 |
| ParticipantIDs | jstage_primary_article_jjasnaoe_31_0_31_229_article_char_ja |
| PublicationCentury | 2000 |
| PublicationDate | 20200000 |
| PublicationDateYYYYMMDD | 2020-01-01 |
| PublicationDate_xml | – year: 2020 text: 20200000 |
| PublicationDecade | 2020 |
| PublicationTitle | 日本船舶海洋工学会論文集 |
| PublicationTitleAlternate | 日本船舶海洋工学会論文集 |
| PublicationYear | 2020 |
| Publisher | 公益社団法人 日本船舶海洋工学会 |
| Publisher_xml | – name: 公益社団法人 日本船舶海洋工学会 |
| References | Makino, T., Sibuya., T., et.al: A Introduction of Latest Reinforcement Learning, Morikita Publishing Co., Ltd., 2016 (in Japanese). 前田久明，山下誠也：海洋エネルギー利用＜特集＞1.2 波浪エネルギー一次変換装置，日本造船学会誌，第637号，pp. 306-326, 1982. 合田良実：数値シミュレーションによる波浪の標準スペクトルと統計的性質，海岸工学講演会論文集，第34 巻，pp.131-135, 1987. 谷口友基, 藤原敏文, 井上俊司, 大塚敏之, モデル予測制御による波力発電装置の高効率化, 日本船舶海洋工学会論文集，Vol. 29, pp. 171-179，2019. 10) Mikami, S., Minagawa, M., et.al: Reinforcement Learning, Morikita Publishing Co., Ltd., 2000 (in Japanese). Goda, Y.: Standard Spectrums and Statistical Properties of Sea waves Based on Numerical Simulations, Proceedings of the Japanese Conference on Coastal Engineering, Vol. 34, pp. 131-135, 1987 (in Japanese). 4) De La Villa Jaén, A., García-Santana, A. and Montoya-Andrade, D.E.: Maximizing Output Power of Linear Generators for Wave Energy Conversion, International Transactions on Electrical Energy Systems, Vol. 24, No. 6, pp. 875-890, 2014. 11) 三上貞芳，皆川雅章ほか：強化学習, 森北出版, 2000. 15) Mnih, V., et al.: Human-level control through deep reinforcement learning, Nature 518.7540 529, 2015. 9) Taniguchi, T., Fujiwara, T., Inoue, S. and Ohtsuka, T.: Power Production Efficiency Improvement of a Point Absorber Type Wave Energy Converter by Model Predictive Control, Journal of the Japan Society of Naval Architects and Ocean Engineers, Vol. 29, pp. 171-179, 2019 (in Japanese). 梅田隼，後藤博樹，藤原敏文，谷口友基，井上俊司：リニア式波力発電装置のモデル予測制御に関する研究，日本船舶海洋工学会論文集，Vol.28, pp. 27-36，2018. 12) Anderlini, E., Forehand, D. I., Bannon, E., and Abusara, M.: Control of a Realistic Wave Energy Converter Model Using Least-Squares Policy Iteration, IEEE Transactions on Sustainable Energy, 8(4), 1618-1628, 2017. 18) AKingma, Diederik P., and Ba, J.: Adam: A Method for Stochastic Optimization, arXiv preprint arXiv:1412.6980, 2014. 14) 牧野貴樹，澁谷長史ほか：これからの強化学習，森北出版，2016. 7) Umeda, J., Goto, H., Fujiwara, T., Taniguchi, T. and Inoue, S.: Study on Model Predictive Control for the Wave Energy Converter with a Linear Generator, Journal of the Japan Society of Naval Architects and Ocean Engineers, 2018, Vol. 28, pp. 27-36, 2018 (in Japanese). 5) Taniguchi, T., Umeda, J., Fujiwara, T., Goto, H., Inoue, S.: Experimental and numerical study on point absorber type wave energy converter with linear generator, ASME 2017 36th International Conference on Ocean, Offshore and Arctic Engineering, OMAE2017-61846, 2017. 2) New Energy and Industrial Technology Development Organization (NEDO): Project Report of “Development of the Next-generation ocean energy power generation technology Wave energy Converters with Linear generators”, 20180000000866, 2019. (in Japanese 3) Falnes, J.: Ocean Waves and Oscillating System, Cambridge University Press, Cambridge, 2002. 16) Wang, Z., Schaul, T., Hessel, M., van Hasselt, H., Lanctot, M., De Freitas, N.: Dueling Network Architectures for Deep Reinforcement Learning, arXiv preprint arXiv:1511.06581, 2015. 6) Hals, J., Falnes, J. and Moan, T.: Constrained optimal control of a heaving buoy wave-energy converter, Journal of Offshore Mechanics and Arctic Engineering, Vol.133, No.1, 011401, 2011. 1) Maeda, H. and Yamashita S.: Section 1.2 Wave Energy Converters, Bulletin of the Society of Naval Architects of Japan, Vol. 637, pp.306-327,1982. (in Japanese 13) Anderlini, E., Forehand, D. I., Bannon, E., and Abusara, M.: Constraints implementation in the application of reinforcement learning to the reactive control of a point absorber, Proceedings of the 36th International Conference on Ocean, Offshore and Arctic Engineering. American Society of Mechanical Engineers, OMAE2017-61294, 2017. 8) Nguyen, H.-N., Sabiron, G., Tona, P., Kramer, M. M. and Vidal Sanchez, E.: Experimental Validation of a Nonlinear MPC Strategy for a Wave Energy Converter Prototype, ASME 2016 35th International Conference on Ocean, Offshore and Arctic Engineering, OMAE2016-54455, 2016. 17) Lee, C.-H.: WAMIT Theory Manual., Report No. 95-2, Dept. of Ocean Engineering, Massachusetts Institute of Technology, 1995. 国立研究開発法人新エネルギー・産業技術総合開発機構：平成26年度～平成29年度成果報告書 海洋エネルギー技術研究開発 次世代海洋エネルギー発電技術開発 リニア式波力発電, 20180000000866, 2019. |
| References_xml | – reference: 谷口友基, 藤原敏文, 井上俊司, 大塚敏之, モデル予測制御による波力発電装置の高効率化, 日本船舶海洋工学会論文集，Vol. 29, pp. 171-179，2019. – reference: 16) Wang, Z., Schaul, T., Hessel, M., van Hasselt, H., Lanctot, M., De Freitas, N.: Dueling Network Architectures for Deep Reinforcement Learning, arXiv preprint arXiv:1511.06581, 2015. – reference: 12) Anderlini, E., Forehand, D. I., Bannon, E., and Abusara, M.: Control of a Realistic Wave Energy Converter Model Using Least-Squares Policy Iteration, IEEE Transactions on Sustainable Energy, 8(4), 1618-1628, 2017. – reference: 18) AKingma, Diederik P., and Ba, J.: Adam: A Method for Stochastic Optimization, arXiv preprint arXiv:1412.6980, 2014. – reference: 9) Taniguchi, T., Fujiwara, T., Inoue, S. and Ohtsuka, T.: Power Production Efficiency Improvement of a Point Absorber Type Wave Energy Converter by Model Predictive Control, Journal of the Japan Society of Naval Architects and Ocean Engineers, Vol. 29, pp. 171-179, 2019 (in Japanese). – reference: 8) Nguyen, H.-N., Sabiron, G., Tona, P., Kramer, M. M. and Vidal Sanchez, E.: Experimental Validation of a Nonlinear MPC Strategy for a Wave Energy Converter Prototype, ASME 2016 35th International Conference on Ocean, Offshore and Arctic Engineering, OMAE2016-54455, 2016. – reference: 13) Anderlini, E., Forehand, D. I., Bannon, E., and Abusara, M.: Constraints implementation in the application of reinforcement learning to the reactive control of a point absorber, Proceedings of the 36th International Conference on Ocean, Offshore and Arctic Engineering. American Society of Mechanical Engineers, OMAE2017-61294, 2017. – reference: 合田良実：数値シミュレーションによる波浪の標準スペクトルと統計的性質，海岸工学講演会論文集，第34 巻，pp.131-135, 1987. – reference: 1) Maeda, H. and Yamashita S.: Section 1.2 Wave Energy Converters, Bulletin of the Society of Naval Architects of Japan, Vol. 637, pp.306-327,1982. (in Japanese) – reference: 前田久明，山下誠也：海洋エネルギー利用＜特集＞1.2 波浪エネルギー一次変換装置，日本造船学会誌，第637号，pp. 306-326, 1982. – reference: 2) New Energy and Industrial Technology Development Organization (NEDO): Project Report of “Development of the Next-generation ocean energy power generation technology Wave energy Converters with Linear generators”, 20180000000866, 2019. (in Japanese) – reference: 11) 三上貞芳，皆川雅章ほか：強化学習, 森北出版, 2000. – reference: Goda, Y.: Standard Spectrums and Statistical Properties of Sea waves Based on Numerical Simulations, Proceedings of the Japanese Conference on Coastal Engineering, Vol. 34, pp. 131-135, 1987 (in Japanese). – reference: 梅田隼，後藤博樹，藤原敏文，谷口友基，井上俊司：リニア式波力発電装置のモデル予測制御に関する研究，日本船舶海洋工学会論文集，Vol.28, pp. 27-36，2018. – reference: 3) Falnes, J.: Ocean Waves and Oscillating System, Cambridge University Press, Cambridge, 2002. – reference: 7) Umeda, J., Goto, H., Fujiwara, T., Taniguchi, T. and Inoue, S.: Study on Model Predictive Control for the Wave Energy Converter with a Linear Generator, Journal of the Japan Society of Naval Architects and Ocean Engineers, 2018, Vol. 28, pp. 27-36, 2018 (in Japanese). – reference: 5) Taniguchi, T., Umeda, J., Fujiwara, T., Goto, H., Inoue, S.: Experimental and numerical study on point absorber type wave energy converter with linear generator, ASME 2017 36th International Conference on Ocean, Offshore and Arctic Engineering, OMAE2017-61846, 2017. – reference: 15) Mnih, V., et al.: Human-level control through deep reinforcement learning, Nature 518.7540 529, 2015. – reference: 4) De La Villa Jaén, A., García-Santana, A. and Montoya-Andrade, D.E.: Maximizing Output Power of Linear Generators for Wave Energy Conversion, International Transactions on Electrical Energy Systems, Vol. 24, No. 6, pp. 875-890, 2014. – reference: 14) 牧野貴樹，澁谷長史ほか：これからの強化学習，森北出版，2016. – reference: 10) Mikami, S., Minagawa, M., et.al: Reinforcement Learning, Morikita Publishing Co., Ltd., 2000 (in Japanese). – reference: Makino, T., Sibuya., T., et.al: A Introduction of Latest Reinforcement Learning, Morikita Publishing Co., Ltd., 2016 (in Japanese). – reference: 国立研究開発法人新エネルギー・産業技術総合開発機構：平成26年度～平成29年度成果報告書 海洋エネルギー技術研究開発 次世代海洋エネルギー発電技術開発 リニア式波力発電, 20180000000866, 2019. – reference: 17) Lee, C.-H.: WAMIT Theory Manual., Report No. 95-2, Dept. of Ocean Engineering, Massachusetts Institute of Technology, 1995. – reference: 6) Hals, J., Falnes, J. and Moan, T.: Constrained optimal control of a heaving buoy wave-energy converter, Journal of Offshore Mechanics and Arctic Engineering, Vol.133, No.1, 011401, 2011. |
| SSID | ssj0048818 ssib025353319 ssib031741110 ssib023160803 ssib000936973 ssib020472916 ssib002484520 |
| Score | 2.1825511 |
| Snippet | This paper presents a deep reinforcement learning control method to maximize output energy for a point absorber type wave energy converter (WEC) with a linear... |
| SourceID | jstage |
| SourceType | Publisher |
| StartPage | 229 |
| Title | 強化学習によるリニア式波力発電装置の電力量最大化 |
| URI | https://www.jstage.jst.go.jp/article/jjasnaoe/31/0/31_229/_article/-char/ja |
| Volume | 31 |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| ispartofPNX | 日本船舶海洋工学会論文集, 2020, Vol.31, pp.229-238 |
| link | http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwrR1Na9RANNR68SJ-Uj_Zg-NFsiaTj5nB02TNUpQWhAq9hWQze9hDK7U96Mm2iGKFggdB6qmKgmIvHgSt-BP8EWFb-y98bzLJxuLBqpfw9uXNvI9J8t6bnXljWZcYd1SqqGvnuchsPw8yW_g0t5UKudsPMzfVpZSmpsPJ2_6N2WB27ND3xqqlpcWs3bv_230lfzOqgINxxV2yBxjZulNAAAzjC1cYYbj-0RiTOCBRh0QMAd4hIkRAXicSAEaiLhGUxB5Ei0RGGqCE8wooMR6R0gC8opHU9My7JA5J5BkMl0RE2LMQJIJWAn9GgOFEQsNAMwXuccU0btDUzQXhzPQsOrjSAmX2iRxp0YyYNRkjMjD0soPsQItIVECohQy0HQDwtSL6J7aqDeKjRkJqaSNtqxCvIEwpJK8nKZEEefpaopiILj6Jmj6oJMeGI_oQLVQaQAAfp6QXyM6cHW5mVqhTvwW680ArxDR_iQAwjWK8BRgwaml-YIviQ2fyyn83SMMvwWcWfEG5zbWtKpyL9T2dpjMzLtV4IzOZVAY2tCyjs99n0sDz0WcO0rtz6bxqe267bvdLIXLzmCcVZeK5iYMXIE-qm7hXMBlAwnKYMubiGtubtxrxPh4fyZrxIfeDUb5CsXRpI1-B3CN0-GiZAIgK2cmo_iDEvr6r49cy9AJnpCf0a2uVtbxQw6v79IOAcwDpV7V0U0eTM8esoyYNbMlSn-PW2CA9YU1M6ZL5C_dal1vTKXwoWsbznrQmh9ufhk-fDz-82f32rFh-X6w8LlbWitV3xepasbI53F7f-bg5fLKx--Lz3saXH68e7n7dKpa3AAbk3qP1nZcPhq_fQg-nrJluPNOZtM0hKPYAUg_f5j3XVZCjCUXDXOC_3HnKFc1ZkHl4nibtKycUPIe8SgmX5T2aKZ6HrJ_1eZpR77Q1Pjc_pyasFktV2GNZ5ru91Pdxh3yvD-lN1g88PA9VnbGulSZJ7pSFbpKDjPjZf2p9zjqCb2A5t3neGl9cWFIXINpfzC7qJ-gnTYfOnQ |
| linkProvider | Colorado Alliance of Research Libraries |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=%E5%BC%B7%E5%8C%96%E5%AD%A6%E7%BF%92%E3%81%AB%E3%82%88%E3%82%8B%E3%83%AA%E3%83%8B%E3%82%A2%E5%BC%8F%E6%B3%A2%E5%8A%9B%E7%99%BA%E9%9B%BB%E8%A3%85%E7%BD%AE%E3%81%AE%E9%9B%BB%E5%8A%9B%E9%87%8F%E6%9C%80%E5%A4%A7%E5%8C%96&rft.jtitle=%E6%97%A5%E6%9C%AC%E8%88%B9%E8%88%B6%E6%B5%B7%E6%B4%8B%E5%B7%A5%E5%AD%A6%E4%BC%9A%E8%AB%96%E6%96%87%E9%9B%86&rft.au=%E8%97%A4%E5%8E%9F%2C+%E6%95%8F%E6%96%87&rft.au=%E6%A2%85%E7%94%B0%2C+%E9%9A%BC&rft.date=2020&rft.pub=%E5%85%AC%E7%9B%8A%E7%A4%BE%E5%9B%A3%E6%B3%95%E4%BA%BA+%E6%97%A5%E6%9C%AC%E8%88%B9%E8%88%B6%E6%B5%B7%E6%B4%8B%E5%B7%A5%E5%AD%A6%E4%BC%9A&rft.issn=1880-3717&rft.eissn=1881-1760&rft.volume=31&rft.spage=229&rft.epage=238&rft_id=info:doi/10.2534%2Fjjasnaoe.31.229&rft.externalDocID=article_jjasnaoe_31_0_31_229_article_char_ja |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=1880-3717&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=1880-3717&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=1880-3717&client=summon |