Multirobot Cooperative Learning for Predator Avoidance
Multirobot collaboration has great potentials in tasks, such as reconnaissance and surveillance. In this paper, we propose a multirobot system that integrates reinforcement learning and flocking control to allow robots to learn collaboratively to avoid predator/enemy. Our system can conduct concurre...
Saved in:
| Published in | IEEE transactions on control systems technology Vol. 23; no. 1; pp. 52 - 63 |
|---|---|
| Main Authors | , , |
| Format | Journal Article |
| Language | English |
| Published |
New York
IEEE
01.01.2015
The Institute of Electrical and Electronics Engineers, Inc. (IEEE) |
| Subjects | |
| Online Access | Get full text |
| ISSN | 1063-6536 1558-0865 |
| DOI | 10.1109/TCST.2014.2312392 |
Cover
Loading…
| Abstract | Multirobot collaboration has great potentials in tasks, such as reconnaissance and surveillance. In this paper, we propose a multirobot system that integrates reinforcement learning and flocking control to allow robots to learn collaboratively to avoid predator/enemy. Our system can conduct concurrent learning in a distributed fashion as well as generate efficient combination of high-level behaviors (discrete states and actions) and low-level behaviors (continuous states and actions) for multirobot cooperation. In addition, the combination of reinforcement learning and flocking control enables multirobot networks to learn how to avoid predators while maintaining network topology and connectivity. The convergence and scalability of the proposed system are investigated. Simulations and experiments are performed to demonstrate the effectiveness of the proposed system. |
|---|---|
| AbstractList | Multirobot collaboration has great potentials in tasks, such as reconnaissance and surveillance. In this paper, we propose a multirobot system that integrates reinforcement learning and flocking control to allow robots to learn collaboratively to avoid predator/enemy. Our system can conduct concurrent learning in a distributed fashion as well as generate efficient combination of high-level behaviors (discrete states and actions) and low-level behaviors (continuous states and actions) for multirobot cooperation. In addition, the combination of reinforcement learning and flocking control enables multirobot networks to learn how to avoid predators while maintaining network topology and connectivity. The convergence and scalability of the proposed system are investigated. Simulations and experiments are performed to demonstrate the effectiveness of the proposed system. |
| Author | Lim, Ronny Hung Manh La Weihua Sheng |
| Author_xml | – sequence: 1 surname: Hung Manh La fullname: Hung Manh La email: hung.la11@rutgers.edu organization: Center for Adv. Infrastruct. & Transp., Rutgers Univ., Piscataway, NJ, USA – sequence: 2 givenname: Ronny surname: Lim fullname: Lim, Ronny email: ronny.lim@rutgers.edu organization: Center for Adv. Infrastruct. & Transp., Rutgers Univ., Piscataway, NJ, USA – sequence: 3 surname: Weihua Sheng fullname: Weihua Sheng email: weihua.sheng@okstate.edu organization: Sch. of Electr. & Comput. Eng., Oklahoma State Univ., Stillwater, OK, USA |
| BookMark | eNp9kMFKAzEQhoNUsK0-gHhZ8Lw1k-wm2WNZtAoVBes5ZLMTSambmk0Lvr1bWjx48DT_4ftmmH9CRl3okJBroDMAWt2t6rfVjFEoZowD4xU7I2MoS5VTJcrRkKnguSi5uCCTvl_TgSyZHBPxvNskH0MTUlaHsMVokt9jtkQTO999ZC7E7DVia9IQ5vvgW9NZvCTnzmx6vDrNKXl_uF_Vj_nyZfFUz5e5LViV8paKRpbQcIOVYU42jStYwWmhLChlKrSyKp1SbcsUc8KC42icq8ByZNQJPiW3x73bGL522Ce9DrvYDSc1CK4KCYVkAyWPlI2h7yM6bX0a_ghdisZvNFB9KEkfStKHkvSppMGEP-Y2-k8Tv_91bo6OR8RfXkjFpQL-A6medDg |
| CODEN | IETTE2 |
| CitedBy_id | crossref_primary_10_1109_TCYB_2019_2944892 crossref_primary_10_1177_1729881420960342 crossref_primary_10_3390_s20164546 crossref_primary_10_1155_2022_4253558 crossref_primary_10_1007_s12555_021_0170_5 crossref_primary_10_1109_TCYB_2021_3119330 crossref_primary_10_1109_ACCESS_2020_2967225 crossref_primary_10_1016_j_sysarc_2023_102870 crossref_primary_10_1126_scirobotics_abd9285 crossref_primary_10_1016_j_arcontrol_2021_03_004 crossref_primary_10_1109_TCE_2023_3332587 crossref_primary_10_1109_TII_2019_2941916 crossref_primary_10_1016_j_neucom_2020_02_109 crossref_primary_10_1049_iet_cta_2017_1021 crossref_primary_10_1088_1674_1056_27_10_108901 crossref_primary_10_1088_2632_2153_ad1c33 crossref_primary_10_1109_TNNLS_2017_2770172 crossref_primary_10_1109_TVT_2024_3409687 crossref_primary_10_1109_TIM_2022_3227558 crossref_primary_10_1007_s11432_019_2735_6 crossref_primary_10_1109_LRA_2020_3013920 crossref_primary_10_1109_TSMC_2017_2789335 crossref_primary_10_1109_TSMC_2019_2931060 crossref_primary_10_1109_TRO_2020_2998766 crossref_primary_10_1109_TSMC_2017_2693400 crossref_primary_10_1109_TSMC_2018_2815988 crossref_primary_10_1016_j_isatra_2021_08_023 crossref_primary_10_1109_ACCESS_2019_2916004 crossref_primary_10_1109_TRO_2024_3359528 crossref_primary_10_1088_2632_2153_aca7b0 crossref_primary_10_1017_S0263574717000601 crossref_primary_10_3390_drones8090475 crossref_primary_10_1109_TNNLS_2021_3106777 crossref_primary_10_1109_JSSC_2019_2935533 crossref_primary_10_1016_j_paerosci_2022_100869 crossref_primary_10_1016_j_robot_2020_103594 crossref_primary_10_1002_rnc_5097 crossref_primary_10_1109_ACCESS_2018_2880485 crossref_primary_10_1142_S230138502050020X crossref_primary_10_1109_TCSS_2019_2958522 crossref_primary_10_1109_TCYB_2023_3246985 crossref_primary_10_1145_2910584 crossref_primary_10_1016_j_engappai_2018_12_001 crossref_primary_10_1016_j_ifacsc_2020_100096 crossref_primary_10_1016_j_arcontrol_2019_01_003 crossref_primary_10_1109_TCAD_2016_2524653 crossref_primary_10_1177_1729881418773874 crossref_primary_10_1007_s10846_015_0287_y crossref_primary_10_1145_3625544 crossref_primary_10_1109_TRO_2023_3339989 crossref_primary_10_1109_TCYB_2015_2509646 crossref_primary_10_2478_bipie_2023_0020 crossref_primary_10_1109_TCST_2022_3211130 crossref_primary_10_1016_j_pmcj_2018_10_007 crossref_primary_10_1109_TCST_2021_3109646 crossref_primary_10_1016_j_asoc_2020_106700 crossref_primary_10_1109_TSMC_2017_2702701 crossref_primary_10_1016_j_neucom_2018_02_059 crossref_primary_10_1177_1729881419847897 crossref_primary_10_1109_TCNS_2016_2542978 crossref_primary_10_1109_JSSC_2020_3028298 crossref_primary_10_1109_TCST_2023_3301141 crossref_primary_10_1209_0295_5075_acc270 crossref_primary_10_1016_j_automatica_2018_11_001 crossref_primary_10_1016_j_neucom_2023_126379 crossref_primary_10_1007_s11071_020_06062_y crossref_primary_10_1098_rsta_2024_0136 crossref_primary_10_1109_TCNS_2017_2754364 crossref_primary_10_1016_j_ins_2023_119074 crossref_primary_10_1109_ACCESS_2018_2827362 crossref_primary_10_3390_drones2040033 |
| Cites_doi | 10.1023/A:1022140919877 10.1109/TSMCB.2012.2215919 10.1109/TSMCB.2008.921000 10.1109/ICIEA.2011.5975893 10.1109/TSMCC.2007.913919 10.1515/9781400831470 10.1016/j.robot.2012.03.006 10.1007/978-3-540-85565-1_53 10.1007/s00285-003-0209-7 10.1109/TAC.2005.864190 10.1103/PhysRevLett.75.1226 10.1016/j.automatica.2009.09.025 10.1038/275642a0 10.1117/12.895519 10.1109/MRA.2011.942113 10.1145/37402.37406 10.1109/TSP.2010.2086450 10.1111/j.1095-8649.1998.tb02012.x 10.1016/j.physleta.2006.04.065 10.1016/j.automatica.2012.05.074 10.1006/jtbi.2002.3065 10.1016/B978-1-55860-377-6.50040-2 10.1109/IROS.2009.5354188 10.1103/PhysRevE.63.017101 10.1006/anbe.1996.0109 10.1007/s10846-010-9429-4 10.1109/JPROC.2006.887293 10.1109/TAC.2007.895948 10.1177/105971239700600201 10.1109/TAC.2008.2010897 10.1109/ROBOT.2009.5152747 10.1613/jair.301 |
| ContentType | Journal Article |
| Copyright | Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) Jan 2015 |
| Copyright_xml | – notice: Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) Jan 2015 |
| DBID | 97E RIA RIE AAYXX CITATION 7SP 7TB 8FD FR3 L7M |
| DOI | 10.1109/TCST.2014.2312392 |
| DatabaseName | IEEE All-Society Periodicals Package (ASPP) 2005–Present IEEE All-Society Periodicals Package (ASPP) 1998–Present IEL CrossRef Electronics & Communications Abstracts Mechanical & Transportation Engineering Abstracts Technology Research Database Engineering Research Database Advanced Technologies Database with Aerospace |
| DatabaseTitle | CrossRef Engineering Research Database Technology Research Database Mechanical & Transportation Engineering Abstracts Advanced Technologies Database with Aerospace Electronics & Communications Abstracts |
| DatabaseTitleList | Engineering Research Database |
| Database_xml | – sequence: 1 dbid: RIE name: IEEE Electronic Library (IEL) url: https://proxy.k.utb.cz/login?url=https://ieeexplore.ieee.org/ sourceTypes: Publisher |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Engineering |
| EISSN | 1558-0865 |
| EndPage | 63 |
| ExternalDocumentID | 3532201161 10_1109_TCST_2014_2312392 6783781 |
| Genre | orig-research |
| GrantInformation_xml | – fundername: U.S. Department of Defense through the DoD ARO DURIP grantid: 55628-CS-RIP – fundername: DEPSCoR grantid: W911NF-10-1-0015 – fundername: Vietnamese Government through the 322 Project, Ministry of Education and Training – fundername: NSF grantid: CISE/IIS 1231671 |
| GroupedDBID | -~X .DC 0R~ 29I 4.4 5GY 5VS 6IK 97E AAJGR AARMG AASAJ AAWTH ABAZT ABQJQ ABVLG ACBEA ACGFO ACGFS ACIWK ACKIV AENEX AETIX AGQYO AGSQL AHBIQ AI. AIBXA AKJIK AKQYR ALLEH ALMA_UNASSIGNED_HOLDINGS ATWAV BEFXN BFFAM BGNUA BKEBE BPEOZ CS3 DU5 EBS EJD HZ~ H~9 ICLAB IFIPE IFJZH IPLJI JAVBF LAI M43 O9- OCL P2P RIA RIE RNS TN5 VH1 AAYOK AAYXX CITATION RIG 7SP 7TB 8FD FR3 L7M |
| ID | FETCH-LOGICAL-c429t-d06b751b3ae9a2f7bbf4243048c188a9ec795f88dd282f6c1f3eaff91c3e20f63 |
| IEDL.DBID | RIE |
| ISSN | 1063-6536 |
| IngestDate | Sun Jun 29 16:17:11 EDT 2025 Thu Apr 24 23:02:54 EDT 2025 Tue Jul 01 01:28:12 EDT 2025 Wed Aug 27 02:55:08 EDT 2025 |
| IsPeerReviewed | true |
| IsScholarly | true |
| Issue | 1 |
| Keywords | Flocking control hybrid system multirobot systems reinforcement learning |
| Language | English |
| License | https://ieeexplore.ieee.org/Xplorehelp/downloads/license-information/IEEE.html |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-c429t-d06b751b3ae9a2f7bbf4243048c188a9ec795f88dd282f6c1f3eaff91c3e20f63 |
| Notes | ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 14 |
| PQID | 1638471472 |
| PQPubID | 85425 |
| PageCount | 12 |
| ParticipantIDs | crossref_citationtrail_10_1109_TCST_2014_2312392 ieee_primary_6783781 crossref_primary_10_1109_TCST_2014_2312392 proquest_journals_1638471472 |
| ProviderPackageCode | CITATION AAYXX |
| PublicationCentury | 2000 |
| PublicationDate | 2015-01-01 |
| PublicationDateYYYYMMDD | 2015-01-01 |
| PublicationDate_xml | – month: 01 year: 2015 text: 2015-01-01 day: 01 |
| PublicationDecade | 2010 |
| PublicationPlace | New York |
| PublicationPlace_xml | – name: New York |
| PublicationTitle | IEEE transactions on control systems technology |
| PublicationTitleAbbrev | TCST |
| PublicationYear | 2015 |
| Publisher | IEEE The Institute of Electrical and Electronics Engineers, Inc. (IEEE) |
| Publisher_xml | – name: IEEE – name: The Institute of Electrical and Electronics Engineers, Inc. (IEEE) |
| References | ref35 ref34 ref37 ref15 la (ref16) 2010 ref36 ref14 ref31 ref11 ref10 ref2 luo (ref12) 2014 ref1 (ref40) 2011 ref17 ref19 ref18 ren (ref8) 2005 sutton (ref13) 1998 ref24 ref23 (ref33) 2011 ref26 ref25 ref20 ref41 ref22 ref21 la (ref42) 2012; 43 coggan (ref39) 2001 ref28 ref27 gaskett (ref29) 1999 kaelbling (ref38) 1996; 4 ref7 smart (ref30) 2000 ref9 ref4 ref3 ref6 ref5 la (ref32) 2010 |
| References_xml | – start-page: 903 year: 2000 ident: ref30 article-title: Practical reinforcement learning in continuous spaces publication-title: Proc Int Conf Mach Learn – ident: ref37 doi: 10.1023/A:1022140919877 – volume: 43 start-page: 766 year: 2012 ident: ref42 article-title: Distributed sensor fusion for scalar field mapping using mobile sensor networks publication-title: IEEE Trans Cybern doi: 10.1109/TSMCB.2012.2215919 – start-page: 7 year: 2010 ident: ref32 article-title: Hybrid system of reinforcement learning and flocking control in multi-robot domain publication-title: Proc Conf Theoretical Comput Sci – ident: ref31 doi: 10.1109/TSMCB.2008.921000 – ident: ref25 doi: 10.1109/ICIEA.2011.5975893 – ident: ref14 doi: 10.1109/TSMCC.2007.913919 – ident: ref34 doi: 10.1515/9781400831470 – ident: ref17 doi: 10.1016/j.robot.2012.03.006 – ident: ref36 doi: 10.1007/978-3-540-85565-1_53 – ident: ref21 doi: 10.1007/s00285-003-0209-7 – ident: ref11 doi: 10.1109/TAC.2005.864190 – ident: ref19 doi: 10.1103/PhysRevLett.75.1226 – ident: ref1 doi: 10.1016/j.automatica.2009.09.025 – start-page: 1 year: 2001 ident: ref39 article-title: Exploration and exploitation in reinforcement learning publication-title: Proc 4th Int Conf Comput Intell and Multimedia Appl – ident: ref5 doi: 10.1038/275642a0 – ident: ref41 doi: 10.1117/12.895519 – ident: ref4 doi: 10.1109/MRA.2011.942113 – ident: ref18 doi: 10.1145/37402.37406 – ident: ref10 doi: 10.1109/TSP.2010.2086450 – ident: ref7 doi: 10.1111/j.1095-8649.1998.tb02012.x – ident: ref35 doi: 10.1016/j.physleta.2006.04.065 – start-page: 4964 year: 2010 ident: ref16 article-title: Flocking control of multiple agents in noisy environments publication-title: Proc IEEE Int Conf Robotics Autom – year: 1998 ident: ref13 publication-title: Reinforcement Learning An Introduction – start-page: 417 year: 1999 ident: ref29 publication-title: $Q$-Learning in continuous state and action spaces – ident: ref2 doi: 10.1016/j.automatica.2012.05.074 – ident: ref22 doi: 10.1006/jtbi.2002.3065 – ident: ref28 doi: 10.1016/B978-1-55860-377-6.50040-2 – ident: ref3 doi: 10.1109/IROS.2009.5354188 – ident: ref20 doi: 10.1103/PhysRevE.63.017101 – year: 2011 ident: ref33 publication-title: Rovio Robot – start-page: 1859 year: 2005 ident: ref8 article-title: A survey of consensus problems in multi-agent coordination publication-title: Proc Amer Control Conf – year: 2014 ident: ref12 article-title: Off-policy reinforcement learning for H? control design publication-title: Preprint arXiv 1311 6107 – ident: ref6 doi: 10.1006/anbe.1996.0109 – ident: ref26 doi: 10.1007/s10846-010-9429-4 – ident: ref9 doi: 10.1109/JPROC.2006.887293 – ident: ref23 doi: 10.1109/TAC.2007.895948 – ident: ref27 doi: 10.1177/105971239700600201 – ident: ref24 doi: 10.1109/TAC.2008.2010897 – ident: ref15 doi: 10.1109/ROBOT.2009.5152747 – volume: 4 start-page: 237 year: 1996 ident: ref38 article-title: Reinforcement learning: A survey publication-title: J Artif Intell Res doi: 10.1613/jair.301 – year: 2011 ident: ref40 publication-title: Vicon Motion Capture System |
| SSID | ssj0014527 |
| Score | 2.4887118 |
| Snippet | Multirobot collaboration has great potentials in tasks, such as reconnaissance and surveillance. In this paper, we propose a multirobot system that integrates... |
| SourceID | proquest crossref ieee |
| SourceType | Aggregation Database Enrichment Source Index Database Publisher |
| StartPage | 52 |
| SubjectTerms | Aerospace electronics Collision avoidance Flocking control hybrid system Learning (artificial intelligence) multirobot systems Network topology reinforcement learning Robot kinematics Robot sensing systems |
| Title | Multirobot Cooperative Learning for Predator Avoidance |
| URI | https://ieeexplore.ieee.org/document/6783781 https://www.proquest.com/docview/1638471472 |
| Volume | 23 |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| link | http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwjV1LS8NAEB5qT3rwVcVolRw8iWmT7GaTHEuxFKEi2EJvIfsSUZpSUw_-emc3afCFeMthNywzs_PamW8ALn2OOo-IxMtDDFcpF8rjGAt5kWZCCkq1bxNukzs2ntHbeTRvwXXTC6OUssVnqmc-7Vu-LMTapMr6qFhJbPqst1DMql6t5sWAVuNZMcIhHrNPkk6Np9mfDh-mpoiL9tCZCUkafrFBdqjKD01szctoDyabg1VVJc-9dcl74v0bZuN_T74Pu7Wf6Q4qwTiAllocws4n9MEOMNt8uyp4UbrDoliqCgTcrSFXH130Z937lZImLncHb8WTNCJyBLPRzXQ49uoxCp5AY1N60mc8jgJOcpXmoY451zSkBK-uCJIkT5WI00gniZQYfiGLAk1UrnUaCKJCXzNyDO1FsVAn4EoS5yzSQuUkMJY_pehAcJJEUjGekMABf0PYTNQY42bUxUtmYw0_zQwvMsOLrOaFA1fNlmUFsPHX4o6hbbOwJqsD3Q33svoKvmbG0UTLS-Pw9PddZ7CN_46qfEoX2uVqrc7Rwyj5hRWtD8g1y6g |
| linkProvider | IEEE |
| linkToHtml | http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwjV1LS8NAEB6KHtSDrypGq-bgSUybZDevYymWqm0RTKG3JfsSUZpSUw_-enc3afCFeMthlywzs_PamW8ALlyqdB5isZP5KlzFlAmHqljICWTIOMNYuibhNhqHgwm-nQbTBlzVvTBCCFN8Jtr607zl85wtdaqsoxQrinSf9bqy-zgou7XqNwNcDmhVMQ5yQvMoaVWImp2095DqMi7cVu6MjxL_ixUyY1V-6GJjYPo7MFodrawreW4vC9pm799QG_979l3YrjxNu1uKxh40xGwftj7hDzYhNO23i5zmhd3L87koYcDtCnT10VYerX2_EFxH5nb3LX_iWkgOYNK_TnsDpxqk4DBlbgqHuyGNAo-iTCSZLyNKJfYxUpeXeXGcJYJFSSDjmHMVgCkmeRKJTMrEY0j4rgzRIazN8pk4ApujKAsDyUSGPG37E6xcCIrigIuQxsizwF0RlrAKZVwPu3ghJtpwE6J5QTQvSMULCy7rLfMSYuOvxU1N23phRVYLWivukeoSvhLtairbiyP_-Pdd57AxSEdDMrwZ353ApvpPUGZXWrBWLJbiVPkbBT0zYvYBn7zO9Q |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Multirobot+Cooperative+Learning+for+Predator+Avoidance&rft.jtitle=IEEE+transactions+on+control+systems+technology&rft.au=La%2C+Hung+Manh&rft.au=Lim%2C+Ronny&rft.au=Sheng%2C+Weihua&rft.date=2015-01-01&rft.pub=The+Institute+of+Electrical+and+Electronics+Engineers%2C+Inc.+%28IEEE%29&rft.issn=1063-6536&rft.eissn=1558-0865&rft.volume=23&rft.issue=1&rft.spage=52&rft_id=info:doi/10.1109%2FTCST.2014.2312392&rft.externalDBID=NO_FULL_TEXT&rft.externalDocID=3532201161 |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=1063-6536&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=1063-6536&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=1063-6536&client=summon |