Stochastic Optimal Control for Robot Manipulation Skill Learning Under Time-Varying Uncertain Environment

In this article, a novel stochastic optimal control method is developed for robot manipulator interacting with a time-varying uncertain environment. The unknown environment model is described as a nonlinear system with time-varying parameters as well as stochastic information, which is learned via t...

Full description

Saved in:
Bibliographic Details
Published inIEEE transactions on cybernetics Vol. 54; no. 4; pp. 2015 - 2025
Main Authors Liu, Xing, Liu, Zhengxiong, Huang, Panfeng
Format Journal Article
LanguageEnglish
Published United States IEEE 01.04.2024
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subjects
Control methods
Environment models
Feedforward control
Gaussian process
Gaussian processes
Impedance
iterative linear quadratic Gaussian with learned external dynamic (ILQG-LED) method
Iterative methods
Manipulator dynamics
model-based reinforcement learning
Nonlinear systems
Optimal control
Parameters
Robot arms
Robot control
robot manipulation skill
robot-environment interaction
Robots
stochastic optimal manipulation control
Stochastic processes
System dynamics
Task complexity
Time-varying systems
time-varying uncertain environment
Trajectory
Unknown environments
Online AccessGet full text
ISSN2168-2267
2168-2275
2168-2275
DOI10.1109/TCYB.2022.3211440

Cover

Abstract In this article, a novel stochastic optimal control method is developed for robot manipulator interacting with a time-varying uncertain environment. The unknown environment model is described as a nonlinear system with time-varying parameters as well as stochastic information, which is learned via the Gaussian process regression (GPR) method as the external dynamics. Integrating the learned external dynamics as well as the stochastic uncertainties, the complete interaction system dynamics are obtained. Then the iterative linear quadratic Gaussian with learned external dynamics (ILQG-LEDs) method is presented to obtain the optimal manipulation control parameters, namely, the feedforward force, the reference trajectory, as well as the impedance parameters, subject to time-varying environment dynamics. The comparative simulation studies verify the advantages of the presented method, and the experimental studies of the peg-hole-insertion task prove that this method can deal with complex manipulation tasks.
AbstractList In this article, a novel stochastic optimal control method is developed for robot manipulator interacting with a time-varying uncertain environment. The unknown environment model is described as a nonlinear system with time-varying parameters as well as stochastic information, which is learned via the Gaussian process regression (GPR) method as the external dynamics. Integrating the learned external dynamics as well as the stochastic uncertainties, the complete interaction system dynamics are obtained. Then the iterative linear quadratic Gaussian with learned external dynamics (ILQG-LEDs) method is presented to obtain the optimal manipulation control parameters, namely, the feedforward force, the reference trajectory, as well as the impedance parameters, subject to time-varying environment dynamics. The comparative simulation studies verify the advantages of the presented method, and the experimental studies of the peg-hole-insertion task prove that this method can deal with complex manipulation tasks.In this article, a novel stochastic optimal control method is developed for robot manipulator interacting with a time-varying uncertain environment. The unknown environment model is described as a nonlinear system with time-varying parameters as well as stochastic information, which is learned via the Gaussian process regression (GPR) method as the external dynamics. Integrating the learned external dynamics as well as the stochastic uncertainties, the complete interaction system dynamics are obtained. Then the iterative linear quadratic Gaussian with learned external dynamics (ILQG-LEDs) method is presented to obtain the optimal manipulation control parameters, namely, the feedforward force, the reference trajectory, as well as the impedance parameters, subject to time-varying environment dynamics. The comparative simulation studies verify the advantages of the presented method, and the experimental studies of the peg-hole-insertion task prove that this method can deal with complex manipulation tasks.
In this article, a novel stochastic optimal control method is developed for robot manipulator interacting with a time-varying uncertain environment. The unknown environment model is described as a nonlinear system with time-varying parameters as well as stochastic information, which is learned via the Gaussian process regression (GPR) method as the external dynamics. Integrating the learned external dynamics as well as the stochastic uncertainties, the complete interaction system dynamics are obtained. Then the iterative linear quadratic Gaussian with learned external dynamics (ILQG-LEDs) method is presented to obtain the optimal manipulation control parameters, namely, the feedforward force, the reference trajectory, as well as the impedance parameters, subject to time-varying environment dynamics. The comparative simulation studies verify the advantages of the presented method, and the experimental studies of the peg-hole-insertion task prove that this method can deal with complex manipulation tasks.
Author Liu, Xing
Liu, Zhengxiong
Huang, Panfeng
Author_xml – sequence: 1
  givenname: Xing
  orcidid: 0000-0002-5327-4908
  surname: Liu
  fullname: Liu, Xing
  email: xingliu@nwpu.edu.cn
  organization: Research Center for Intelligent Robotics, School of Astronautics, and the National Key Laboratory of Aerospace Flight Dynamics, Northwestern Polytechnical University, Xi'an, China
– sequence: 2
  givenname: Zhengxiong
  orcidid: 0000-0002-9427-4066
  surname: Liu
  fullname: Liu, Zhengxiong
  organization: Research Center for Intelligent Robotics, School of Astronautics, and the National Key Laboratory of Aerospace Flight Dynamics, Northwestern Polytechnical University, Xi'an, China
– sequence: 3
  givenname: Panfeng
  orcidid: 0000-0002-5132-9602
  surname: Huang
  fullname: Huang, Panfeng
  email: pfhuang@nwpu.edu.cn
  organization: Research Center for Intelligent Robotics, School of Astronautics, and the National Key Laboratory of Aerospace Flight Dynamics, Northwestern Polytechnical University, Xi'an, China
BackLink https://www.ncbi.nlm.nih.gov/pubmed/36256715$$D View this record in MEDLINE/PubMed
BookMark eNp9kU1vFCEcxompsS_2AxgTQ9KLl1l5mYHhqJtqm6xpYrcmnggwjFIZWIFp0m9f1l330INcIP_8ngd4nlNwFGKwALzBaIExEh_Wyx-fFgQRsqAE47ZFL8AJwaxvCOHd0eHM-DE4z_ke1dXXkehfgWPKSMc47k6Auy3R_FK5OANvNsVNysNlDCVFD8eY4LeoY4FfVXCb2aviYoC3v533cGVVCi78hHdhsAmu3WSb7yo97kbGpqJcgJfhwaUYJhvKa_ByVD7b8_1-Bu4-X66XV83q5sv18uOqMbQVpSHCMDzUH2LNNNVCE90xxaimI7dCdNhQ3XGmsWCCEtszZCltcUvUSIduUPQMvN_5blL8M9tc5OSysd6rYOOcJeGEtajv276iF8_Q-zinUF8nSbVHLa7WlXq3p2Y92UFuUk0pPcp_IVYA7wCTYs7JjgcEI7ntSm67ktuu5L6rquHPNMaVv_mWpJz_r_LtTumstYebhCC045w-ASVan8Q
CODEN ITCEB8
CitedBy_id crossref_primary_10_1109_TIE_2022_3227279
crossref_primary_10_1109_TCYB_2024_3436021
crossref_primary_10_1109_TSMC_2024_3514154
crossref_primary_10_1109_TASE_2024_3469961
crossref_primary_10_1016_j_compeleceng_2024_109605
Cites_doi 10.1080/00207179.2013.827799
10.1109/ICRA.2012.6224586
10.1109/TSMCB.2010.2043839
10.1109/TASE.2020.2983225
10.1109/TCST.2013.2286194
10.1177/0278364911402527
10.1109/TII.2020.3036693
10.1109/TASE.2020.3045655
10.1109/DEVLRN.2011.6037312
10.1109/TMECH.2020.3047919
10.1109/TRO.2015.2419873
10.1109/TAMD.2012.2205924
10.1109/IROS.2011.6094877
10.1109/TRO.2007.892229
10.1126/science.aat8414
10.1109/TCYB.2020.2998984
10.1109/IROS.2016.7759417
10.1109/TCYB.2018.2828654
10.3389/frobt.2016.00030
10.1109/TRO.2018.2830405
10.1109/TSMC.2019.2920870
10.1109/ECC.2015.7330913
10.1007/978-3-642-05181-4_4
10.1109/TRO.2016.2597322
10.1109/TCST.2020.2971944
10.1109/TNNLS.2014.2378812
10.1109/ACC.2005.1469949
ContentType Journal Article
Copyright Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2024
Copyright_xml – notice: Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2024
DBID 97E
RIA
RIE
AAYXX
CITATION
NPM
7SC
7SP
7TB
8FD
F28
FR3
H8D
JQ2
L7M
L~C
L~D
7X8
DOI 10.1109/TCYB.2022.3211440
DatabaseName IEEE Xplore (IEEE)
IEEE All-Society Periodicals Package (ASPP) 1998–Present
IEEE Xplore Electronic Library (IEL)
CrossRef
PubMed
Computer and Information Systems Abstracts
Electronics & Communications Abstracts
Mechanical & Transportation Engineering Abstracts
Technology Research Database
ANTE: Abstracts in New Technology & Engineering
Engineering Research Database
Aerospace Database
ProQuest Computer Science Collection
Advanced Technologies Database with Aerospace
Computer and Information Systems Abstracts – Academic
Computer and Information Systems Abstracts Professional
MEDLINE - Academic
DatabaseTitle CrossRef
PubMed
Aerospace Database
Technology Research Database
Computer and Information Systems Abstracts – Academic
Mechanical & Transportation Engineering Abstracts
Electronics & Communications Abstracts
ProQuest Computer Science Collection
Computer and Information Systems Abstracts
Engineering Research Database
Advanced Technologies Database with Aerospace
ANTE: Abstracts in New Technology & Engineering
Computer and Information Systems Abstracts Professional
MEDLINE - Academic
DatabaseTitleList MEDLINE - Academic

PubMed
Aerospace Database
Database_xml – sequence: 1
  dbid: NPM
  name: PubMed
  url: https://proxy.k.utb.cz/login?url=http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed
  sourceTypes: Index Database
– sequence: 2
  dbid: RIE
  name: IEEE Xplore Electronic Library (IEL)
  url: https://proxy.k.utb.cz/login?url=https://ieeexplore.ieee.org/
  sourceTypes: Publisher
DeliveryMethod fulltext_linktorsrc
Discipline Sciences (General)
EISSN 2168-2275
EndPage 2025
ExternalDocumentID 36256715
10_1109_TCYB_2022_3211440
9923577
Genre orig-research
Journal Article
GrantInformation_xml – fundername: National Natural Science Foundation of China
  grantid: 61725303; 62103334
  funderid: 10.13039/501100001809
– fundername: China Postdoctoral Science Foundation
  grantid: 2021M702669
  funderid: 10.13039/501100002858
GroupedDBID 0R~
4.4
6IK
97E
AAJGR
AARMG
AASAJ
AAWTH
ABAZT
ABQJQ
ABVLG
ACIWK
AENEX
AGQYO
AGSQL
AHBIQ
AKJIK
AKQYR
ALMA_UNASSIGNED_HOLDINGS
ATWAV
BEFXN
BFFAM
BGNUA
BKEBE
BPEOZ
EBS
EJD
HZ~
IFIPE
IPLJI
JAVBF
M43
O9-
OCL
PQQKQ
RIA
RIE
RNS
AAYXX
CITATION
RIG
NPM
7SC
7SP
7TB
8FD
F28
FR3
H8D
JQ2
L7M
L~C
L~D
7X8
ID FETCH-LOGICAL-c349t-29c61d1101b6b3b9b2b56a63b3f7e9951c3b576b196932e860e334142af3d5da3
IEDL.DBID RIE
ISSN 2168-2267
2168-2275
IngestDate Fri Jul 11 07:58:42 EDT 2025
Mon Jun 30 04:11:27 EDT 2025
Mon Jul 21 06:07:44 EDT 2025
Thu Apr 24 23:10:57 EDT 2025
Tue Jul 01 00:54:03 EDT 2025
Wed Aug 27 03:03:27 EDT 2025
IsPeerReviewed true
IsScholarly true
Issue 4
Language English
License https://ieeexplore.ieee.org/Xplorehelp/downloads/license-information/IEEE.html
https://doi.org/10.15223/policy-029
https://doi.org/10.15223/policy-037
LinkModel DirectLink
MergedId FETCHMERGED-LOGICAL-c349t-29c61d1101b6b3b9b2b56a63b3f7e9951c3b576b196932e860e334142af3d5da3
Notes ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
content type line 23
ORCID 0000-0002-9427-4066
0000-0002-5132-9602
0000-0002-5327-4908
PMID 36256715
PQID 2969041334
PQPubID 85422
PageCount 11
ParticipantIDs ieee_primary_9923577
proquest_journals_2969041334
crossref_primary_10_1109_TCYB_2022_3211440
crossref_citationtrail_10_1109_TCYB_2022_3211440
pubmed_primary_36256715
proquest_miscellaneous_2726408848
ProviderPackageCode CITATION
AAYXX
PublicationCentury 2000
PublicationDate 2024-04-01
PublicationDateYYYYMMDD 2024-04-01
PublicationDate_xml – month: 04
  year: 2024
  text: 2024-04-01
  day: 01
PublicationDecade 2020
PublicationPlace United States
PublicationPlace_xml – name: United States
– name: Piscataway
PublicationTitle IEEE transactions on cybernetics
PublicationTitleAbbrev TCYB
PublicationTitleAlternate IEEE Trans Cybern
PublicationYear 2024
Publisher IEEE
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Publisher_xml – name: IEEE
– name: The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
References ref13
ref12
ref15
ref14
ref31
ref30
Wang (ref29) 2019
ref10
Deisenroth (ref18) 2010; 9
ref2
ref1
ref17
ref16
ref19
ref24
ref23
ref26
ref25
ref20
ref22
ref21
ref28
ref27
ref8
ref7
ref9
ref3
ref6
ref5
Hogan (ref4) 1985; 107
Li (ref11)
References_xml – ident: ref8
  doi: 10.1080/00207179.2013.827799
– ident: ref6
  doi: 10.1109/ICRA.2012.6224586
– ident: ref26
  doi: 10.1109/TSMCB.2010.2043839
– ident: ref5
  doi: 10.1109/TASE.2020.2983225
– ident: ref20
  doi: 10.1109/TCST.2013.2286194
– ident: ref15
  doi: 10.1177/0278364911402527
– ident: ref31
  doi: 10.1109/TII.2020.3036693
– ident: ref30
  doi: 10.1109/TASE.2020.3045655
– ident: ref14
  doi: 10.1109/DEVLRN.2011.6037312
– ident: ref28
  doi: 10.1109/TMECH.2020.3047919
– ident: ref21
  doi: 10.1109/TRO.2015.2419873
– ident: ref3
  doi: 10.1109/TAMD.2012.2205924
– ident: ref16
  doi: 10.1109/IROS.2011.6094877
– ident: ref27
  doi: 10.1109/TRO.2007.892229
– start-page: 222
  volume-title: Proc. ICINCO
  ident: ref11
  article-title: Iterative linear quadratic regulator design for nonlinear biological movement systems
– volume: 9
  volume-title: Efficient Reinforcement Learning Using Gaussian Processes
  year: 2010
  ident: ref18
– ident: ref1
  doi: 10.1126/science.aat8414
– ident: ref13
  doi: 10.1109/TCYB.2020.2998984
– ident: ref24
  doi: 10.1109/IROS.2016.7759417
– ident: ref10
  doi: 10.1109/TCYB.2018.2828654
– ident: ref25
  doi: 10.3389/frobt.2016.00030
– volume: 107
  start-page: 304
  issue: 1
  year: 1985
  ident: ref4
  article-title: Impedance control: An approach to manipulation, part I—Theory, part II—Implementation, part III—Applications
  publication-title: ASME Trans. J. Dyn. Syst. Meas. Control B
– ident: ref23
  doi: 10.1109/TRO.2018.2830405
– volume-title: arXiv:1907.02057
  year: 2019
  ident: ref29
  article-title: Benchmarking model-based reinforcement learning
– ident: ref7
  doi: 10.1109/TSMC.2019.2920870
– ident: ref19
  doi: 10.1109/ECC.2015.7330913
– ident: ref17
  doi: 10.1007/978-3-642-05181-4_4
– ident: ref22
  doi: 10.1109/TRO.2016.2597322
– ident: ref9
  doi: 10.1109/TCST.2020.2971944
– ident: ref2
  doi: 10.1109/TNNLS.2014.2378812
– ident: ref12
  doi: 10.1109/ACC.2005.1469949
SSID ssj0000816898
Score 2.3877468
Snippet In this article, a novel stochastic optimal control method is developed for robot manipulator interacting with a time-varying uncertain environment. The...
SourceID proquest
pubmed
crossref
ieee
SourceType Aggregation Database
Index Database
Enrichment Source
Publisher
StartPage 2015
SubjectTerms Control methods
Environment models
Feedforward control
Gaussian process
Gaussian processes
Impedance
iterative linear quadratic Gaussian with learned external dynamic (ILQG-LED) method
Iterative methods
Manipulator dynamics
model-based reinforcement learning
Nonlinear systems
Optimal control
Parameters
Robot arms
Robot control
robot manipulation skill
robot-environment interaction
Robots
stochastic optimal manipulation control
Stochastic processes
System dynamics
Task complexity
Time-varying systems
time-varying uncertain environment
Trajectory
Unknown environments
Title Stochastic Optimal Control for Robot Manipulation Skill Learning Under Time-Varying Uncertain Environment
URI https://ieeexplore.ieee.org/document/9923577
https://www.ncbi.nlm.nih.gov/pubmed/36256715
https://www.proquest.com/docview/2969041334
https://www.proquest.com/docview/2726408848
Volume 54
hasFullText 1
inHoldings 1
isFullTextHit
isPrint
link http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwjV1LT9wwEB4Bp15oKX1sgcqVemgrvHht53VsVyBUaUEqu4ieIttx6IptgiB7gF_P2HECQlD1ZsV2YmfGnhl7Zj6Az1aWOklLS5GDIyrTWNBUpopK5B1tUT4w6QKFJ0fx4Uz-PIvOVmC3j4Wx1nrnMzt0RX-XX9Rm6Y7K9rLMJWdJVmEVDbc2Vqs_T_EAEh76lmOBolaRhEvMEcv2puPfP9AY5Hwo0OKR0gHA4dYdxYnDw30gkTzEyvPappc6By9h0o23dTa5GC4bPTS3j1I5_u-EXsF6UD_J95ZfNmDFVq9hIyzwa_IlZKH-ugnzk6Y2f5RL40yOcV_5i93GrV87QUWX_Kp13ZCJquYdAhg5uZgvFiRkbD0nHlKJuBgTeqqubtpHpnVBIPv3AXZvYHawPx0f0oDLQI2QWUN5ZuJRgf91pGMtdKa5jmIVCy3KxGaoshmh0YzRLvOO4DaNmRUoLCVXpSiiQom3sFbVlX0PRCtsppNCiBLrWalGhUYVjVvFeGFYOgDW0SY3IWm5w85Y5N54YVnuKJs7yuaBsgP41ne5bDN2_KvxpqNK3zAQZADbHQPkYU1f5xynw1DmCzmAT301rkZ3xaIqWy-xTYKjR7aXOPJ3LeP07-747cPT39yCFziy4BW0DWvN1dLuoMLT6I-e0-8Abgz4dg
linkProvider IEEE
linkToHtml http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwjV1Lb9QwEB6VcoAL9MFjSylG4gAIb7228zqWVauldItEt6icIttx2lW3CWqzB_j1jB0nRQgQNyu2k3Fm7BnbM_MBvLKy1ElaWooSHFGZxoKmMlVUouxoi_qBSRcoPD2OJ6fy8Cw6W4F3fSyMtdY7n9mhK_q7_KI2S3dUtptlLjlLcgfuRi4Yt43W6k9UPISEB7_lWKBoVyThGnPEst3Z-Ot73A5yPhS455HSQcDh4h3FiUPE_UUneZCVv9ubXu8cPIRpR3HrbnI5XDZ6aH78lszxf4e0Bg-CAUr2WolZhxVbbcB6mOI35HXIQ_1mE-YnTW0ulEvkTD7hynKF3catZztBU5d8rnXdkKmq5h0GGDm5nC8WJORsPSceVIm4KBP6RV1_bx-Z1gmB7N-G2D2C04P92XhCAzIDNUJmDeWZiUcF_teRjrXQmeY6ilUstCgTm6HRZoTGjYx2uXcEt2nMrEB1KbkqRREVSjyG1aqu7FMgWmEznRRClFjPSjUqNBpp3CrGC8PSAbCON7kJacsdesYi99sXluWOs7njbB44O4C3fZdvbc6OfzXedFzpGwaGDGC7E4A8zOqbnONwGGp9IQfwsq_G-eguWVRl6yW2SZB6FHyJlD9pBad_dydvW3_-5gu4N5lNj_KjD8cfn8F9pDL4CG3DanO9tM_R_Gn0jpf6n4a2-74
openUrl ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Stochastic+Optimal+Control+for+Robot+Manipulation+Skill+Learning+Under+Time-Varying+Uncertain+Environment&rft.jtitle=IEEE+transactions+on+cybernetics&rft.au=Liu%2C+Xing&rft.au=Liu%2C+Zhengxiong&rft.au=Huang%2C+Panfeng&rft.date=2024-04-01&rft.eissn=2168-2275&rft.volume=PP&rft_id=info:doi/10.1109%2FTCYB.2022.3211440&rft_id=info%3Apmid%2F36256715&rft.externalDocID=36256715
thumbnail_l http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=2168-2267&client=summon
thumbnail_m http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=2168-2267&client=summon
thumbnail_s http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=2168-2267&client=summon