Robust Nonlinear Model Predictive Control Based Visual Servoing of Quadrotor UAVs

In this article, a robust nonlinear model predictive control (NMPC) scheme is proposed for the visual servoing of quadrotors subject to external disturbances. By using the virtual camera approach, the image moments are defined in the virtual camera plane and adopted as visual features to derive the...

Full description

Saved in:
Bibliographic Details
Published inIEEE/ASME transactions on mechatronics Vol. 26; no. 2; pp. 700 - 708
Main Authors Zhang, Kunwu, Shi, Yang, Sheng, Huaiyuan
Format Journal Article
LanguageEnglish
Published New York IEEE 01.04.2021
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Subjects
Online AccessGet full text
ISSN1083-4435
1941-014X
DOI10.1109/TMECH.2021.3053267

Cover

Loading…
Abstract In this article, a robust nonlinear model predictive control (NMPC) scheme is proposed for the visual servoing of quadrotors subject to external disturbances. By using the virtual camera approach, the image moments are defined in the virtual camera plane and adopted as visual features to derive the decoupled image kinematics. As a result, the image-based visual servoing (IBVS) system model is established by integrating the image kinematics and quadrotor dynamics. To handle the visibility constraint, a robust NMPC scheme is developed for the IBVS of the quadrotor such that the visual target can stay within the field of view of the camera. In addition, based on the Lipschitz condition, the tightened state constraints are constructed to tackle external disturbances. The sufficient conditions on guaranteeing recursive feasibility of the proposed NMPC algorithm are derived. Furthermore, we theoretically show that the tracking error will converge to a small set around the origin in finite time under some derived conditions. Finally, simulation studies and experimental tests are conducted to verify the efficacy of the proposed method.
AbstractList In this article, a robust nonlinear model predictive control (NMPC) scheme is proposed for the visual servoing of quadrotors subject to external disturbances. By using the virtual camera approach, the image moments are defined in the virtual camera plane and adopted as visual features to derive the decoupled image kinematics. As a result, the image-based visual servoing (IBVS) system model is established by integrating the image kinematics and quadrotor dynamics. To handle the visibility constraint, a robust NMPC scheme is developed for the IBVS of the quadrotor such that the visual target can stay within the field of view of the camera. In addition, based on the Lipschitz condition, the tightened state constraints are constructed to tackle external disturbances. The sufficient conditions on guaranteeing recursive feasibility of the proposed NMPC algorithm are derived. Furthermore, we theoretically show that the tracking error will converge to a small set around the origin in finite time under some derived conditions. Finally, simulation studies and experimental tests are conducted to verify the efficacy of the proposed method.
Author Zhang, Kunwu
Sheng, Huaiyuan
Shi, Yang
Author_xml – sequence: 1
  givenname: Kunwu
  orcidid: 0000-0002-2734-5813
  surname: Zhang
  fullname: Zhang, Kunwu
  email: kunwu@uvic.ca
  organization: Department of Mechanical Engineering, University of Victoria, Victoria, BC, Canada
– sequence: 2
  givenname: Yang
  orcidid: 0000-0003-1337-5322
  surname: Shi
  fullname: Shi, Yang
  email: yshi@uvic.ca
  organization: Department of Mechanical Engineering, University of Victoria, Victoria, BC, Canada
– sequence: 3
  givenname: Huaiyuan
  surname: Sheng
  fullname: Sheng, Huaiyuan
  email: huaiyuan@uvic.ca
  organization: Department of Mechanical Engineering, University of Victoria, Victoria, BC, Canada
BookMark eNp9kE1PAjEQhhuDiYD-Ab008bzYabf7cUSCYgIqCsTbZrZbTMm6xXaXxH_vIsSDB08zh_d5J_P0SKeylSbkEtgAgKU3i9l4NBlwxmEgmBQ8ik9IF9IQAgbhW6fdWSKCMBTyjPS83zDGQmDQJfMXmze-po-2Kk2l0dGZLXRJn50ujKrNTtORrWpnS3qLXhd0ZXyDJX3VbmdN9U7tms4bLJytraPL4cqfk9M1ll5fHGefLO_Gi9EkmD7dP4yG00DxVNYBQhJCqAqFLEUlUKGKQIKUaSQTGUnQa4V5jgkXMQehY8FZHEEKMk8AUYs-uT70bp39bLSvs41tXNWezHjbE4cyBdamkkNKOeu90-tMmRprs_8JTZkBy_YCsx-B2V5gdhTYovwPunXmA93X_9DVATJa618gFYJFnIlvu4R9RA
CODEN IATEFW
CitedBy_id crossref_primary_10_1007_s40435_025_01593_x
crossref_primary_10_1109_TASE_2022_3180506
crossref_primary_10_1016_j_arcontrol_2021_10_008
crossref_primary_10_1142_S2301385024500171
crossref_primary_10_1109_TASE_2024_3434637
crossref_primary_10_1109_TNNLS_2023_3264511
crossref_primary_10_3390_drones8040136
crossref_primary_10_1109_TCST_2022_3188399
crossref_primary_10_1109_TFUZZ_2023_3328884
crossref_primary_10_1007_s11465_024_0782_6
crossref_primary_10_1016_j_amc_2023_127967
crossref_primary_10_3390_wevj15040144
crossref_primary_10_1080_23307706_2022_2068684
crossref_primary_10_1109_JAS_2022_106016
crossref_primary_10_1109_TIV_2022_3163315
crossref_primary_10_1177_10775463221108542
crossref_primary_10_1109_TMECH_2023_3316165
crossref_primary_10_1016_j_ifacol_2022_11_209
crossref_primary_10_1109_LRA_2023_3245416
crossref_primary_10_1109_LRA_2023_3339064
crossref_primary_10_1007_s10514_024_10183_3
crossref_primary_10_1016_j_neucom_2022_06_110
crossref_primary_10_1016_j_jfranklin_2022_11_036
crossref_primary_10_1016_j_eng_2023_05_017
crossref_primary_10_1109_LRA_2024_3521679
crossref_primary_10_1109_ACCESS_2024_3407684
crossref_primary_10_1109_JAS_2024_124533
crossref_primary_10_1016_j_robot_2024_104754
crossref_primary_10_1002_btpr_3426
crossref_primary_10_3390_math11244876
crossref_primary_10_1109_TSMC_2024_3354893
crossref_primary_10_3390_drones9020082
crossref_primary_10_1109_LRA_2024_3416772
crossref_primary_10_1109_TCST_2024_3387228
crossref_primary_10_1049_itr2_12284
crossref_primary_10_1007_s12559_024_10290_4
crossref_primary_10_1016_j_ast_2025_110041
crossref_primary_10_1109_TFUZZ_2023_3338466
crossref_primary_10_1177_01423312241228886
crossref_primary_10_1002_rnc_6973
crossref_primary_10_1016_j_rineng_2024_102497
crossref_primary_10_1109_TNNLS_2023_3342078
crossref_primary_10_3390_math12152413
crossref_primary_10_1109_TIE_2022_3152012
crossref_primary_10_1109_TMECH_2023_3274762
crossref_primary_10_1109_LRA_2023_3322345
crossref_primary_10_1109_TMECH_2023_3316423
crossref_primary_10_1016_j_automatica_2023_111015
crossref_primary_10_1109_TIE_2022_3163526
crossref_primary_10_1115_1_4067770
crossref_primary_10_1007_s12555_022_0205_6
crossref_primary_10_3390_drones8080389
crossref_primary_10_1109_TMECH_2022_3175969
crossref_primary_10_1016_j_conengprac_2023_105816
crossref_primary_10_1109_TCYB_2024_3413072
crossref_primary_10_1109_TMECH_2022_3231405
crossref_primary_10_1007_s11768_023_00174_7
crossref_primary_10_3390_drones7020100
crossref_primary_10_1109_TMECH_2022_3217034
crossref_primary_10_1016_j_jfranklin_2022_07_056
crossref_primary_10_1109_TMECH_2023_3276211
crossref_primary_10_1007_s11432_022_3735_5
Cites_doi 10.1109/TMECH.2019.2906430
10.1016/S0005-1098(98)00073-9
10.1109/TMECH.2020.2978529
10.1109/CDC.2006.377243
10.1109/TMECH.2016.2639531
10.1109/TMECH.2019.2891083
10.1109/9.262032
10.1109/TIE.2015.2510505
10.1109/TMECH.2017.2660528
10.1109/TRO.2005.853500
10.1109/TMECH.2016.2608862
10.1109/TMECH.2020.2990582
10.1007/s12532-018-0139-4
10.1109/MRA.2006.250573
10.1016/j.automatica.2014.03.015
10.1109/TCYB.2015.2475376
10.1002/rob.20414
10.1109/ISIE.2019.8781212
10.1109/ICRA.2014.6907511
10.1109/ICRA.2013.6631405
10.1109/TCYB.2017.2695499
10.1109/TIE.2017.2711575
10.1016/j.automatica.2017.12.034
ContentType Journal Article
Copyright Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2021
Copyright_xml – notice: Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) 2021
DBID 97E
RIA
RIE
AAYXX
CITATION
7SC
7SP
7TB
8FD
FR3
JQ2
L7M
L~C
L~D
DOI 10.1109/TMECH.2021.3053267
DatabaseName IEEE All-Society Periodicals Package (ASPP) 2005–Present
IEEE All-Society Periodicals Package (ASPP) 1998–Present
IEEE Xplore
CrossRef
Computer and Information Systems Abstracts
Electronics & Communications Abstracts
Mechanical & Transportation Engineering Abstracts
Technology Research Database
Engineering Research Database
ProQuest Computer Science Collection
Advanced Technologies Database with Aerospace
Computer and Information Systems Abstracts – Academic
Computer and Information Systems Abstracts Professional
DatabaseTitle CrossRef
Technology Research Database
Computer and Information Systems Abstracts – Academic
Mechanical & Transportation Engineering Abstracts
Electronics & Communications Abstracts
ProQuest Computer Science Collection
Computer and Information Systems Abstracts
Engineering Research Database
Advanced Technologies Database with Aerospace
Computer and Information Systems Abstracts Professional
DatabaseTitleList Technology Research Database

Database_xml – sequence: 1
  dbid: RIE
  name: IEEE/IET Electronic Library (IEL) (UW System Shared)
  url: https://proxy.k.utb.cz/login?url=https://ieeexplore.ieee.org/
  sourceTypes: Publisher
DeliveryMethod fulltext_linktorsrc
Discipline Engineering
EISSN 1941-014X
EndPage 708
ExternalDocumentID 10_1109_TMECH_2021_3053267
9330620
Genre orig-research
GrantInformation_xml – fundername: Natural Sciences and Engineering Research Council of Canada
  funderid: 10.13039/501100000038
GroupedDBID -~X
0R~
29I
4.4
5GY
5VS
6IK
97E
9M8
AAJGR
AARMG
AASAJ
AAWTH
ABAZT
ABQJQ
ABVLG
ACGFS
ACIWK
ACKIV
AETIX
AGQYO
AGSQL
AHBIQ
AI.
AIBXA
AKJIK
AKQYR
ALLEH
ALMA_UNASSIGNED_HOLDINGS
ATWAV
BEFXN
BFFAM
BGNUA
BKEBE
BPEOZ
CS3
EBS
EJD
F5P
H~9
IFIPE
IFJZH
IPLJI
JAVBF
LAI
M43
OCL
RIA
RIE
RNS
TN5
VH1
AAYXX
CITATION
RIG
7SC
7SP
7TB
8FD
FR3
JQ2
L7M
L~C
L~D
ID FETCH-LOGICAL-c295t-a18414cdca09ac3acac61515596585651efcabba8237213e7320761915b81aae3
IEDL.DBID RIE
ISSN 1083-4435
IngestDate Mon Jun 30 04:44:45 EDT 2025
Tue Jul 01 04:23:20 EDT 2025
Thu Apr 24 23:07:32 EDT 2025
Wed Aug 27 02:30:53 EDT 2025
IsPeerReviewed true
IsScholarly true
Issue 2
Language English
License https://ieeexplore.ieee.org/Xplorehelp/downloads/license-information/IEEE.html
https://doi.org/10.15223/policy-029
https://doi.org/10.15223/policy-037
LinkModel DirectLink
MergedId FETCHMERGED-LOGICAL-c295t-a18414cdca09ac3acac61515596585651efcabba8237213e7320761915b81aae3
Notes ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ORCID 0000-0003-1337-5322
0000-0002-2734-5813
PQID 2515745910
PQPubID 85420
PageCount 9
ParticipantIDs crossref_citationtrail_10_1109_TMECH_2021_3053267
proquest_journals_2515745910
crossref_primary_10_1109_TMECH_2021_3053267
ieee_primary_9330620
ProviderPackageCode CITATION
AAYXX
PublicationCentury 2000
PublicationDate 2021-April
2021-4-00
20210401
PublicationDateYYYYMMDD 2021-04-01
PublicationDate_xml – month: 04
  year: 2021
  text: 2021-April
PublicationDecade 2020
PublicationPlace New York
PublicationPlace_xml – name: New York
PublicationTitle IEEE/ASME transactions on mechatronics
PublicationTitleAbbrev TMECH
PublicationYear 2021
Publisher IEEE
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Publisher_xml – name: IEEE
– name: The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
References ref13
ref12
ref15
ref14
ref11
ref10
f (ref1) 2012; 29
ref2
ref17
ref16
ref18
bradski (ref28) 0; 25
khalil (ref26) 2002
ref24
ref23
ref25
ref20
ref22
ref21
li (ref8) 2016; 83
bouabdallah (ref19) 2007
ref29
ref9
ref4
lee (ref27) 2013; 15
ref3
ref6
ref5
xie (ref7) 2016
References_xml – year: 2002
  ident: ref26
  publication-title: Nonlinear Systems
– volume: 25
  start-page: 120
  year: 0
  ident: ref28
  article-title: The OpenCV library
  publication-title: Dr Dobb's J Softw Tools
– ident: ref12
  doi: 10.1109/TMECH.2019.2906430
– ident: ref25
  doi: 10.1016/S0005-1098(98)00073-9
– ident: ref5
  doi: 10.1109/TMECH.2020.2978529
– ident: ref13
  doi: 10.1109/CDC.2006.377243
– ident: ref20
  doi: 10.1109/TMECH.2016.2639531
– ident: ref3
  doi: 10.1109/TMECH.2019.2891083
– ident: ref24
  doi: 10.1109/9.262032
– volume: 15
  start-page: 391
  year: 2013
  ident: ref27
  article-title: Nonlinear robust tracking control of a quadrotor UAV on
  publication-title: SE
– ident: ref16
  doi: 10.1109/TIE.2015.2510505
– ident: ref9
  doi: 10.1109/TMECH.2017.2660528
– ident: ref21
  doi: 10.1109/TRO.2005.853500
– ident: ref11
  doi: 10.1109/TMECH.2016.2608862
– ident: ref4
  doi: 10.1109/TMECH.2020.2990582
– year: 2016
  ident: ref7
  article-title: Dynamic visual servoing of rotary wing unmanned aerial vehicles
– ident: ref29
  doi: 10.1007/s12532-018-0139-4
– year: 2007
  ident: ref19
  article-title: Design and control of quadrotors with application to autonomous flying
– ident: ref6
  doi: 10.1109/MRA.2006.250573
– ident: ref23
  doi: 10.1016/j.automatica.2014.03.015
– ident: ref14
  doi: 10.1109/TCYB.2015.2475376
– volume: 29
  start-page: 315
  year: 2012
  ident: ref1
  article-title: Survey of advances in guidance, navigation, and control of unmanned rotorcraft systems
  publication-title: J Field Robot
  doi: 10.1002/rob.20414
– volume: 83
  year: 2016
  ident: ref8
  publication-title: Robust Receding Horizon Control for Networked and Distributed Nonlinear Systems
– ident: ref15
  doi: 10.1109/ISIE.2019.8781212
– ident: ref17
  doi: 10.1109/ICRA.2014.6907511
– ident: ref10
  doi: 10.1109/ICRA.2013.6631405
– ident: ref18
  doi: 10.1109/TCYB.2017.2695499
– ident: ref2
  doi: 10.1109/TIE.2017.2711575
– ident: ref22
  doi: 10.1016/j.automatica.2017.12.034
SSID ssj0004101
Score 2.5792372
Snippet In this article, a robust nonlinear model predictive control (NMPC) scheme is proposed for the visual servoing of quadrotors subject to external disturbances....
SourceID proquest
crossref
ieee
SourceType Aggregation Database
Enrichment Source
Index Database
Publisher
StartPage 700
SubjectTerms Algorithms
Cameras
Disturbances
Field of view
Image-based visual servoing (IBVS)
Kinematics
Lipschitz condition
Mechatronics
Nonlinear control
Predictive control
quadrotor
Robust control
robust nonlinear model predictive control (NMPC)
Rotary wing aircraft
Servocontrol
Tracking errors
Uncertainty
Unmanned aerial vehicles
Virtual cameras
Visibility
Visual servoing
Visualization
Title Robust Nonlinear Model Predictive Control Based Visual Servoing of Quadrotor UAVs
URI https://ieeexplore.ieee.org/document/9330620
https://www.proquest.com/docview/2515745910
Volume 26
hasFullText 1
inHoldings 1
isFullTextHit
isPrint
link http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwjV1bS8MwFA5zT_rgbYrTKXnwTdstbdquj3M4hrDhZBt7KyeXgjhW6eXFX2-StpuoiG-lJCHk5PJ9yTnfQegWOJFEhNIC7eVAOYDV95mwqCcZ94Ew30TITab-eEGfVt6qge63sTBSSuN8Jm39ad7yRcILfVXW1eTbdxRB31PErYzV2sVAEpPqmChIYVGFAeoAmV7YnU8eh2NFBR1iuzoRgskpvzuETFaVH1uxOV9GR2hS96x0K3mzi5zZ_OObaON_u36MDiugiQflzDhBDbk5RQdf5AdbaPaSsCLL8bSUy4AU68xoa_yc6tcbvQ_iYenJjh_UYSfw8jUrVJt6f0lUCziJ8awAkSaKuOPFYJmdocXocT4cW1WKBYs7oZdboAgeoVxw6IXAXeDADcTxtCaMwnpExhwYAy1p4xBXBq5jLj6Ix_oEQLrnqLlJNvICYSkAYkFDKmNNY3ohU39E7PgxJUHghW1E6jGPeKU_rtNgrCPDQ3phZOwUaTtFlZ3a6G5b571U3_izdEsP_LZkNeZt1KlNG1ULNIsUrPMC6imwdPl7rSu0r9sunXQ6qJmnhbxW-CNnN2bifQLgfdYm
linkProvider IEEE
linkToHtml http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwjV3Nb9owFH-q2KHdYWvXTmPrhw-7rQGc2IEcOwSiH6B1AsQtev6IVA2RCZLL_vr5OYFOa1X1FkW2Y_k59u9nv_d7AF9Rc8tNYgMkLwehEYNerEwgpFU6Rq5iHyE3nsSjmbhZyMUeXO5iYay13vnMtujR3-WbXJd0VNYm8h2HjqC_kRSMW0VrPUZBcp_smDtQEQiHArYhMp2kPR0P-iNHBkPeiigVgs8q_7gN-bwqTxZjv8MM38N427fKseRXqyxUS__5T7bxtZ0_hHc11GRX1dw4gj27-gBv_xEgPIb7n7kqNwWbVIIZuGaUG23Jfqzp_oZWQtavfNnZd7fdGTZ_2JSuTVphctcCyzN2X6JZ5466s9nVfHMCs-Fg2h8FdZKFQIeJLAJ0FI8LbTR2EtQRatQe5EhShXFoj9tMo1JIojYhj2w3Cv3RB5eqxxFt9BEaq3xlPwGzBjEzIhE2IyLTSZR7Y7IwzgTvdmXSBL4d81TXCuSUCGOZeibSSVJvp5TslNZ2asK3XZ3flf7Gi6WPaeB3Jesxb8Lp1rRp_YtuUgfsZFdIB5c-P1_rAvZH0_Fdenc9uf0CB_SdymXnFBrFurRnDo0U6txPwr82Mtlu
openUrl ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Robust+Nonlinear+Model+Predictive+Control+Based+Visual+Servoing+of+Quadrotor+UAVs&rft.jtitle=IEEE%2FASME+transactions+on+mechatronics&rft.au=Zhang%2C+Kunwu&rft.au=Yang%2C+Shi&rft.au=Sheng%2C+Huaiyuan&rft.date=2021-04-01&rft.pub=The+Institute+of+Electrical+and+Electronics+Engineers%2C+Inc.+%28IEEE%29&rft.issn=1083-4435&rft.eissn=1941-014X&rft.volume=26&rft.issue=2&rft.spage=700&rft_id=info:doi/10.1109%2FTMECH.2021.3053267&rft.externalDBID=NO_FULL_TEXT
thumbnail_l http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=1083-4435&client=summon
thumbnail_m http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=1083-4435&client=summon
thumbnail_s http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=1083-4435&client=summon