A Hybrid GA-PSO Method for Evolving Architecture and Short Connections of Deep Convolutional Neural Networks
Image classification is a difficult machine learning task, where Convolutional Neural Networks (CNNs) have been applied for over 20 years in order to solve the problem. In recent years, instead of the traditional way of only connecting the current layer with its next layer, shortcut connections have...
Saved in:
Published in | PRICAI 2019: Trends in Artificial Intelligence pp. 650 - 663 |
---|---|
Main Authors | , , , |
Format | Book Chapter |
Language | English |
Published |
Cham
Springer International Publishing
|
Series | Lecture Notes in Computer Science |
Subjects | |
Online Access | Get full text |
Cover
Loading…
Abstract | Image classification is a difficult machine learning task, where Convolutional Neural Networks (CNNs) have been applied for over 20 years in order to solve the problem. In recent years, instead of the traditional way of only connecting the current layer with its next layer, shortcut connections have been proposed to connect the current layer with its forward layers apart from its next layer, which has been proved to be able to facilitate the training process of deep CNNs. However, there are various ways to build the shortcut connections, it is hard to manually design the best shortcut connections when solving a particular problem, especially given the design of the network architecture is already very challenging. In this paper, a hybrid evolutionary computation (EC) method is proposed to automatically evolve both the architecture of deep CNNs and the shortcut connections. Three major contributions of this work are: Firstly, a new encoding strategy is proposed to encode a CNN, where the architecture and the shortcut connections are encoded separately; Secondly, a hybrid two-level EC method, which combines particle swarm optimisation and genetic algorithms, is developed to search for the optimal CNNs; Lastly, an adjustable learning rate is introduced for the fitness evaluations, which provides a better learning rate for the training process given a fixed number of epochs. The proposed algorithm is evaluated on three widely used benchmark datasets of image classification and compared with 12 peer Non-EC based competitors and one EC based competitor. The experimental results demonstrate that the proposed method outperforms all of the peer competitors in terms of classification accuracy. |
---|---|
AbstractList | Image classification is a difficult machine learning task, where Convolutional Neural Networks (CNNs) have been applied for over 20 years in order to solve the problem. In recent years, instead of the traditional way of only connecting the current layer with its next layer, shortcut connections have been proposed to connect the current layer with its forward layers apart from its next layer, which has been proved to be able to facilitate the training process of deep CNNs. However, there are various ways to build the shortcut connections, it is hard to manually design the best shortcut connections when solving a particular problem, especially given the design of the network architecture is already very challenging. In this paper, a hybrid evolutionary computation (EC) method is proposed to automatically evolve both the architecture of deep CNNs and the shortcut connections. Three major contributions of this work are: Firstly, a new encoding strategy is proposed to encode a CNN, where the architecture and the shortcut connections are encoded separately; Secondly, a hybrid two-level EC method, which combines particle swarm optimisation and genetic algorithms, is developed to search for the optimal CNNs; Lastly, an adjustable learning rate is introduced for the fitness evaluations, which provides a better learning rate for the training process given a fixed number of epochs. The proposed algorithm is evaluated on three widely used benchmark datasets of image classification and compared with 12 peer Non-EC based competitors and one EC based competitor. The experimental results demonstrate that the proposed method outperforms all of the peer competitors in terms of classification accuracy. |
Author | Xue, Bing Zhang, Mengjie Sun, Yanan Wang, Bin |
Author_xml | – sequence: 1 givenname: Bin surname: Wang fullname: Wang, Bin email: bin.wang@ecs.vuw.ac.nz organization: School of Engineering and Computer Science, Victoria University of Wellington, Wellington, New Zealand – sequence: 2 givenname: Yanan surname: Sun fullname: Sun, Yanan email: yanan.sun@ecs.vuw.ac.nz organization: School of Engineering and Computer Science, Victoria University of Wellington, Wellington, New Zealand – sequence: 3 givenname: Bing surname: Xue fullname: Xue, Bing email: bing.xue@ecs.vuw.ac.nz organization: School of Engineering and Computer Science, Victoria University of Wellington, Wellington, New Zealand – sequence: 4 givenname: Mengjie surname: Zhang fullname: Zhang, Mengjie email: mengjie.zhang@ecs.vuw.ac.nz organization: School of Engineering and Computer Science, Victoria University of Wellington, Wellington, New Zealand |
BookMark | eNpFkN1OAjEQRqtiIqBv4EVfoNp2Ct1ebpAfExQTuG92twVWNi1pF4xvbxdNvPomZzKTL2eAes47i9Ajo0-MUvmsZEaAUKCEq0wJIvSIX6EBJHIB6hr12ZgxAiDUzf8CaA_1u5koKeAODWL8pJRyqXgfNTlefJehNniek4_1Cr_Zdu8N3vqAp2ffnGu3w3mo9nVrq_YULC6cweu9Dy2eeOcSrL2L2G_xi7XHjqWrUweLBr_bU7hE--XDId6j223RRPvwl0O0mU03kwVZruavk3xJdjCGNtU0DDIwtpRZWXLOmBWWg6GCG1FlkheyGCVgVdKijBJUVVRyGJfUZoWEIeK_b-MxpPo26NL7Q9SM6k6kTiI16GREX7zpTiT8AI8VY6k |
ContentType | Book Chapter |
Copyright | Springer Nature Switzerland AG 2019 |
Copyright_xml | – notice: Springer Nature Switzerland AG 2019 |
DOI | 10.1007/978-3-030-29894-4_52 |
DatabaseTitleList | |
DeliveryMethod | fulltext_linktorsrc |
Discipline | Computer Science |
EISBN | 3030298949 9783030298944 |
EISSN | 1611-3349 |
Editor | Nayak, Abhaya C. Sharma, Alok |
Editor_xml | – sequence: 1 givenname: Abhaya C. orcidid: 0000-0003-0681-9570 surname: Nayak fullname: Nayak, Abhaya C. email: abhaya.nayak@mq.edu.au – sequence: 2 givenname: Alok orcidid: 0000-0002-7668-3501 surname: Sharma fullname: Sharma, Alok email: alok.fj@gmail.com |
EndPage | 663 |
GroupedDBID | -DT -GH -~X 1SB 29L 2HA 2HV 5QI 875 AASHB ABMNI ACGFS ADCXD AEFIE ALMA_UNASSIGNED_HOLDINGS EJD F5P FEDTE HVGLF LAS LDH P2P RIG RNI RSU SVGTG VI1 ~02 |
ID | FETCH-LOGICAL-g363t-97d1383deb78bb2211e4e23d042d4c872a7a5e23e90079d9409c07236b0e8a73 |
ISBN | 3030298930 9783030298937 |
ISSN | 0302-9743 |
IngestDate | Tue Oct 01 19:47:32 EDT 2024 |
IsDoiOpenAccess | false |
IsOpenAccess | true |
IsPeerReviewed | true |
IsScholarly | true |
Language | English |
LinkModel | OpenURL |
MergedId | FETCHMERGED-LOGICAL-g363t-97d1383deb78bb2211e4e23d042d4c872a7a5e23e90079d9409c07236b0e8a73 |
OpenAccessLink | https://figshare.com/articles/chapter/A_Hybrid_GA-PSO_Method_for_Evolving_Architecture_and_Short_Connections_of_Deep_Convolutional_Neural_Networks/13158299/1/files/25303565.pdf |
PageCount | 14 |
ParticipantIDs | springer_books_10_1007_978_3_030_29894_4_52 |
PublicationPlace | Cham |
PublicationPlace_xml | – name: Cham |
PublicationSeriesSubtitle | Lecture Notes in Artificial Intelligence |
PublicationSeriesTitle | Lecture Notes in Computer Science |
PublicationSeriesTitleAlternate | Lect.Notes Computer |
PublicationSubtitle | 16th Pacific Rim International Conference on Artificial Intelligence, Cuvu, Yanuca Island, Fiji, August 26-30, 2019, Proceedings, Part III |
PublicationTitle | PRICAI 2019: Trends in Artificial Intelligence |
Publisher | Springer International Publishing |
Publisher_xml | – name: Springer International Publishing |
RelatedPersons | Hartmanis, Juris Gao, Wen Bertino, Elisa Woeginger, Gerhard Goos, Gerhard Steffen, Bernhard Yung, Moti |
RelatedPersons_xml | – sequence: 1 givenname: Gerhard surname: Goos fullname: Goos, Gerhard organization: Karlsruhe Institute of Technology, Karlsruhe, Germany – sequence: 2 givenname: Juris surname: Hartmanis fullname: Hartmanis, Juris organization: Cornell University, Ithaca, USA – sequence: 3 givenname: Elisa surname: Bertino fullname: Bertino, Elisa organization: Purdue University, West Lafayette, USA – sequence: 4 givenname: Wen surname: Gao fullname: Gao, Wen organization: Peking University, Beijing, China – sequence: 5 givenname: Bernhard surname: Steffen fullname: Steffen, Bernhard organization: TU Dortmund University, Dortmund, Germany – sequence: 6 givenname: Gerhard surname: Woeginger fullname: Woeginger, Gerhard organization: RWTH Aachen, Aachen, Germany – sequence: 7 givenname: Moti surname: Yung fullname: Yung, Moti organization: Columbia University, New York, USA |
SSID | ssj0002792 ssj0002209143 |
Score | 2.3707764 |
Snippet | Image classification is a difficult machine learning task, where Convolutional Neural Networks (CNNs) have been applied for over 20 years in order to solve the... |
SourceID | springer |
SourceType | Publisher |
StartPage | 650 |
SubjectTerms | Convolutional Neural Networks Evolutionary computation Image classification Shortcut connections |
Title | A Hybrid GA-PSO Method for Evolving Architecture and Short Connections of Deep Convolutional Neural Networks |
URI | http://link.springer.com/10.1007/978-3-030-29894-4_52 |
hasFullText | 1 |
inHoldings | 1 |
isFullTextHit | |
isPrint | |
link | http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwnV07b9swECbcZOvQ9IW2SAsO3QQVNkk9OGRQU7duECQF7AbOJEgilaYo7CCRCzQ_KL8zd3xIsp0lXSSDIGjpPurueLz7SMhHWPRoEdd1yFM8wkypKExFyUJeSlHpeiSUNgmyJ_HkpziaR_PB4K6XtbRqyk_V7YN1Jf-DKrQBrlgl-whk20GhAX4DvnAFhOG64fyuh1kt6QWe1vg9ANMqTYy8TW7Nrk3-j2XR6Ag3u9C5_bw_X7bzYroyuue8WHRzZb7SrtfFVnAZU2F_X65NtyyY_MPir-BbFv6YnkIXPJnaJDGOQQOasEW2uWkx_QXOf2Bybao2I--L1lfY9teJDl4DGUTMzaSs21UACljfHBy74U6WjUktC_wxFV5rbYUwN4KgXRxubc0LNtfQxluuGF_7BXodVkZWVWqrymMkaOSWENWp59iS3DpLH1vVumVE-nkjMHJoSOpDkUdg6p8kEnTnbjY-Oj5rY3mMgdclOg8ASRnt7pV9Kqwp8k_teMa6t-jVcz70l1s79Mbxme2Rp1gMQ7FKBYT2nAz04gV55mVMnYxfkj8ZtfhTiz-1-FPAn3r8aR9_CvhTgz_t4U-XNUX86Rr-1OJPPf6vyOzreHY4Cd1BHuEFj3kDMlAjUApKl0laloyNRlpoxhUYDCWqNGFFUkTQoCVIQSophrIaJozH5VCnRcJfk53FcqHfEFqnnOtYCqTJE-Ccy4hHVV0UhajYUKXJWxJ4aeX4Zd7knpYbZJvzHGSbG9nmKNt3j-q9T3aa65V-D05oU35wc-AenmyAnQ |
link.rule.ids | 785,786,790,799,27956 |
linkProvider | Library Specific Holdings |
openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=bookitem&rft.title=PRICAI+2019%3A+Trends+in+Artificial+Intelligence&rft.au=Wang%2C+Bin&rft.au=Sun%2C+Yanan&rft.au=Xue%2C+Bing&rft.au=Zhang%2C+Mengjie&rft.atitle=A+Hybrid+GA-PSO+Method+for+Evolving+Architecture+and+Short+Connections+of+Deep+Convolutional+Neural+Networks&rft.series=Lecture+Notes+in+Computer+Science&rft.pub=Springer+International+Publishing&rft.isbn=9783030298937&rft.issn=0302-9743&rft.eissn=1611-3349&rft.spage=650&rft.epage=663&rft_id=info:doi/10.1007%2F978-3-030-29894-4_52 |
thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=0302-9743&client=summon |
thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=0302-9743&client=summon |
thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=0302-9743&client=summon |