How fair can we go in machine learning? Assessing the boundaries of accuracy and fairness
Fair machine learning has been focusing on the development of equitable algorithms that address discrimination. Yet, many of these fairness‐aware approaches aim to obtain a unique solution to the problem, which leads to a poor understanding of the statistical limits of bias mitigation interventions....
Saved in:
Published in | International journal of intelligent systems Vol. 36; no. 4; pp. 1619 - 1643 |
---|---|
Main Authors | , , |
Format | Journal Article |
Language | English |
Published |
New York
Hindawi Limited
01.04.2021
|
Subjects | |
Online Access | Get full text |
Cover
Loading…
Abstract | Fair machine learning has been focusing on the development of equitable algorithms that address discrimination. Yet, many of these fairness‐aware approaches aim to obtain a unique solution to the problem, which leads to a poor understanding of the statistical limits of bias mitigation interventions. In this study, a novel methodology is presented to explore the tradeoff in terms of a Pareto front between accuracy and fairness. To this end, we propose a multiobjective framework that seeks to optimize both measures. The experimental framework is focused on logistiregression and decision tree classifiers since they are well‐known by the machine learning community. We conclude experimentally that our method can optimize classifiers by being fairer with a small cost on the classification accuracy. We believe that our contribution will help stakeholders of sociotechnical systems to assess how far they can go being fair and accurate, thus serving in the support of enhanced decision making where machine learning is used. |
---|---|
AbstractList | Fair machine learning has been focusing on the development of equitable algorithms that address discrimination. Yet, many of these fairness‐aware approaches aim to obtain a unique solution to the problem, which leads to a poor understanding of the statistical limits of bias mitigation interventions. In this study, a novel methodology is presented to explore the tradeoff in terms of a Pareto front between accuracy and fairness. To this end, we propose a multiobjective framework that seeks to optimize both measures. The experimental framework is focused on logistiregression and decision tree classifiers since they are well‐known by the machine learning community. We conclude experimentally that our method can optimize classifiers by being fairer with a small cost on the classification accuracy. We believe that our contribution will help stakeholders of sociotechnical systems to assess how far they can go being fair and accurate, thus serving in the support of enhanced decision making where machine learning is used. Fair machine learning has been focusing on the development of equitable algorithms that address discrimination. Yet, many of these fairness‐aware approaches aim to obtain a unique solution to the problem, which leads to a poor understanding of the statistical limits of bias mitigation interventions. In this study, a novel methodology is presented to explore the tradeoff in terms of a Pareto front between accuracy and fairness. To this end, we propose a multiobjective framework that seeks to optimize both measures. The experimental framework is focused on logistiregression and decision tree classifiers since they are well‐known by the machine learning community. We conclude experimentally that our method can optimize classifiers by being fairer with a small cost on the classification accuracy. We believe that our contribution will help stakeholders of sociotechnical systems to assess how far they can go being fair and accurate, thus serving in the support of enhanced decision making where machine learning is used. |
Author | Sánchez‐Monedero, Javier Casillas, Jorge Valdivia, Ana |
Author_xml | – sequence: 1 givenname: Ana surname: Valdivia fullname: Valdivia, Ana email: ana.valdivia@kcl.ac.uk organization: King's College London – sequence: 2 givenname: Javier surname: Sánchez‐Monedero fullname: Sánchez‐Monedero, Javier organization: Cardiff University – sequence: 3 givenname: Jorge surname: Casillas fullname: Casillas, Jorge organization: University of Granada |
BookMark | eNp10E1PAjEQBuDGYCKgB_9BE08eFvq1bPdkCFEhIXrBRE9NtzsLJdBiy4bw762sV08zh-edSd4B6jnvAKF7SkaUEDa27jhijOfiCvUpKWVGKf3soT6RUmSSFvwGDWLcEkJpIfI--pr7E260Ddhoh0-A1x5bh_fabKwDvAMdnHXrJzyNEWJMKz5uAFe-dbUOFiL2DdbGtEGbM9auvhxzid6i60bvItz9zSH6eHlezebZ8v11MZsuM8M5Exmj3NRMFIZwVlaVLEpNdN1Iqg0ppExIAK2JKGVZ0aKaVLU0bNIApJicGMOH6KG7ewj-u4V4VFvfBpdeKiZKXkgu8jypx06Z4GMM0KhDsHsdzooS9ducSs2pS3PJjjt7sjs4_w_V4m3VJX4A4IxxPg |
CitedBy_id | crossref_primary_10_1016_j_eswa_2022_118987 crossref_primary_10_1007_s13042_023_01870_1 crossref_primary_10_1007_s12065_022_00702_5 crossref_primary_10_1002_widm_1452 crossref_primary_10_1111_exsy_13467 crossref_primary_10_1016_j_eswa_2022_117240 crossref_primary_10_1145_3631326 crossref_primary_10_2139_ssrn_4704138 crossref_primary_10_3389_frai_2022_844817 crossref_primary_10_1080_01605682_2024_2354364 |
Cites_doi | 10.1145/3351095.3372857 10.1016/j.knosys.2019.07.028 10.1007/s10618-019-00658-8 10.1162/evco.1994.2.3.221 10.1145/3351095.3372849 10.1002/widm.1356 10.1145/3173574.3173951 10.1145/3287560.3287598 10.1038/s42256-019-0048-x 10.1162/106365600568202 10.1145/3287560.3287589 10.1109/ICDE48307.2020.00203 10.1145/3038912.3052660 10.1109/4235.996017 10.1089/big.2016.0047 10.1145/3287560.3287596 10.1109/ICDM.2010.50 |
ContentType | Journal Article |
Copyright | 2020 Wiley Periodicals LLC 2021 Wiley Periodicals LLC |
Copyright_xml | – notice: 2020 Wiley Periodicals LLC – notice: 2021 Wiley Periodicals LLC |
DBID | AAYXX CITATION 7SC 8FD JQ2 L7M L~C L~D |
DOI | 10.1002/int.22354 |
DatabaseName | CrossRef Computer and Information Systems Abstracts Technology Research Database ProQuest Computer Science Collection Advanced Technologies Database with Aerospace Computer and Information Systems Abstracts Academic Computer and Information Systems Abstracts Professional |
DatabaseTitle | CrossRef Computer and Information Systems Abstracts Technology Research Database Computer and Information Systems Abstracts – Academic Advanced Technologies Database with Aerospace ProQuest Computer Science Collection Computer and Information Systems Abstracts Professional |
DatabaseTitleList | Computer and Information Systems Abstracts |
DeliveryMethod | fulltext_linktorsrc |
Discipline | Computer Science |
EISSN | 1098-111X |
EndPage | 1643 |
ExternalDocumentID | 10_1002_int_22354 INT22354 |
Genre | article |
GrantInformation_xml | – fundername: Spanish Ministry of Science and Innovation – fundername: Institute of Health Carlos III funderid: PI20/01435 – fundername: European Union's Horizon 2020 research and innovation programme funderid: 819213 – fundername: European Research Council funderid: 759903 |
GroupedDBID | -~X .3N .4S .DC .GA .Y3 05W 0R~ 10A 1L6 1OB 1OC 24P 31~ 33P 3SF 3WU 4.4 50Y 50Z 51W 51X 52M 52N 52O 52P 52S 52T 52U 52W 52X 5GY 5VS 66C 702 7PT 8-0 8-1 8-3 8-4 8-5 8UM 930 A03 AAESR AAEVG AAHHS AAJEY AAONW AASGY AAXRX AAYOK AAZKR ABCQN ABCUV ABEML ABIJN ABJCF ABJNI ABPVW ABTAH ABUWG ACAHQ ACBWZ ACCFJ ACCZN ACGFS ACIWK ACPOU ACSCC ACXBN ACXQS ADBBV ADEOM ADIZJ ADKYN ADMGS ADOZA ADXAS ADZMN ADZOD AEEZP AEIMD AENEX AEQDE AEUQT AFBPY AFGKR AFKRA AFPWT AFZJQ AI. AIURR AIWBW AJBDE AJXKR ALAGY ALMA_UNASSIGNED_HOLDINGS ALUQN AMBMR AMYDB ARAPS ARCSS ASPBG ATUGU AUFTA AVWKF AZBYB AZFZN AZQEC AZVAB BAFTC BDRZF BENPR BFHJK BGLVJ BHBCM BMNLL BMXJE BNHUX BROTX BRXPI BY8 CCPQU CMOOK CS3 D-E D-F DCZOG DPXWK DR2 DRFUL DRSTM DU5 DWQXO EBS EDO EJD F00 F01 F04 FEDTE G-S G.N G8K GNP GNUQQ GODZA H.T H.X H13 HBH HCIFZ HF~ HHY HVGLF HZ~ I-F IX1 J0M JPC K7- KQQ LATKE LAW LC2 LC3 LEEKS LH4 LITHE LOXES LP6 LP7 LUTES LW6 LYRES M59 M7S MK4 MK~ MRFUL MRSTM MSFUL MSSTM MVM MXFUL MXSTM N04 N05 N9A NF~ O66 O9- OIG P2P P2W P2X P4D PALCI PIMPY PQQKQ PTHSS Q.N Q11 QB0 QRW R.K RHX RIWAO RJQFR ROL RWI RX1 RYL SAMSI SUPJJ TN5 TUS UB1 V2E VH1 W8V W99 WBKPD WH7 WIH WIK WOHZO WQJ WRC WWI WXSBR WYISQ WZISG XG1 XPP XV2 ZY4 ZZTAW ~IA ~WT AAYXX CITATION 7SC 8FD JQ2 L7M L~C L~D |
ID | FETCH-LOGICAL-c3324-213cd247c0329bb879a0adf81ac0788c334e1d04989b17b6bd8c26fee3cd86cc3 |
IEDL.DBID | DR2 |
ISSN | 0884-8173 |
IngestDate | Thu Oct 10 15:21:11 EDT 2024 Fri Aug 23 02:38:18 EDT 2024 Sat Aug 24 01:06:27 EDT 2024 |
IsDoiOpenAccess | false |
IsOpenAccess | true |
IsPeerReviewed | true |
IsScholarly | true |
Issue | 4 |
Language | English |
LinkModel | DirectLink |
MergedId | FETCHMERGED-LOGICAL-c3324-213cd247c0329bb879a0adf81ac0788c334e1d04989b17b6bd8c26fee3cd86cc3 |
OpenAccessLink | https://doi.org/10.1002/int.22354 |
PQID | 2493783455 |
PQPubID | 1026350 |
PageCount | 25 |
ParticipantIDs | proquest_journals_2493783455 crossref_primary_10_1002_int_22354 wiley_primary_10_1002_int_22354_INT22354 |
PublicationCentury | 2000 |
PublicationDate | April 2021 2021-04-00 20210401 |
PublicationDateYYYYMMDD | 2021-04-01 |
PublicationDate_xml | – month: 04 year: 2021 text: April 2021 |
PublicationDecade | 2020 |
PublicationPlace | New York |
PublicationPlace_xml | – name: New York |
PublicationTitle | International journal of intelligent systems |
PublicationYear | 2021 |
Publisher | Hindawi Limited |
Publisher_xml | – name: Hindawi Limited |
References | 2017; 5 2019; 20 2010 2020 2019; 1 2002; 6 2000; 8 2009 2019 2018 2020; 106 2017 2016 2020; 34 2020; 10 2019; 182 1994; 2 e_1_2_7_5_1 e_1_2_7_4_1 Menon AK (e_1_2_7_18_1) 2018 e_1_2_7_9_1 O'Neil C. (e_1_2_7_2_1) 2016 e_1_2_7_19_1 Buolamwini J (e_1_2_7_8_1) 2018 e_1_2_7_17_1 e_1_2_7_16_1 e_1_2_7_15_1 e_1_2_7_14_1 e_1_2_7_13_1 Hardt M (e_1_2_7_29_1) 2016 Lipton Z (e_1_2_7_12_1) 2018 e_1_2_7_11_1 e_1_2_7_10_1 e_1_2_7_26_1 e_1_2_7_28_1 Eubanks V. (e_1_2_7_3_1) 2018 Kearns M (e_1_2_7_7_1) 2018 Bolukbasi T (e_1_2_7_6_1) 2016 Hellman D. (e_1_2_7_27_1) 2020; 106 e_1_2_7_30_1 e_1_2_7_25_1 e_1_2_7_31_1 e_1_2_7_24_1 e_1_2_7_32_1 e_1_2_7_23_1 Agarwal A (e_1_2_7_20_1) 2018 Zafar MB (e_1_2_7_22_1) 2019; 20 Balashankar A (e_1_2_7_21_1) 2019 |
References_xml | – start-page: 77 year: 2018 end-page: 91 – volume: 182 year: 2019 article-title: WE‐Rec: a fairness‐aware reciprocal recommendation based on Walrasian equilibrium publication-title: Knowl‐Based Syst. – start-page: 8 year: 2019 – start-page: 535 year: 2020 end-page: 545 – volume: 8 start-page: 173 issue: 2 year: 2000 end-page: 195 article-title: Comparison of multiobjective evolutionary algorithms: empirical results publication-title: Evol Comput. – start-page: 8125 year: 2018 end-page: 8135 – start-page: 174 year: 2009 – start-page: 220 year: 2019 end-page: 229 – volume: 5 start-page: 153 issue: 2 year: 2017 end-page: 163 article-title: Fair prediction with disparate impact: a study of bias in recidivism prediction instruments publication-title: Big Data. – start-page: 1171 year: 2017 end-page: 1180 – volume: 10 issue: 3 year: 2020 article-title: Bias in data‐driven artificial intelligence systems—an introductory survey publication-title: WIREs Data Mining Knowl Disc. – start-page: 3315 year: 2016 end-page: 3323 – start-page: 458 year: 2020 end-page: 468 – volume: 34 start-page: 163 issue: 1 year: 2020 end-page: 200 article-title: Matching code and law: achieving algorithmic fairness with optimal transport publication-title: Data Mining Knowl Discov. – start-page: 1918 year: 2020 end-page: 1921 – start-page: 1 year: 2018 end-page: 14 – year: 2016 – start-page: 329 year: 2019 end-page: 338 – year: 2018 – volume: 106 start-page: 811 issue: 4 year: 2020 article-title: Measuring algorithmic fairness publication-title: Virgin Law Rev. – volume: 2 start-page: 221 issue: 3 year: 1994 end-page: 248 article-title: Multi‐objective function optimisation using non‐dominated sorting genetic algorithm publication-title: Evol Comput. – start-page: 9 year: 2016 – start-page: 2564 year: 2018 end-page: 2572 – start-page: 107 year: 2018 end-page: 118 – start-page: 60 year: 2018 end-page: 69 – start-page: 869 year: 2010 end-page: 874 – volume: 20 start-page: 1 issue: 75 year: 2019 end-page: 42 article-title: Fairness constraints: a flexible approach for fair classification publication-title: J Mach Learn Res – volume: 1 start-page: 206 issue: 5 year: 2019 end-page: 215 article-title: Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead publication-title: Nat Mach Intell – volume: 6 start-page: 182 issue: 2 year: 2002 end-page: 197 article-title: A fast and elitist multiobjective genetic algorithm: NSGA‐II publication-title: IEEE Trans Evol Comput – start-page: 59 year: 2019 end-page: 68 – ident: e_1_2_7_23_1 doi: 10.1145/3351095.3372857 – ident: e_1_2_7_10_1 doi: 10.1016/j.knosys.2019.07.028 – ident: e_1_2_7_11_1 doi: 10.1007/s10618-019-00658-8 – ident: e_1_2_7_31_1 doi: 10.1162/evco.1994.2.3.221 – volume-title: Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy year: 2016 ident: e_1_2_7_2_1 contributor: fullname: O'Neil C. – start-page: 2564 volume-title: Proceedings of the 35th International Conference on Machine Learning, Vol. 80 of Proceedings of Machine Learning Research year: 2018 ident: e_1_2_7_7_1 contributor: fullname: Kearns M – ident: e_1_2_7_16_1 doi: 10.1145/3351095.3372849 – start-page: 8125 volume-title: Advances in Neural Information Processing Systems year: 2018 ident: e_1_2_7_12_1 contributor: fullname: Lipton Z – ident: e_1_2_7_26_1 doi: 10.1002/widm.1356 – ident: e_1_2_7_13_1 doi: 10.1145/3173574.3173951 – volume-title: Automating Inequality year: 2018 ident: e_1_2_7_3_1 contributor: fullname: Eubanks V. – ident: e_1_2_7_15_1 doi: 10.1145/3287560.3287598 – ident: e_1_2_7_25_1 doi: 10.1038/s42256-019-0048-x – ident: e_1_2_7_4_1 – volume: 106 start-page: 811 issue: 4 year: 2020 ident: e_1_2_7_27_1 article-title: Measuring algorithmic fairness publication-title: Virgin Law Rev. contributor: fullname: Hellman D. – ident: e_1_2_7_32_1 doi: 10.1162/106365600568202 – start-page: 77 volume-title: Conference on Fairness, Accountability and Transparency year: 2018 ident: e_1_2_7_8_1 contributor: fullname: Buolamwini J – ident: e_1_2_7_5_1 – ident: e_1_2_7_17_1 doi: 10.1145/3287560.3287589 – ident: e_1_2_7_30_1 doi: 10.1109/ICDE48307.2020.00203 – ident: e_1_2_7_9_1 doi: 10.1145/3038912.3052660 – start-page: 8 volume-title: International Conference on Machine Learning AI for Social Good Workshop year: 2019 ident: e_1_2_7_21_1 contributor: fullname: Balashankar A – start-page: 3315 volume-title: Advances in Neural Information Processing Systems 29 year: 2016 ident: e_1_2_7_29_1 contributor: fullname: Hardt M – start-page: 107 volume-title: Proceedings of the 1st Conference on Fairness, Accountability and Transparency, Vol. 81 of Proceedings of Machine Learning Research year: 2018 ident: e_1_2_7_18_1 contributor: fullname: Menon AK – ident: e_1_2_7_24_1 doi: 10.1109/4235.996017 – start-page: 9 volume-title: 30th Conference on Neural Information Processing Systems (NIPS 2016) year: 2016 ident: e_1_2_7_6_1 contributor: fullname: Bolukbasi T – ident: e_1_2_7_28_1 doi: 10.1089/big.2016.0047 – volume: 20 start-page: 1 issue: 75 year: 2019 ident: e_1_2_7_22_1 article-title: Fairness constraints: a flexible approach for fair classification publication-title: J Mach Learn Res contributor: fullname: Zafar MB – ident: e_1_2_7_14_1 doi: 10.1145/3287560.3287596 – ident: e_1_2_7_19_1 doi: 10.1109/ICDM.2010.50 – start-page: 60 volume-title: Proceedings of the 35th International Conference on Machine Learning, Vol. 80 of Proceedings of Machine Learning Research year: 2018 ident: e_1_2_7_20_1 contributor: fullname: Agarwal A |
SSID | ssj0011745 |
Score | 2.5045328 |
Snippet | Fair machine learning has been focusing on the development of equitable algorithms that address discrimination. Yet, many of these fairness‐aware approaches... |
SourceID | proquest crossref wiley |
SourceType | Aggregation Database Publisher |
StartPage | 1619 |
SubjectTerms | Accuracy algorithmic fairness Algorithms Classifiers Decision making Decision trees group fairness Intelligent systems Machine learning multiobjective optimization |
Title | How fair can we go in machine learning? Assessing the boundaries of accuracy and fairness |
URI | https://onlinelibrary.wiley.com/doi/abs/10.1002%2Fint.22354 https://www.proquest.com/docview/2493783455 |
Volume | 36 |
hasFullText | 1 |
inHoldings | 1 |
isFullTextHit | |
isPrint | |
link | http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwnV3NS8MwFA_DkxfnJ06nBPHgpVubNm2KBxFxTMEdZIMJQslXxxBb2TqG_vW-pO2mgiDeUkhC-z6S30tefw-hc6Z4HKWRyWpkEKCAYh2h4JFwHvse1Sqw7PwPg7A_Cu7HdNxAl_W_MCU_xOrAzXiGXa-Ng3Mx765JQ6dZ0YG9jRouUEOkZwDR44o6ygOkTUsEGTjMi_yaVcgl3dXI73vRGmB-hal2n-k10XP9hmV6yUtnUYiO_PhB3vjPT9hGWxX-xNelweyghs52UbOu7YArV99DT_18ic1lDwbZ46XGkxxPM_xqcy81ropNTK5weWsMTQxQEgtbpcmE3zhPMZdyMePyHfNM2cnMurqPRr3b4U3fqcowONIHuOUQz5eKBJF0fRILwaKYu1ylzOMSFMygU6A9BZEGi4UXiVAoJkmYag3DWCilf4A2sjzThwgDVmQxjQgXMgyoSkWqPDAW7geB4NrlLXRWKyR5K9k2kpJXmSQgrMQKq4XataqSyuHmCUSRvqkZQmkLXViZ_z5BcjcY2sbR37seo01isllszk4bbRSzhT4BOFKIU2t3n1L5218 |
link.rule.ids | 315,786,790,1382,27955,27956,46327,46751 |
linkProvider | Wiley-Blackwell |
linkToHtml | http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwnV1JS8QwFH64HPTiLo5rEA9eOk7TpE1BEHFh3OYgI-hBSraKiB3RGUR_vS9pOy4giLcUktC-Jfle8vo9gC1hZJrkictqFBigoGIDZfCRSplGIbeGeXb-i07cvmKn1_x6BHbrf2FKfojhgZvzDL9eOwd3B9I7n6yh90W_iZsbZ6Mwju7OnVseXg7Jo0LE2rzEkCwQYRLVvEItujMc-n03-oSYX4Gq32mOp-G2fscyweShOeirpn7_Qd_434-YgakKgpL90mZmYcQWczBdl3cglbfPw02790rcfQ9B8ZNXS-565L4gjz790pKq3sTdHikvjrFJEE0S5Qs1uQic9HIitR48S_1GZGH8ZG5pXYCr46PuQTuoKjEEOkLEFdAw0oayRLcimiolklS2pMlFKDXqWGAnZkODwYZIVZioWBmhaZxbi8NErHW0CGNFr7BLQBAuipQnVCodM25ylZsQ7UVGjClpW7IBm7VGsqeScCMrqZVphsLKvLAasFrrKqt87iXDQDJyZUM4b8C2F_rvE2Qnna5vLP-96wZMtLsX59n5SedsBSapS27xKTyrMNZ_Htg1RCd9te6N8AMayt9_ |
linkToPdf | http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwnV1bS-QwFD54gcUXR10Xx2sQH3zpOE3SNsUHEXUYd3WQRUFhoeTWQcSOjDOI_npP0nZ0BUF8SyEJ7bkk30lOvwOwI4xMkzxxWY0CAxRUbKAMPlIpUxZG1nDPzn_ei7tX_Pd1dD0F-_W_MCU_xOTAzXmGX6-dgz-YfO-NNPS2GLVwb4v4NMzymFEXeR3_nXBHhQi1oxJC8kCECatphdp0bzL0_83oDWG-x6l-o-k04F_9imV-yV1rPFIt_fKBvfGb37AA8xUAJYelxSzClC2WoFEXdyCVr_-Em-7gibjbHoLCJ0-W9AfktiD3PvnSkqraRP-AlNfG2CSIJYnyZZpc_E0GOZFaj4dSPxNZGD-ZW1iX4apzcnnUDao6DIFmiLcCGjJtKE90m9FUKZGksi1NLkKpUcMCO3EbGgw1RKrCRMXKCE3j3FocJmKt2S-YKQaFXQGCYFGkUUKl0jGPTK5yE6K1SMa5krYtm7BdKyR7KOk2spJYmWYorMwLqwnrtaqyyuMeMwwjmSsaEkVN2PUy_3yC7LR36RurX--6BT8ujjvZ2WnvzxrMUZfZ4vN31mFmNBzbDYQmI7XpTfAVnezeLg |
openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=How+fair+can+we+go+in+machine+learning+%3F+Assessing+the+boundaries+of+accuracy+and+fairness&rft.jtitle=International+journal+of+intelligent+systems&rft.au=Valdivia%2C+Ana&rft.au=S%C3%A1nchez%E2%80%90Monedero%2C+Javier&rft.au=Casillas%2C+Jorge&rft.date=2021-04-01&rft.issn=0884-8173&rft.eissn=1098-111X&rft.volume=36&rft.issue=4&rft.spage=1619&rft.epage=1643&rft_id=info:doi/10.1002%2Fint.22354&rft.externalDBID=n%2Fa&rft.externalDocID=10_1002_int_22354 |
thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=0884-8173&client=summon |
thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=0884-8173&client=summon |
thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=0884-8173&client=summon |