Lossy Compression of Discrete Sources via the Viterbi Algorithm
We present a new lossy compressor for finite-alphabet sources. For coding a sequence x n , the encoder starts by assigning a certain cost to each possible reconstruction sequence. It then finds the one that minimizes this cost and describes it losslessly to the decoder via a universal lossless compr...
Saved in:
| Published in | IEEE transactions on information theory Vol. 58; no. 4; pp. 2475 - 2489 |
|---|---|
| Main Authors | , , |
| Format | Journal Article |
| Language | English |
| Published |
New York, NY
IEEE
01.04.2012
Institute of Electrical and Electronics Engineers The Institute of Electrical and Electronics Engineers, Inc. (IEEE) |
| Subjects | |
| Online Access | Get full text |
| ISSN | 0018-9448 1557-9654 |
| DOI | 10.1109/TIT.2011.2178059 |
Cover
Loading…
| Abstract | We present a new lossy compressor for finite-alphabet sources. For coding a sequence x n , the encoder starts by assigning a certain cost to each possible reconstruction sequence. It then finds the one that minimizes this cost and describes it losslessly to the decoder via a universal lossless compressor. The cost of each sequence is a linear combination of its distance from the sequence x n and a linear function of its k th order empirical distribution. The structure of the cost function allows the encoder to employ the Viterbi algorithm to find the sequence with minimum cost. We identify a choice of the coefficients used in the cost function which ensures that the algorithm universally achieves the optimum rate-distortion performance for any stationary ergodic source, in the limit of large , provided that increases as o(log n). Iterative techniques for approximating the coefficients, which alleviate the computational burden of finding the optimal coefficients, are proposed and studied. |
|---|---|
| AbstractList | We present a new lossy compressor for finite-alphabet sources. For coding a sequence x n , the encoder starts by assigning a certain cost to each possible reconstruction sequence. It then finds the one that minimizes this cost and describes it losslessly to the decoder via a universal lossless compressor. The cost of each sequence is a linear combination of its distance from the sequence x n and a linear function of its k th order empirical distribution. The structure of the cost function allows the encoder to employ the Viterbi algorithm to find the sequence with minimum cost. We identify a choice of the coefficients used in the cost function which ensures that the algorithm universally achieves the optimum rate-distortion performance for any stationary ergodic source, in the limit of large , provided that increases as o(log n). Iterative techniques for approximating the coefficients, which alleviate the computational burden of finding the optimal coefficients, are proposed and studied. We present a new lossy compressor for finite-alphabet sources. For coding a sequence $x^{n}$, the encoder starts by assigning a certain cost to each possible reconstruction sequence. It then finds the one that minimizes this cost and describes it losslessly to the decoder via a universal lossless compressor. The cost of each sequence is a linear combination of its distance from the sequence $x^{n}$ and a linear function of its $k^{rm th}$ order empirical distribution. The structure of the cost function allows the encoder to employ the Viterbi algorithm to find the sequence with minimum cost. We identify a choice of the coefficients used in the cost function which ensures that the algorithm universally achieves the optimum rate-distortion performance for any stationary ergodic source, in the limit of large $n$, provided that $k$ increases as $o(log n)$. Iterative techniques for approximating the coefficients, which alleviate the computational burden of finding the optimal coefficients, are proposed and studied. [PUBLICATION ABSTRACT] |
| Author | Montanari, A. Jalali, S. Weissman, T. |
| Author_xml | – sequence: 1 givenname: S. surname: Jalali fullname: Jalali, S. email: shirin@caltech.edu organization: Center for the Math. of Inf., California Inst. of Technol., Pasadena, CA, USA – sequence: 2 givenname: A. surname: Montanari fullname: Montanari, A. email: montanari@stanford.edu organization: Dept. of Electr. Eng., Stanford Univ., Stanford, CA, USA – sequence: 3 givenname: T. surname: Weissman fullname: Weissman, T. email: tsachy@stanford.edu organization: Center for the Math. of Inf., California Inst. of Technol., Pasadena, CA, USA |
| BackLink | http://pascal-francis.inist.fr/vibad/index.php?action=getRecordDetail&idt=25721960$$DView record in Pascal Francis |
| BookMark | eNp9kM1LAzEQxYNUsK3eBS-L4HHr5GuzOUmpX4WCB6vXJZtNbEq7W5NU6H9vSksPHjwNw7w3b-Y3QL22aw1C1xhGGIO8n0_nIwIYjwgWJXB5hvqYc5HLgrMe6gPgMpeMlRdoEMIytYxj0kcPsy6EXTbp1htvQnBdm3U2e3RBexNN9t5tvTYh-3EqiwuTfbpofO2y8eqr8y4u1pfo3KpVMFfHOkQfz0_zyWs-e3uZTsazXFMCMbdAmQXbgBBGMNsUxJaCYNrImkrFBccyzRpBlAZGjSKNJTWUwISBUteCDtHtYe_Gd99bE2K1TKe1KbKSRIKgOG0borujSAWtVtarVrtQbbxbK7-rCE-RsoCkg4NO-_S9N_YkwVDtaVaJZrWnWR1pJkvxx6JdVDHxil651X_Gm4PRGWNOOQVIAVzQX2Dgggw |
| CODEN | IETTAW |
| CitedBy_id | crossref_primary_10_1109_TIT_2013_2293339 crossref_primary_10_1109_TIT_2015_2428238 crossref_primary_10_1016_j_acha_2015_03_003 crossref_primary_10_1109_TCOMM_2020_3020796 crossref_primary_10_1093_imaiai_iax016 crossref_primary_10_1109_TIT_2019_2947487 crossref_primary_10_1109_TNET_2013_2283071 crossref_primary_10_1134_S0032946012040060 |
| Cites_doi | 10.1109/ISIT.2007.4557286 10.1109/TIT.2009.2021379 10.1109/TIT.1975.1055440 10.1109/TIT.1980.1056164 10.1109/PROC.1973.9030 10.1109/TIT.1970.1054394 10.1109/18.623145 10.1214/aoap/1029962749 10.1109/TIT.1971.1054604 10.1109/29.17498 10.1109/18.904515 10.1109/TIT.2011.2145930 10.1109/TIT.2009.2016040 10.1109/18.108250 10.1109/18.623143 10.1109/ISIT.2005.1523592 10.1109/26.46532 10.1109/83.806615 10.1109/TIT.1975.1055439 10.1109/TIT.1982.1056474 10.1109/TIT.1967.1054010 10.1109/18.256495 10.1080/01621459.1963.10500830 10.1109/18.749005 10.1109/TIT.1978.1055934 10.1109/TIT.2010.2040961 10.1007/978-1-4615-3626-0 10.1109/ISIT.2008.4595107 10.1109/TIT.1972.1054831 10.1109/ISIT.2008.4595106 10.1109/34.777372 10.1109/18.720552 10.1109/18.650987 10.1109/TIT.1978.1055881 10.1145/214762.214771 |
| ContentType | Journal Article |
| Copyright | 2015 INIST-CNRS Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) Apr 2012 |
| Copyright_xml | – notice: 2015 INIST-CNRS – notice: Copyright The Institute of Electrical and Electronics Engineers, Inc. (IEEE) Apr 2012 |
| DBID | 97E RIA RIE AAYXX CITATION IQODW 7SC 7SP 8FD JQ2 L7M L~C L~D |
| DOI | 10.1109/TIT.2011.2178059 |
| DatabaseName | IEEE Xplore (IEEE) IEEE All-Society Periodicals Package (ASPP) 1998–Present IEEE Xplore Digital Library CrossRef Pascal-Francis Computer and Information Systems Abstracts Electronics & Communications Abstracts Technology Research Database ProQuest Computer Science Collection Advanced Technologies Database with Aerospace Computer and Information Systems Abstracts Academic Computer and Information Systems Abstracts Professional |
| DatabaseTitle | CrossRef Technology Research Database Computer and Information Systems Abstracts – Academic Electronics & Communications Abstracts ProQuest Computer Science Collection Computer and Information Systems Abstracts Advanced Technologies Database with Aerospace Computer and Information Systems Abstracts Professional |
| DatabaseTitleList | Technology Research Database |
| Database_xml | – sequence: 1 dbid: RIE name: IEEE Electronic Library (IEL) url: https://proxy.k.utb.cz/login?url=https://ieeexplore.ieee.org/ sourceTypes: Publisher |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Engineering Computer Science Applied Sciences |
| EISSN | 1557-9654 |
| EndPage | 2489 |
| ExternalDocumentID | 2613734191 25721960 10_1109_TIT_2011_2178059 6097057 |
| Genre | orig-research Feature |
| GroupedDBID | -~X .DC 0R~ 29I 3EH 4.4 5GY 5VS 6IK 97E AAJGR AARMG AASAJ AAWTH ABAZT ABFSI ABQJQ ABVLG ACGFO ACGFS ACGOD ACIWK AENEX AETEA AETIX AGQYO AGSQL AHBIQ AI. AIBXA AKJIK AKQYR ALLEH ALMA_UNASSIGNED_HOLDINGS ASUFR ATWAV BEFXN BFFAM BGNUA BKEBE BPEOZ CS3 DU5 E.L EBS EJD F5P HZ~ H~9 IAAWW IBMZZ ICLAB IDIHD IFIPE IFJZH IPLJI JAVBF LAI M43 MS~ O9- OCL P2P PQQKQ RIA RIE RNS RXW TAE TN5 VH1 VJK AAYOK AAYXX CITATION RIG IQODW 7SC 7SP 8FD JQ2 L7M L~C L~D |
| ID | FETCH-LOGICAL-c320t-f034f0fd077e74fd62f87213d9b39a57519d07d72ac043ea2df2b08047e08cb73 |
| IEDL.DBID | RIE |
| ISSN | 0018-9448 |
| IngestDate | Sun Jun 29 15:40:32 EDT 2025 Mon Jul 21 09:14:31 EDT 2025 Tue Jul 01 02:34:30 EDT 2025 Thu Apr 24 23:00:06 EDT 2025 Tue Aug 26 17:18:12 EDT 2025 |
| IsPeerReviewed | true |
| IsScholarly | true |
| Issue | 4 |
| Keywords | Alphabet Lossy compression Performance evaluation Cost minimization Lossless circuit Coding circuit Rate distortion theory Ergodicity Lossy medium Iterative method universal compression Decoding Algorithm Viterbi code Viterbi algorithm Linear combination Coding Rate-distortion coding Cost function |
| Language | English |
| License | https://ieeexplore.ieee.org/Xplorehelp/downloads/license-information/IEEE.html CC BY 4.0 |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-c320t-f034f0fd077e74fd62f87213d9b39a57519d07d72ac043ea2df2b08047e08cb73 |
| Notes | SourceType-Scholarly Journals-1 ObjectType-Feature-1 content type line 14 |
| PQID | 929073121 |
| PQPubID | 36024 |
| PageCount | 15 |
| ParticipantIDs | ieee_primary_6097057 crossref_primary_10_1109_TIT_2011_2178059 pascalfrancis_primary_25721960 proquest_journals_929073121 crossref_citationtrail_10_1109_TIT_2011_2178059 |
| ProviderPackageCode | CITATION AAYXX |
| PublicationCentury | 2000 |
| PublicationDate | 2012-04-01 |
| PublicationDateYYYYMMDD | 2012-04-01 |
| PublicationDate_xml | – month: 04 year: 2012 text: 2012-04-01 day: 01 |
| PublicationDecade | 2010 |
| PublicationPlace | New York, NY |
| PublicationPlace_xml | – name: New York, NY – name: New York |
| PublicationTitle | IEEE transactions on information theory |
| PublicationTitleAbbrev | TIT |
| PublicationYear | 2012 |
| Publisher | IEEE Institute of Electrical and Electronics Engineers The Institute of Electrical and Electronics Engineers, Inc. (IEEE) |
| Publisher_xml | – name: IEEE – name: Institute of Electrical and Electronics Engineers – name: The Institute of Electrical and Electronics Engineers, Inc. (IEEE) |
| References | ref35 ref13 ref34 garcia-munoz (ref11) 1982; it 28 ref12 ref37 ref36 ref31 ref30 ref33 ref32 ref10 ref17 ref38 ref16 ref19 ref18 ref24 ref23 ref26 berger (ref4) 1971 ref25 gallager (ref3) 1968 ref20 ref41 ref22 ref21 ref28 ref27 morita (ref15) 1989 gray (ref39) 1971; it 17 ref29 ref8 ref7 ref9 cover (ref1) 2006 cheung (ref14) 1990 ref6 ref5 shannon (ref2) 1960 ref40 |
| References_xml | – ident: ref38 doi: 10.1109/ISIT.2007.4557286 – ident: ref29 doi: 10.1109/TIT.2009.2021379 – ident: ref37 doi: 10.1109/TIT.1975.1055440 – ident: ref10 doi: 10.1109/TIT.1980.1056164 – ident: ref36 doi: 10.1109/PROC.1973.9030 – start-page: 93 year: 1960 ident: ref2 publication-title: Information and Decision Processes – ident: ref6 doi: 10.1109/TIT.1970.1054394 – ident: ref5 doi: 10.1109/18.623145 – ident: ref21 doi: 10.1214/aoap/1029962749 – volume: it 17 start-page: 127 year: 1971 ident: ref39 article-title: Rate distortion functions for finite-state finite-alphabet markov sources publication-title: IEEE Trans Inform Theory doi: 10.1109/TIT.1971.1054604 – ident: ref32 doi: 10.1109/29.17498 – year: 2006 ident: ref1 publication-title: Elements of Information Theory – ident: ref19 doi: 10.1109/18.904515 – ident: ref31 doi: 10.1109/TIT.2011.2145930 – year: 1989 ident: ref15 article-title: An extension of LZW coding algorithm to source coding subject to a fidelity criterion publication-title: Proc 4th Joint Swedish-Soviet Int Workshop on Information Theory – year: 1968 ident: ref3 publication-title: Information Theory and Reliable Communication – ident: ref27 doi: 10.1109/TIT.2009.2016040 – start-page: 1473 year: 1990 ident: ref14 article-title: A locally adaptive source coding scheme publication-title: Proc Bilkent Conf on New Trends in Communication Control and Signal Processing – ident: ref40 doi: 10.1109/18.108250 – ident: ref18 doi: 10.1109/18.623143 – ident: ref26 doi: 10.1109/ISIT.2005.1523592 – ident: ref22 doi: 10.1109/26.46532 – ident: ref25 doi: 10.1109/83.806615 – ident: ref8 doi: 10.1109/TIT.1975.1055439 – volume: it 28 start-page: 285 year: 1982 ident: ref11 article-title: Strong universal source coding subject to a rate-distortion constraint publication-title: IEEE Trans Inform Theory doi: 10.1109/TIT.1982.1056474 – ident: ref35 doi: 10.1109/TIT.1967.1054010 – ident: ref16 doi: 10.1109/18.256495 – ident: ref41 doi: 10.1080/01621459.1963.10500830 – ident: ref34 doi: 10.1109/18.749005 – ident: ref12 doi: 10.1109/TIT.1978.1055934 – ident: ref30 doi: 10.1109/TIT.2010.2040961 – ident: ref24 doi: 10.1007/978-1-4615-3626-0 – ident: ref33 doi: 10.1109/ISIT.2008.4595107 – ident: ref7 doi: 10.1109/TIT.1972.1054831 – ident: ref28 doi: 10.1109/ISIT.2008.4595106 – ident: ref20 doi: 10.1109/34.777372 – ident: ref23 doi: 10.1109/18.720552 – year: 1971 ident: ref4 publication-title: Rate-Distortion Theory A Mathematical Basis for Data Compression – ident: ref17 doi: 10.1109/18.650987 – ident: ref9 doi: 10.1109/TIT.1978.1055881 – ident: ref13 doi: 10.1145/214762.214771 |
| SSID | ssj0014512 |
| Score | 2.0885842 |
| Snippet | We present a new lossy compressor for finite-alphabet sources. For coding a sequence x n , the encoder starts by assigning a certain cost to each possible... We present a new lossy compressor for finite-alphabet sources. For coding a sequence $x^{n}$, the encoder starts by assigning a certain cost to each possible... |
| SourceID | proquest pascalfrancis crossref ieee |
| SourceType | Aggregation Database Index Database Enrichment Source Publisher |
| StartPage | 2475 |
| SubjectTerms | Algorithm design and analysis Algorithms Applied sciences Approximation Approximation algorithms Approximation methods Codes Coding, codes Compression algorithms Entropy Exact sciences and technology Information theory Information, signal and communications theory Rate-distortion coding Signal and communications theory Telecommunications and information theory universal compression Vectors Viterbi algorithm |
| Title | Lossy Compression of Discrete Sources via the Viterbi Algorithm |
| URI | https://ieeexplore.ieee.org/document/6097057 https://www.proquest.com/docview/929073121 |
| Volume | 58 |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| link | http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwjV1LT8QgEJ6oJz34Nq6vcPBiYncpZdtyMsZH1KgXV-OtKRR0o-4at2uiv94B2sZXjLcm0BYYGL5hhm8AtkPhQhNVEJu0G3CZ54FUPAwUbtYqNVxQR_Z8cRmfXPOz2-7tBOw2d2G01i74TLfto_PlF0M1tkdlnZiKBPHFJEyi4ebvajUeA94NPTN4iAsYbY7aJUlFp3fa81ydCL9Tx0r6aQtyOVVsRGQ-wkExPpvFD8XsdpvjObio2-mDTB7a41K21fs3Csf_dmQeZivYSfb9PFmACT1YhLk6pQOpVvgizHziJ1yCvXNs_xux1Xy47IAMDTnso6pBrE2u3MH_iLz2c4I4ktzY68yyT_Yf74Yv_fL-aRmuj496BydBlXEhUBGjZWBoxA01BU0SnXBTxMykaCJGhZCRyK2LRmBZkbBcUR7pnBWGScScPNE0VTKJVmBqMBzoVSBhIYxiCTNKaZ4aKlE1UmX1q0TQkeoWdGohZKqiI7dZMR4zZ5ZQkaHYMiu2rBJbC3aaN549FccfdZfsqDf1qgFvwdYXOTflqLiwfTFtwXot-KxazKMMESQqwpCFa79_dR2m8d_Mx_NswFT5MtabCFVKueXm6Ad-jeNx |
| linkProvider | IEEE |
| linkToHtml | http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwjV3dT9swED8xeGA88D1RvuaHvSAtreO4Sfw0IRgqo-VlZeItih2bVZQW0RQJ_vqd7SRiG0K8RbITXXz23c--8-8AvoTCpSaqIDZpN-AyzwOpeBgodNYqNVxQR_Y8uIx7V_zHdfd6Ab42d2G01i75TLfto4vlF1M1t0dlnZiKBPHFB1hCv98N_W2tJmbAu6HnBg9xCeOuow5KUtEZng89WycC8NTxkr5wQq6qis2JzGc4LMbXs_jPNDt_c7YGg1pSn2Zy256Xsq2e_yFxfO-vrMNqBTzJsZ8pG7CgJ5uwVhd1INUa34SVFwyFW_Ctj_I_EdvNJ8xOyNSQ0xEaG0Tb5Kc7-p-Rx1FOEEmSX_ZCsxyR4_HN9GFU_r7bhquz78OTXlDVXAhUxGgZGBpxQ01Bk0Qn3BQxMyluEqNCyEjkNkgjsK1IWK4oj3TOCsMkok6eaJoqmUSfYHEynegdIGEhjGIJM0ppnhoq0ThSZS2sRNiR6hZ0aiVkqiIkt3UxxpnbmFCRodoyq7asUlsLjpo37j0Zxxt9t-yoN_2qAW_B4V96btrRdKF8MW3BXq34rFrOswwxJJrCkIW7r3_1Myz3hoN-1j-_vNiDjygH89k9-7BYPsz1AQKXUh66-foHhD_mug |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Lossy+Compression+of+Discrete+Sources+via+the+Viterbi+Algorithm&rft.jtitle=IEEE+transactions+on+information+theory&rft.au=Jalali%2C+S.&rft.au=Montanari%2C+A.&rft.au=Weissman%2C+T.&rft.date=2012-04-01&rft.pub=IEEE&rft.issn=0018-9448&rft.volume=58&rft.issue=4&rft.spage=2475&rft.epage=2489&rft_id=info:doi/10.1109%2FTIT.2011.2178059&rft.externalDocID=6097057 |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=0018-9448&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=0018-9448&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=0018-9448&client=summon |