From finite-system entropy to entropy rate for a hidden Markov process

Or Zuk; Eytan Domany; Ido Kanter; Michael Aizenman

doi:10.1109/LSP.2006.874466

From finite-system entropy to entropy rate for a hidden Markov process

Or Zuk
, Eytan Domany
, Ido Kanter
, Michael Aizenman

Research output: Contribution to journal › Article › peer-review

13 Scopus citations

Abstract

A recent result presented the expansion for the entropy rate of a hidden Markov process (HMP) as a power series in the noise variable ε. The coefficients of the expansion around the noiseless (ε = 0) limit were calculated up to 11th order, using a conjecture that relates the entropy rate of an HMP to the entropy of a process of finite length (which is calculated analytically). In this letter, we generalize and prove the conjecture and discuss its theoretical and practical consequences.

Original language	English (US)
Pages (from-to)	517-520
Number of pages	4
Journal	IEEE Signal Processing Letters
Volume	13
Issue number	9
DOIs	https://doi.org/10.1109/LSP.2006.874466
State	Published - Sep 2006

All Science Journal Classification (ASJC) codes

Signal Processing
Electrical and Electronic Engineering
Applied Mathematics

Keywords

Entropy
Hidden markov process (HMP)
Taylor series

Access to Document

10.1109/LSP.2006.874466

Cite this

@article{0ce8b62c808e4b83a6c9e01ef5df15aa,

title = "From finite-system entropy to entropy rate for a hidden Markov process",

abstract = "A recent result presented the expansion for the entropy rate of a hidden Markov process (HMP) as a power series in the noise variable ε. The coefficients of the expansion around the noiseless (ε = 0) limit were calculated up to 11th order, using a conjecture that relates the entropy rate of an HMP to the entropy of a process of finite length (which is calculated analytically). In this letter, we generalize and prove the conjecture and discuss its theoretical and practical consequences.",

keywords = "Entropy, Hidden markov process (HMP), Taylor series",

author = "Or Zuk and Eytan Domany and Ido Kanter and Michael Aizenman",

note = "Funding Information: Manuscript received January 4, 2006; revised January 23, 2006. The work of M. Aizenman was supported in part by the Einstein Center for Theoretical Physics and in part by the Minerva Center for Nonlinear Physics. The work of I. Kanter at the Weizmann Institute was supported by the Einstein Center for Theoretical Physics. E. Domany and O. Zuk were supported in part by the Minerva Foundation and in part by the European Community{\textquoteright}s Human Potential Programme under Contract HPRN-CT-2002-00319, STIPCO. The associate editor coordinating the review of this manuscript and approving it for publication was Dr. Cihan Tepedelenlioglu.",

year = "2006",

month = sep,

doi = "10.1109/LSP.2006.874466",

language = "English (US)",

volume = "13",

pages = "517--520",

journal = "IEEE Signal Processing Letters",

issn = "1070-9908",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

number = "9",

}

TY - JOUR

T1 - From finite-system entropy to entropy rate for a hidden Markov process

AU - Zuk, Or

AU - Domany, Eytan

AU - Kanter, Ido

AU - Aizenman, Michael

N1 - Funding Information: Manuscript received January 4, 2006; revised January 23, 2006. The work of M. Aizenman was supported in part by the Einstein Center for Theoretical Physics and in part by the Minerva Center for Nonlinear Physics. The work of I. Kanter at the Weizmann Institute was supported by the Einstein Center for Theoretical Physics. E. Domany and O. Zuk were supported in part by the Minerva Foundation and in part by the European Community’s Human Potential Programme under Contract HPRN-CT-2002-00319, STIPCO. The associate editor coordinating the review of this manuscript and approving it for publication was Dr. Cihan Tepedelenlioglu.

PY - 2006/9

Y1 - 2006/9

N2 - A recent result presented the expansion for the entropy rate of a hidden Markov process (HMP) as a power series in the noise variable ε. The coefficients of the expansion around the noiseless (ε = 0) limit were calculated up to 11th order, using a conjecture that relates the entropy rate of an HMP to the entropy of a process of finite length (which is calculated analytically). In this letter, we generalize and prove the conjecture and discuss its theoretical and practical consequences.

AB - A recent result presented the expansion for the entropy rate of a hidden Markov process (HMP) as a power series in the noise variable ε. The coefficients of the expansion around the noiseless (ε = 0) limit were calculated up to 11th order, using a conjecture that relates the entropy rate of an HMP to the entropy of a process of finite length (which is calculated analytically). In this letter, we generalize and prove the conjecture and discuss its theoretical and practical consequences.

KW - Entropy

KW - Hidden markov process (HMP)

KW - Taylor series

UR - https://www.scopus.com/pages/publications/33747783917

UR - https://www.scopus.com/pages/publications/33747783917#tab=citedBy

U2 - 10.1109/LSP.2006.874466

DO - 10.1109/LSP.2006.874466

M3 - Article

AN - SCOPUS:33747783917

SN - 1070-9908

VL - 13

SP - 517

EP - 520

JO - IEEE Signal Processing Letters

JF - IEEE Signal Processing Letters

IS - 9

ER -

From finite-system entropy to entropy rate for a hidden Markov process

Abstract

All Science Journal Classification (ASJC) codes

Keywords

Access to Document

Other files and links

Fingerprint

Cite this