A review of entropy in mathematics: from numerical to operator forms

Document Type : Research Paper

Author

University of Zanjan

Abstract

The term entropy has acquired different meanings in various scientific disciplines, of which we refer to the most fundamental ones in this article. Here, we aim to initially provide a historical review of the concept of entropy from its numerical form to its operator form. Along the way, we will delve into some of the key theorems, inequalities, and properties of entropy.

Keywords

Main Subjects


[1] S. Abe and Y. Okamoto, Nonextensive statistical mechanics and its applications, Lecture Notes Physics, Springer-Verlage, Berlin, 2001.
[2] R. Badii and A. Politi, Complexity. Hierarchical structures and scaling in physics, Cambridge University Press, 1997.
[3] P. Castiglione, M. Falcioni, A. Lesne and A. Vulpiani, Chaos and coarse-graining in statistical mechanics, Cambridge University Press, 2008.
[4] T. M. Cover and J. A. Thomas, Elements of information theory, 2nd edition, Wiley, New York, 2006.
[5] J. I. Fujii, Operator means and the relative operator entropy, Oper. Theory Adv. Appl., 59 (1992) 161–172.
[6] J. I. Fujii, M. Fujii and Y. Seo, An extension of Kubo-Ando theory: Solidarities, Math. Japonica, 35 (1990) 387–396.
[7] J. I. Fujii and E. Kamei, Uhlmann’s interpolational method for operator means, Math. Japonica, 34 (1989) 541–547.
[8] J. I. Fujii and Y. Seo, Tsallis relative operator entropy with negative parameters, Adv. Oper. Theory, 1 (2016) 219–235.
[9] S. Furuichi, K. Yanagi and K. Kuriyama, A note on operator inequalities of Tsallis relative operator entropy, Linear Algebra Appl., 407 (2005) 19–31.
[10] T. Furuta, Parametric extensions of Shannon inequality and its reverse one in Hilbert space operators, Linear Algebra Appl., 381 (2004) 219–235.
[11] T. Furuta, Reverse inequalities involving two relative operator entropies and two relative entropies, Linear Algebra Appl., 403 (2005) 24–30.
[12] T. Furuta, J. Mićić Hot, J. E. Pečarić and Y. Seo, Mond-Pečarić method in operator inequalities, Element, Zagreb, 2005.
[13] R. M. Gray, Entropy and information theory, Springer, New York, 1990, Available at http://ee.stanford. edu/ gray/it.html.
[14] J. Honerkamp, Statistical physics, Springer, Berlin, 1998.
[15] E. T. Jaynes, Information theory and statistical mechanics Part I, Phys. Rev., 106 (1957) 620–630. Part II, Phys. Rev., 108 171–190.
[16] E. T. Jaynes, The minimum entropy production principle, Ann. Rev. Phys. Chem., 31 (1980) 579–601.
[17] E. T. Jaynes, On the rationale of maximum entropy methods, Proc. IEEE, 70 (1982) 939–952.
[18] G. A. Jones and J. M. Jones, Information and coding theory, Springer-Verlag, 2000.
[19] S. Kullback and R. A. Leibler, On information and sufficiency, Ann. Math. Statist., 22 (1951) 79–86.
[20] A. Lesne, J. L. Blanc and L. Pezard, Entropy estimation of very short symbolic sequences, Phys. Rev. E., 79 (2009) 046208.
[21] E. H. Lieb and M. B. Ruskai, Proof of the strong subadditivity of quantum-mechanical entropy, J. Mathematical Phys., 14 (1973) 1938–1941.
[22] T. Maszczyk and W. Duch, Comparison of Shannon, Renyi and Tsallis entropy used in decision trees, ICAISC, LNAI, 5097 (2008) 643–651.
[23] F. Mirzapour, M. S. Moslehian and A. Morassaei, Operator Bellman type inequalities, Quaest. Math., 37 (2014) 9–17.
[24] M. S. Moslehian and M. Kian, Non-commutative f -divergence functional, Math. Nachr., 286 (2013) 1514–1529.
[25] M. S. Moslehian, F. Mirzapour and A. Morassaei, Operator entropy inequalities, Colloq. Math., 37 (2014) 159–168.
[26] J. Rooin and A. Morassaei, Some refinements of relative information inequality, Creative Math. & Inf., 16 (2007) 95–98.
[27] C. Shannon, A mathematical theory of communication, Bell System Tech. J., 27 (1948) 379–423.
[28] C. Tsallis, Possible generalization of Boltzman-Gibbs statistics, J. Statist. Phys., 52 (1988) 479–487.
[29] H. Umegaki, Condition expectation in an operator algebra IV, Kodai Math. Sem. Rep., 14 (1962) 59–85.
[30] K. Yanagi, K. Kuriyama and S. Furuichi, Generalized Shannon inequalities based on Tsallis relative operator entropy, Linear
[31] S. H. P. Masiha, Probability information and coding theory, K. N. Toosi University Press, 2013.
[32] D. S. Jones, Elementary Information theory, N. R. Arghami and M. A. Poorabdollah nejad, 1998.
Volume 2, Issue 4 - Serial Number 4
December 2017
Pages 71-84
  • Receive Date: 17 September 2017
  • Revise Date: 10 October 2017
  • Accept Date: 04 November 2017
  • Publish Date: 20 February 2018