My cousin in China bought me this book from China.
The title, Shu Xue Zhi Mei, can be translated literally to “The Beauty of Math,” but the content is on information theory and data mining. The author, Jun Wu, was a scientist in Google at its early stage. He graduated from Tsinghua University and Johns Hopkins University. He is an expert of natural language processing and search engines.
I just started reading this book. But I would like to share the very first section that I read and found very interesting. He told a story about a combination of entropy and information theory beautifully.
A function of languages is to convey information (while the theologians further say that language is related to act, in speech-act theory, in the doctrine of Scripture. See this.) Ancient Egyptians and Chinese invented hieroglyphs, a language system that represents information, which can be seen as clustering in the sense of machine learning. Indeed, a character or a symbol in Chinese do represent an area of meaning. And when we have more concepts, we introduce more characters, or equivalently, add more clusters. It is indeed what has been happening: the Chinese invented new words to cover new knowledge.
Thanks to the Phoenicians, phonetic languages actually reduce the problem of introducing new clusters that require much effort for human to learn. A combination of a small number of letters (or alphabets, or aleph-bets…), together with a set of grammar rules, can represent complicated enough concepts.
Later John von Neumann introduced the concept of information entropy, which is essentially the number of bits (0 or 1) that are required to represent a variety of concepts. See my previous post on entropy. Bit might be the most compact way of representing information, but redundancy in all languages is necessary in case of loss in transmission.
- Jun Wu, Shu Xue Zhi Mei (Beauty of Math), Posts and Telecom Press (2014). [Douban]
- Kwan-yuet Ho, “The Legacy of Entropy,” WordPress (2015). [WordPress]
- L. N. Hoang, “Shannon’s Information Entropy,” Science4All (2013). [Science4All]
- Richard S. Briggs, “Getting Involved: Speech Acts and Biblical Interpretation,” ANVIL 20, pp. 25-34 (2003).
- Kwan-yuet Ho, “Theological Outline of the Doctrine of Scripture,” WordPress (2016). [WordPress]