Shreider's Semantic Theory of Information

          Carnap's semantic theory of information may be contrasted with a more recent semantic information theory proposed by the Russian information scientist, Yu A. Shreider (also rendered from the Russian as Ju A. Srejder).  In his "Basic Trends in the Field of Semantics" in Statistical Methods in Linguis­tics (1971) Shreider distinguishes three classifications or trends in works on semantics, and he relates his views to Carnap's in this context.  The three classifications are ontological semantics, logical semantics, and linguistic semantics.  He says that all three of these try to solve the same problem: to ascertain what meaning is and how it can be described.  The first classification, ontological semantics, is the study of the various philosophical aspects of the relation between sign and signified.  He says that it inquires into the very nature of existence, into the degrees of reality possessed by signified objects, classes and situations, and that it is closely related to the logic and methodology of science and to the theoretical foundations of library classification.
          The second classification, logical semantics, studies formal sign systems as opposed to natural languages.  This is the trend in which he locates Carnap, as well as Quine, Tarski, and Bar-Hillel.  The semantical systems con­sidered in logical semantics are basic to the metatheory of the sciences.  The meaning postulates determine the class of permissible models for a given system of formal relations.  A formal theory fixes a class of syntactical relations, whence there arises a fixed system of semantic relations between a text describing a possible world. 
          The third classification, linguistic semantics, seeks to elucidate the inherent organization in a natural language, to formulate the inherent regularities in texts and to construct a system of basic semantic relations.  The examination of properties of extralinguistic reality, which determines permissible semantic relations and the ways of combining them, is carried considerably farther in lin­guistic semantics than in logical semantics, where the question is touched upon only in the selection of meaning postulates.  However, linguistic semantics is still rather vague and inexact, being an auxiliary investigation in lin­guistics used only as necessity dictates.  Shreider locates his work midway between logical and linguistic semantics, because it involves the examination of natural language texts with logical calculi.
          Shreider's theory is a theory of communication that explains phenomena not explained by Shannon's statistical theory.  Bibliographies in Shreider's English-language articles contain references to Carnap's and Bar-Hillel's 1953 paper, and Shreider explicitly advocates Carnap's explication of intensional synonymy in terms of L-equiva­lence.  But Shreider's theory is more accurately described as a development of Shannon's theory, even though Shreider's theory is not statistical.  English language works by Shreider include "On the Semantic Characteristics of Infor­mation" in Information Storage and Retrieval (1965), which is also reprinted in Introduction to Information Science (ed. Tefko Saracevic, 1970), and "Semantic Aspects of Infor­mation Theory" in On Theoretical Problems On Information (Moscow, 1969).  Furthermore comments on Shreider and other contributors to Russian information science (or "informatics" as it is called in Russia) can be found in "Some Soviet Concepts of Information for Information Science" in the American Society for Information Science Journal (1975) by Nicholas J. Belkin.
          Like many information scientists who take up semantical considerations, Shreider notes that there are many situa­tions involving information, in which one may wish to consi­der the content of the message signals instead of the sta­tistical frequency of signal transmission considered by Shannon's theory.  But Shreider furthermore maintains that a semantical concept of information implies an alternative theory of communication in contrast to Shannon's "classical" theory.  Shannon's concept pertains only to the potential ability of the receiver to determine from a given message text a quantity of information; it does not account for the information that the receiver can effectively derive from the message, that is, the receiver's ability to "understand" the message.  In Shreider's theory the knowledge had by the receiver prior to receiving the message is considered, in order to determine the amount of information effectively communicated.
          More specifically, in Shannon's probability-theoretic approach, before even considering the information contained in a message about some event, it is necessary to consider the a priori probability of the event.  Furthermore according to Shannon's first theorem, in the optimum method of coding a statement containing more information requires more binary symbols or bits.  In Shreider's view, however, a theory of information should be able to account for cases that do not conform to this theorem.  For example much information is contained in a statement describing a newly discovered chemical element, which could be coded in a small number of binary symbols, and for which it would be meaningless to speak of an a priori probability.  On the other hand a statement describing the measurements of the well known physicochemical properties of some substance may be considerably less informative, while it may need a much more extensive description for its coding.  The newly discovered element will change our knowledge about the world much more than measurement of known substances.  Shreider maintains that a theory of information that can take into account the receiver's ability to "understand" a message must include a description of the receiver's background knowledge.  For this reason his information theory includes a thesaurus, by which is meant a unilingual dictionary showing the semantic connections among its constituent words.            Let T denote such a thesaurus to represent a guide in which there is recorded our knowledge about the real world.  The thesaurus T can be in any one of various states, and it can change or be trans­formed from one state to another.  Let M represent a received message, which can transform the thesaurus T.  Then the concept of amount of information, denoted L(T,M), may be defined as the degree of change in the thesaurus T under the action of a given statement M.  And for each admissible text M expressed in a certain code or language, there corresponds a certain transformation operator q, which acts on thesaurus T.  The salient point is that the amount of information con­tained in the statement M relative to the thesaurus T is characterized by the degree of change in the thesaurus under the action of the communicated statement.  And the under­standing of the communicated statement depends on the state of the receiver's thesaurus.  Accordingly the thesaurus T can understand some statements and not others.  There are some statements that cannot be understood by a given thesaurus, and the information for such a thesaurus is zero, which is to say L(T, M)=0, because the thesaurus T is not transformed at all.  One such case is that of a student or a lay­man who does not have the background to understand a transmitted message about a specialized subject.  Another case is that of someone who already knows the transmitted information, so that it is redundant to what the receiver already knows.  In this case too there is no informa­tion communicated, and again L(T,M)=0, but in this case it is because the thesaurus T has been transformed into its initial state.  The interesting situation is that in which the receiver's thesaurus is sufficiently developed that he under­stands the transmitted message, but still finds his thesaurus transformed into a new and different state as a result of receipt of the new information.  If the rules of construction of the trans­formation operator q are viewed as external to the thesaurus T, then the quantity L(T,M) depends on these rules.  And when the transformation operator q is also revised, a preliminary increase of the knowledge stored in the thesaurus T may not only decrease the quantity of information L(T,M), but can also increase it.  Thus some­one who has learned a branch of a science will derive more information from a special text in the branch than he would before he had learned it.  This peculiar property of the semantic theory of information basically distinguishes it from the Shannon's classical theory, in which the increase in a priori information always decreases the amount of information from a message statement M.  In the classical theory there is no question of a receiver's degree of "understand­ing" of a statement; it is always assumed that he is "tuned.”  But in the semantic theory the essential role is played by the very possibility of correct "tuning" of the receiver.
          In his 1975 article Belkin reports that Shreider further developed his theory of information to include the idea of “meta-information.”  Meta-information is information about the mode of the coding of information, i.e. the knowledge about the relation between information and the text in which it is coded.  In this sense of meta-information the recei­ver's thesaurus must contain meta-information in order to understand the information in the received message text, because it enables the receiver to analyze the organization of the semantic information, such as that which reports scientific research findings.  Shreider maintains that informatics, the Russian equivalent to information sci­ence, is concerned not with information as such, but rather with meta-information, and specifically with information as to how scientific information is distributed and organized. Therefore, with his concept of meta-information Shreider has reportedly modified his original theory of communication by analyzing the thesaurus T into two components, such that T=(Tm,To).  The first component Tm consists of the set of rules needed for extracting elementary messages from the text M, while the second component To consists of the fac­tual information that relates those elementary messages sys­tematically and enables the elements to be integrated in T.  The relationship between Tm and To is such that a decrease in the redundancy of coding of To requires an increase of the meta-information in Tm for the decoding of the coding system used for To.  Hence the idea of meta-information may be a means of realizing some limiting efficiency laws for information by analyzing the dependency relation between information and the amount of meta-information necessary to comprehend that information.
          It would appear that if the coding system is taken as a language, then Shreider’s concept of meta-information might include to the idea of metalanguage as used by Carnap and other analytical philosophers, or it might be incorporated into the metalanguage.  Then the elements Tm and To are distinguished as metalanguage and object language respectively, although the philosophers have had little interest in exam­ining the inverse dependency between them.

The Philosophy of Science

Aim of Science

          Carnap’s explicit statement of the aim of sci­ence is set forth in his Aufbau.  The aim of science consists in finding and ordering true propositions firstly through the formulation of the constructional system - the introduction of concepts - and secondly through the ascertainment of the empirical connections between the concepts.  This is completely programmatic, and says nothing about what scientists actually do in their research practices.  For most contemporary philosophers a discussion of the aim of science is a discussion in the pragmatics of science, that is, what the scientist does as a user of scientific language when he does research.  But Carnap identifies the pragmatics of language with the empirical investigation of historically given natural lan­guages.  He always constructs his own languages usually using Russell's symbolic logic, and then uses these artificial languages to address the philosophical problems of interest to the Posi­tivist program for philosophy, namely, the reduction of theoretical terms to demonstrate their meaningfulness and the reduction of the vocabulary of science to the common basis set forth in the Aufbau, to advance its unification.

Scientific Explanation

          Carnap also has explicit views on scientific explanation: He says it always involves laws, and he classifies scientific laws as either empirical laws or theoretical laws.  Empirical laws explain facts, which are statements that describe individual instan­ces.  The explanation has the logical structure of a deduction.  The premises of the deduction consist of at least one law that has the form of a conditional statement, and statements of fact that describe individual instances in the same terms as those occurring in the antecedent sentences of the conditional law.  The conclusion is also a factual sen­tence that describes the individual instances in the same terms as those in the consequent sentence of the conditional law.  In this manner empirical laws explain observed instan­ces described by factual statements.  Theoretical laws are related to empirical laws in a way that is analogous to the way that empirical laws are related to facts.  The theore­tical law is more general.  It helps to explain deductively empirical laws that are already known and to permit the derivation of new empirical laws, just as the empirical laws help to explain facts that have been observed and to predict new facts.  Furthermore the theoretical law puts several empirical laws into an orderly pattern, just as the empiri­cal generalization puts many facts into an orderly pattern.  The supreme value of theory is its power to predict new empirical laws; explaining known laws is of minor value.  Every revolutionary theory in the history of science has predicted new empirical laws that are confirmed by empirical tests.
          Unlike Duhem, Carnap does not stratify the semantics of physics.  To say that theoretical laws explain empirical laws is not for Carnap to say as Duhem did, that the theory is an axiomatic system with conclusions that are statements which parallel the empirical laws, and that have their own semantics that in turn refers to the empirical laws.  In Carnap's view the theoretical terms receive all their seman­tics from the observation terms by means of reduction sen­tences which he calls “correspondence rules.”  When Carnap says that theoretical laws explain empirical laws, he means that a deductive relationship is established between the axioms of the theory and the empirical laws, and that the relationship is mediated by the correspondence rules.  The postulated axioms, which are the theoretical laws, together with the correspondence rules enable the physicist to explain empiri­cal laws by logical deduction.  In Carnap's philosophy the numerical approximation that Duhem saw existing between the solution sets for the equation deduced from the axioms on the one hand and the solution sets for the equation the empirical laws on the other hand, has no semantical implica­tions and is not problematic.  The post-Positivist philoso­phers agree with Duhem, and maintain that while the numeri­cal difference between theoretical and empirical laws are experimentally indistinguishable due to measurement error, nonetheless the solution sets from the two types of laws are logically distinguishable, such that it is incor­rect to say that experimental laws are logically derived from theoretical postulates.  In Popper’s phraseology the derived theoretical laws (such as Newton’s) “correct” the experimental laws (such as Kepler’s) purporting to describe the same phenomena.

Scientific Criticism

          Carnap's philosophy of scientific criticism is his thesis of confirmation.  Both theoretical and empirical laws may be more or less confirmed, but empirical laws are con­firmed directly by observation or measurement, while theore­tical laws are confirmed indirectly through the confirmation of the empirical laws deductively derived from them.  Both empirical and theoretical laws may be classified as either universal or statistical laws.  Most of Carnap's discussion of this dis­tinction is in the context of empirical laws.  All empirical laws are statements expressing observed regularities as pre­cisely as possible.  If a certain regularity is observed at all times and in all places, then that regularity is expressed in the form of a universal law.  But if the law asserts that an event or the relation of one event to another occurs in only a certain percentage of cases, then the statement is called a statistical law.  Both types of laws occur in the object language of science, and both are empirical statements.  Statements about either universal and statistical laws occur in the meta-language, that refers to the object language of science in which the law and theory statements are expressed, and for either types the statements in the metalanguage may refer to the degree of confirmation of the laws.  Statements of the degree of confirmation are statements of logical probability associated with both universal and statistical laws.  Logical probability is an estimate of the long-term relative frequency stated by the statistical laws, and takes values between zero and one inclusively.  The statements associating the degree of confirmation to a statement in the object language are statements in the metalanguage.  The metalanguage is a language of the philosopher of science, and philosophy is not in Carnap's view an empirical or fact­ual science.  Philosophy of science is the logic of science, and the statements in the metalanguage are L-true or analy­tic.  Logical probability is the logical relation similar to logical implication.  By a logical analysis of a stated hypothesis h and the stated evidence e, one may conclude that h is not deductively implied but is partially implied by e to the degree r.  For any pair of sentences e and h inductive logic assigns a number giving the logical proba­bility of h with respect to e.  In this way Carnap views the metalanguage to consist of analytic statements as opposed to the synthetic statements in the object language consisting of laws of nature.

Scientific Discovery

          Carnap's philosophy of scientific discovery gives dif­ferent accounts for the discovery of empirical laws and the discovery of theoretical laws.  His philosophy of discovery of empirical laws is inductivist; induction is the measure­ment of the degree of regularity in observed instances known either passively by casual observation or actively by exper­imentation.  His philosophy of discovery of theoretical laws recognizes the role of the creative imagination.  He gives consideration to the use of computers.  He expresses doubts that rules can be established to enable a scientist to sur­vey millions of sentences giving various observational reports, and then by a mechanized procedure applying these rules to generate a general theory consisting of a system of theoretical laws that would explain the observed phenomena.  This is because theories deal with unobservables and use a conceptual framework that goes far beyond the framework used for the description of observations.  Creative ingenuity is needed to create theories.  Therefore Carnap concludes that there cannot be an inductive machine, a computer system into which the scientist can input all the relevant observation sentences, and then get an output consisting of a system of laws that explain the observed phenomena.  He only believed that given observation e and hypothesis h, there could be an inductive machine which will mechanically determine the logical probability or degree of confirmation of h on the basis of e.  It may be noted in this connection that the post-Positivist philosophers of science rejected the Positivist's strong distinction between theory and observation.  Like Einstein and Heisenberg, they maintained that theory determines what is observed.  Therefore, they maintain that there exists no theory-independent framework for observation.