Download as .pdf or .zip

 

All Books are in Adobe Acrobat PDF Format.
You may download a free plug-in here.

Download Winzip for
[Windows] or [Mac]
 
BOOK VIII - Page 10
 
  HERBERT SIMON, PAUL THAGARD AND OTHERS ON
DISCOVERY SYSTEMS
 
 

 

          In the Bayesian vector autoregression or "BVAR" model, there is a prior matrix, that is included in the formula for the ordinary least squares estimation of the coefficients of the model, and the parameters which are the elements in this prior matrix thereby influence the values of the estimated coefficients.  This prior matrix is a substitute for the a priori imposition of economic theory in the conventional structural-equation econometric model as described by Haavelmo, and it also has the desired effect of restricting the number of variables in the model.  Litterman argues that in the structural models there is rarely an attempt to justify the absence of variables on the basis of economic theory, des­pite the fact that a zero restriction on the excluded vari­able implies the existence of very certain prior informa­tion.  He says that the use of such exclusionary restric­tions does not allow a realistic specification of a priori knowledge.  His Bayesian specification, on the other hand, includes all variables in the system at several time lags, but it also includes the prior matrix indicating uncertainty about the structure of the economy.  Like Sargent, Litterman is critical of the adequacy of conventional macroeconomic theory, and he maintains that economists are more likely to find the regularities needed for better forecasts in the data rather than in some a priori economic theory.
          The difficult part of constructing BVAR models is cons­tructing a realistic prior matrix, and Litterman describes his procedure in his Specifying Vector Autoregression for Macroeconomic Forecasting, a Federal Reserve Bank of Minnea­polis Staff Report published in 1984.  His prior matrix, which he calls the "Minnesota prior", suggests with varying degrees of uncertainty, that all the coefficients in the model except those for the dependent variables' first lagged values are close to zero.  The varying degrees of uncer­tainty are indicated by the standard deviations calculated from benchmark out-of-sample retrodictive forecasts made with simple univariate models, and the variation in the degree of uncertainty is assumed to decrease as the length of the time-lags increases.  The parameters in the prior matrix are calculated from these standard deviations and from "hyperparameter" factors that vary along a continuum that indicates how likely the coefficients on the lagged values of the variables deviate from a prior mean of zero.  One extreme of this continuum is the univariate autoregres­sive model, and the opposite is the multivariate unrestricted VAR containing all the variables in the model in each equation.  By varying such hyperparameters and by making out-of-sample retrodictive forecasts, it is possible to map different prior distributions to a measure of forecasting accuracy according to how much multivariate interaction is allowed.  The measure of accuracy that Litterman uses is the determinant of the logarithms of the out-of-sample retrodic­tive forecast errors for the whole BVAR model.  Forecast errors measured in this manner are minimized in a search along the dimension between univariate and unrestricted VAR models.  Litterman calls this procedure a "prior search", and it is unlike anything described by Simon in his theory of heuristic search.  The procedure has been made commerci­ally available in a computer system called by a memorable acronym, "RATS", which is marketed by VAR Econometrics Inc., Minneapolis, MN.  This system also contains the ability to make the shock simulations of the type that Sims proposed for economic interpretation of the BVAR models.
          Economists typically do not consider the VAR or BVAR models to be econ­omic theories or "theoretical models.”  The concept of the­ory in economics, such as may be found in Haavelmo' paper, originates in the Romanticist philosophy of science, according to which the language of theory must describe the decision-making process in the economic actors' attempts to max­imize utility or profits.  In other words the semantics of the theory must describe the mental deliberations of the econ­omic actors whose behavior the theory explains, and this amounts to the a priori requirement for a mentalistic ont­ology.  The opposing view is that of the Positivists, or more specifically the Behaviorists, who reject all theory in this sense.  Both views are similar in that they have the same concept of theory.  The contemporary Pragmatist on the other hand reject all a priori ontological criteria for scientific cri­ticism, whether mentalistic or antimentalistic, even when these criteria are built into such meta­linguistic terms as "theory" and "observation.”  Contemporary Pragmatists instead define theory language on the basis of its use or function in scientific research, and not on the basis of its semantics or ontology: on the Pragmatist view theory language is that which is proposed for testing.  Theory is dis­tinguished by the hypothetical attitude of the scientist toward a proposed solution to a problem.  Therefore, on the contemporary Pragmatist philosophy of science, Litterman's system is a discovery system, because it produces economic theories.
          Ironically the rejection of the structural-equation type of econometric model by rational expectations advocates is a de facto implementation of the contemporary Pragmatist philosophy of science.  Sargent described rational expecta­tions with its greater fidelity to the maximizing postulates as a "counterrevolution" against the ad hoc aspects of the Keynesian revolution.  But from the point of view of the prevailing Romanticist philosophy of science practiced in economics, their accomplishment in creating the VAR-type of model is a radical revolution in the philosophy and method­ology of economics, because there is no connection between the rational expectations thesis and the VAR-type of model.  Rational expectations play no role in the specification of the VAR-type of model.  Empirical tests of the model could not test the rational expectations "hypothesis" even if it were an empirical hypothesis.  And their exclusion of empirical expectations measurement data justifies denying that the model even describes any mental expectations experienced by the economic actors.  The rational expectations hypothesis associated with the VAR models is merely a decorous discourse, a Romantic fig leaf giving the naked Pragmatism of the VAR models a dubious decency.
          The criterion for scientific criticism that is actually operative in the VAR-type of model is perfectly empirical; it is the forecasting performance.  And it is to this cri­terion that Litterman appeals.  In Forecasting with Bayesian Vector Autoregressions: Four Years of Experience he des­cribes the performance of a monthly national economic BVAR model constructed for the Federal Reserve Bank of Minnea­polis and operated over the period 1981 through 1984.  He reports that this BVAR model demonstrated superior perform­ance in forecasting the unemployment rate and the real GNP during the 1982 recession, which was the worst recession since the Great Depression of the 1930's.  The BVAR model made more accurate forecasts than the three leading struc­tural models: Data Resources, Chase Econometrics, and Wharton Associates.  However, he also reports that the BVAR model did not make a superior forecast of the inflation rate measures by the percent change in the GNP deflator. Thereafter Litterman continued to publish forecasts from the BVAR model in the Federal Reserve Bank of Minnea­polis Quarterly Review.  In the Fall, 1984, issue he fore­casted that the 1984 slowdown was a short pause in the post-1982 recession, and that the national economy would exhibit above-average growth rates in 1985 and 1986.  A year later in the Fall, 1985, issue he noted that his BVAR model forecast for 1985 was overshooting the actual growth rates for 1985, but he also states that his model was more accur­ate than the structural-equation models.  In the Winter, 1987, issue two of his sympathetic colleagues on the Federal Reserve Bank of Minneapolis research staff, William Roberds and Richard Todd, published a critique reporting that the BVAR model forecasts of the real GNP and the unemployment rate were overshooting measurements of actual events, and furthermore that competing structural models had performed better for 1986.  The Federal Reserve Bank of Minneapolis continues to publish forecasts from the BVAR model in its Quarterly Review.  Reports in the Minneapolis Bank's Quarterly Review also contain descriptions of how the BVAR national economic model is revised as part of its continuing development.  In the Fall, 1984, issue the model is described as having altoget­her forty-six descriptive variables and equations, but it has a "core" sector of only eight variables and equations, which receives no feedback from the remainder of the model.  This core sector must make accurate forecasts, in order for the rest of the model to function accurately.  When the BVAR model is revised, changes are made to the selection of variables in this core sector.  Reliance on this small number of variables is the principal weakness of this type of model.  It is not a vulnerability that is intrinsic to the VAR-type of model, but rather is a concession to computational limits of the computer, because construction of the Bayesian prior matrix makes great demands on the computer.  In contrast the structural models typically contain hundreds of different descriptive variables interacting either simultaneously or recursively.  Eventually improved computer hardware design will enable the BVAR models to be larger, but in the meanwhile they must perform heroic feats with very small amounts of descriptive information as they compete with the much larger structural-equation models containing much greater amounts of information.
          Unlike Simon's simulations of historically significant scientific discoveries, Litterman cannot separate the merit of his computerized procedures for constructing his BVAR models form the scientific merit of the BVAR models he makes with his discovery system.  Litterman is not recreating what Russell Hanson called "almanac science", but is operating at the frontier of "research science.” Furthermore, the approach of Litterman and colleagues is much more radical than that of the conventional economist, who needs only to propose a new "theory", and then apply conventional structural-equation econometric modeling techniques.  The BVAR technique has been made commercially available for microcomputer use, but still the econometrician constructing the BVAR model must learn statistical techniques that he had not been taught in his professional education.  Many economists fail to recognize the Pragmatic character of the BVAR models, and reject the technique out of hand, since they reject the rational expectations hypothesis.  Nonetheless several economists working in regional economics have been experimenting with BVAR modeling of state economies.  As of this writing such models are still used by the District Federal Reserve banks of Dallas (Gruben and Donald, 1991), Cleveland (Hoehn and Balazsy, 1985), and Richmond (Kuprianov and Lupoletti, 1984), and by the University of Connecticut (Dua and Ray, 1995).  Only time will tell whether or not this new type of modeling survives much less achieves ascendancy in the economics profession.

Hickey's Metascience or "Logical Pragmatism"

          Thomas J. Hickey received a master's degree in economics from the University of Notre Dame in South Bend, Indiana, where he also studied philosophy in their Ph.D. program.  He found the economics faculty supportive, but he found the philosophy faculty obstructionist due to his prejudicial scientific realism. Since Descartes attempted to prove the existence of the real world, only a few pedantics have thought that realism is a conclusion.  But the philosophy department chairman, a Reverend Ernan McMullin, told Hickey that were Hickey to persist in his views, he could never expect to succeed under their philosophy faculty, that he had a “bad attitude”, and that if he preferred to leave he might do so.  This threat was real, since the department chairman selects faculty for students’ examination and dissertation boards. Hickey had no interest in a metaphysical criticism of science and even less interest in the reformist demand, which he viewed as contemptuous moral violence.  The Reverend chairman also initiated an unprovoked denial that he wanted “to play God.”  But as the Reverend spoke, Hickey vividly recalled witnessing the Reverend’s classroom behavior - shouting down students who expressed alternative ideas, and imputing views to students they had not expressed and then loudly attacking the imputed views.  Hickey’s response to the reformist ultimatum was to drop his enrollment and write a patient farewell letter to the Reverend chairman, in which he stated that he would proceed independently to create his “dynamic model describing the development of science in a manner capable of facilitating the advancement of science”, and that he would pursue his methodological interest as a new discipline, which he then called “empirical metascience.”  At this writing Hickey notes the obstructionist philosophy faculty he knew is still listed at the school’s Internet web site. Furthermore he doubts even a faculty shakeout could remedy their shrill subculture. 
          Before leaving Notre Dame, Hickey had recognized that he needed to acquire computer-programming skills to implement his metascience agenda.  Consequently after leaving he enrolled at San Jose College in San Jose, California, where he took coursework in numerical-analysis computer programming in the FORTRAN computer programming language.  There he developed his METAMODEL computer discovery system, the “dynamic model” he had described in his Notre Dame farewell letter.  He then published a description of his contemporary Pragmatist philosophy of science, his computational metatheory, and his METAMODEL discovery system in a brief seventy-five page monograph titled Introduction to Metascience: An Information Science Approach to Methodology of Scientific Research.  This Metascience monograph was originally intended to be the thesis of his Ph.D. dissertation in philosophy of science at Notre Dame, and he has yet to find any previously published computer discovery system contributed to academic philosophy of science.  Since publishing his Metascience monograph Hickey has also referred to metascience as "Logical Pragmatism", where the "Pragmatism" is the contemporary Pragmatist philosophy of science, and where the "Logic" is emphatically not the irrelevant Russellian “symbolic” logic, but instead consists of logics developed with computer languages and actually used in the empirical sciences.  Hickey intends that his term “Logical Pragmatism” should not be taken as a proper name specifically for his philosophical views or systems, but rather should be taken in a generic sense, which includes alternative system designs and strategies, and admits to variations on the basic themes of the contemporary Pragmatist philosophy, but which essentially includes a mechanized procedural approach.  More recently the phrase “computational philosophy of science” has also come into use, which also includes the alternative psychologistic approach, and thus is not specific to the contemporary Pragmatist philosophy and is an even more generic label than “Logical Pragmatism.”  This section reports Hickey's current statement of his metatheory, the "Pragmatist" part of his Logical Pragmatist philosophy, and the next section describes his METAMODEL discovery system, his own contribution to the computational or "Logical" part of Logical Pragmatism.

Hickey’s Linguistic Analysis  

          Hickey’s metatheory may be summarized in terms of the four basic topics considered in philosophy of science: the aim of science, discovery, criticism, and explanation.  But some preliminary comments are in order, to provide the integrating context supplied by the contemporary Pragmatist philosophy of language. Hickey contrasts his Logical Pragmatism to the alternative psychologistic approach, which descends from Simon and is found in the more recent efforts of Thagard. In this respect Hickey’s linguistic constructionalism with computer systems locates him in the traditional orientation in twentieth-century philosophy of science.  The contemporary Pragmatist philosophy of science has its origins in the “analytical” tradition, which began with the historic “linguistic turn” in twentieth-century philosophy, and which in the United States has since evolved into the contemporary Pragmatist philosophy of language.  Hickey prefers a linguistic-analysis approach to the psychologistic approach for three reasons:
          Firstly he believes that the psychologistic approach reveals a failure to appreciate the new Pragmatist philosophy of language, and he notes that advocates of the psychologistic approach typically include some residual Positivist ideas.  He recognizes that the discovery systems must remain open to the fact that the empirical underdetermination of language limits the decidability of scientific criticism by all available evidence at any given time.  His experience using his METAMODEL system has routinely revealed many alternative system-outputs that are equally acceptable empirically.  Therefore he believes that the psychological approach may supply additional determination based on behavioral considerations that may operate within this range of empirical undecidability.  But he has found that in research practice the resolution of this range of empirical undecidability is better described as more a matter of research strategy than behavioral psychology.  He therefore maintains that psychological determinants are historically incidental, and that they are actually retarding if they operate outside the limits of the empirical constraint.  He maintains that psychological and sociological considerations should not be included in the aim of science as criteria for criticism and that they should be viewed in philosophy of science as purely circumstantial constraints, perhaps historical idiosyncrasies at best incidental to science and often obstructionist to scientific progress.
          Secondly Hickey’s metascience agenda with its Pragmatist statement of the aim of science and its linguistic constructionalism with computer systems makes no claims about representing human psychological processes, including representing the computation constraint that Simon found in intuitive human problem solving activity in the processes of theory development and discovery.  Hickey treats psychological claims to date as he treats the claims made by metaphysicians and philosophers of mind: he dismisses them as speculative baggage lacking independent evidence and as gratuitously attributed to a functioning language-processing mechanized discovery system.  In this regard he views the computational discovery system with the same practicality that the industrial engineer views the computerized robot on the assembly line of an automobile factory, because the engineer’s purpose is not to replicate the practices of the human factory worker much less the worker’s human limitations, but rather to achieve an improved productivity that justifies the financial investment in the robotic equipment and system.  And such mechanized improvement may imply redesigning the relevant factory assembly-line practices initially designed for the human worker.  Similarly the computational philosopher of science may - and probably will - effectively redesign the intuitive theory development practices of the human scientist, in order to exploit the productivity of mechanization and thereby to improve the discovery process.  The computational philosopher of science need not understand the intuitive human discovery process, in order to produce a design yielding manifestly superior outcomes.  He need only understand the characteristics of a good theory and develop a procedure whereby such theories can be produced mechanically and then observe that there are improved results produced with his mechanization implementation.
          Thirdly Hickey believes that the psychological conceptualization of the discovery systems overlooks the sociocultural and historical character of scientific development.  And by this he does not mean the sociological mechanisms of socialization and social control in the scientific community, which can be (and very often have been) retarding to the advancement of some sciences.  He notes that the systems made by the computational psychologists do not actually operate independently of the historical and cultural environment; they too depend on the cultural environment for inputs that are specific to a historical time and state of science, including notably the definition of the scientific problem under consideration.  But the cognitive psychologists conceptualize their systems with little or no regard for the history and culture of the language-using community of the scientific professions.  In sum: rather than view the artificial-intelligence discovery system as psychological investigation, Hickey views it as a language-processing constructional system operating under the regulation of the contemporary rationality postulate set forth in the Pragmatist statement of the institutionalized aim of modern science.  
          Therefore, working in the linguistic-analysis tradition Hickey selects as his point of departure some of Carnap’s views, and modifies them in certain fundamental ways for a Pragmatist computational philosophy of science.  Like Carnap, he distinguishes object language and metalanguage, and he views his METAMODEL discovery system as an object-language-processing system written in a metalanguage which includes notably the computer langauge used for the computer discovery system.  Object language consists of the statements used by scientists for articulating their experimental designs and theories, which describe the extralinguistic real world.  This language includes both colloquial discourse and the written symbols for which mathematics supplies the grammatical syntax.  The metalanguage is used to describe the object language including the computerized systems procedures used to change the object language.  Terms such as "explanation", "falsification", "theory", "test design", “state description”, and "discovery system" are examples of metalinguistic vocabulary.  The Logical Pragmatist philosopher of science, who may also be called a metascientist, uses the metalanguage in the process of formulating his theory, and it is therefore called a "metatheory.”  The computer language used in a discovery system is part of the metalanguage, and the system itself is part of the metatheory.

 

Pages [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [Next 10 >>]
NOTE: Pages do not corresponds with the actual pages from the book

 

Web Design by Global Nexchange Solutions