Supplemental Material by Chapter

Any other criterion that is used for solving problem (4.2) or (4.19) must yield a solution with a lower entropy value—a more informed one where the number of minimal yes/no questions must be lower. (See later chapters and the Classical Dual ME Example2.gms and Classical ME Dice Example.gms in the "Codes and Examples" section for examples of other criteria and different measures of comparing the resulting distributions with the uninformed uniform one.)

For a visual representation of the entropy function, check entropy function.py in the "Codes and Examples" section.

One to one correspondence between Hessians of the primal and of the dual models: "This corresponde is evident when the numerical solutions are coded. But information theory and relative entropy offer a more intuitive perspective."

Please see the code centropy hessian.gms, dual_classical.py and Primal Taylor.gms for an exploration of these concepts.

For more information on the unique relationship between the Covariances of the primal and dual, see, for example, Appendix C of Chapter 3 in Golan, Judge and Miller (1996).

The exact formulation of the Hessian and the covariance for all formulations discussed are included in centropy hessian.gms and Primal Taylor.gms

Exercise 16 (Computer Practice II: Maximizing Entropy Versus Maximizing Other Objective Functions): Use any one of the codes referred to in Chapter 4 and use (or add) any criterion (objective function) you like.

Exercise 17: different examples and graphical analyses of comparing the maximum entropy method with other information-theoretic methods that use the same constraints but different entropies as objective functions can be found in Classical ME Dice Example.gms in the codes and examples section. This file generates graphs in Excel.

The codes for generating the data and solving the firm size problems are available in firm size distribution and firm size distribution (GAMS) in the "Codes and Examples" section. Similar exploration of income distribution is found in income distribution.

A simpler version of the code (for aggregating networks with a small number of nodes) is code is under revision; to be added

Exercise 6 (Multidimensional Size Distribution): see firm size distribution and firm size distributiin (GAMS)in the codes and examples section for further exploration of the multidimensional firm size example. Please see income distribution for exploration of a similar topic.

This code is for transformation of the original data (daily minimum and maximum temperature) into n and k as is shown in Appendix 6A. As an example it presents the transformation of the NYC, March 2013 data. The results are also shown in the table. The LA example is available in LA example of the "Codes and Examples" section of the website.

The estimated T and w in this case are -11.11 and 0.29 respectively for 1931, and -142.86 and 5.29 respectively for 2013. The signs of the w’s in this case are also consistent with our earlier argument (Section 2.4) since the integer n is both negative and positive.

The code for generating these data and that for other related experiments, as well as detailed solution and diagnostics, is available in Appendix 6B. An example of step 1 in Mathematica is below:

            nobs=100  (* number of individuals *)
            (* Symptom 1: hyperactivity *)
            X1=RandomVariate[NormalDistribution[0,2],nobs];
            (* Symptom 2: dislexia *)
            u2=RandomVariate[UniformDistribution[{0,1}],nobs];
            X2=Map[If[#>0.8,1,0]&,u2];
            (* Symptom 3: attention deficit *)
            X3=Map[If[#>1,1,0]&,X1];
            (* Environmental Impact: single parent household *)
            u4=RandomVariate[UniformDistribution[{0,1}],nobs];
            X4=Map[If[#>0.8,1,0]&,u4]
            (* Other: age in years *)
            X5=RandomVariate[UniformDistribution[{3,18}],nobs];
                            

Marginal effects -- to be added

Though this is not a precise number, the lack of precision does not bias the inferred probabilities (in a significant way) in this case, as long as that value is in the neighborhood of the correct magnitude. (See exercises, and chapter 6 examples in the codes and examples section.)

For the direct relationships among these covariances, or for the transformation between the covariance in parameter space to the probability space, see, for example, appendix 3C in Golan, Judge and Miller (1996).

Please see the code centropy hessian.gms, dual_classical.py and Primal Taylor.gms for an exploration of these concepts.

Exercise 3 (Relative Entropy and Maximum Entropy—Computer Practice): Codes for Cross Entropy are available in all commonly used software packages (Stata, SAS < Matlab, GAMS, Python). Please see the codes and examples section. Codes of interest include Dual-Info.gms, primal classical.py, dual_classical.py, centropy hessian.gms, Classical_ME_Example1 gams code, Classical Dual ME Example2.gms, Classical ME Dice Example.gms

Exercise 5 (Grouping: Size Distribution): User needs to modify the code in Firm Size Distribution to add priors (documented in the code).

See the codes related to matrix balancing ( Simple Matrix Balancing.gms, Matrix1.gms, Matrix2.gms, Matrix3.gms, matrix balancing.py.)

See the smoking example (Smoker stata.do, Smoker limdep.lim, SAS smoker.sas) as an example of the discrete choice problem.

Data related to the example is available in the "Data for Examples" section. Please refer to the binomial (and multinomial) choice examples in the "Codes and Examples" section (specifically, Smoker stata.do, Smoker limdep.lim, SAS smoker.sas, gmentropylogit.ado).

Please refer to the "linear" examples in the "Codes and Examples" section (specifically Linear auto.gms, Linear random.gms, GME random.gms, GME auto example.do and gmentropylinear.ado).

For option pricing, please refer to Option.zip. For Improved election prediction, refer to Election.zip

Predicting Coronary Artery—The analysis can be performed in Stata, SAS or GAMS. Users should remember to transform the original data as described in the text. Also refer to heart.zip

Please refer to liver.zip