Supplemental Material by Chapter

The links (by chapter) in this page are for codes, examples, or further derivations and extensions that were referred to in the text.

Chapter Page Description
Chapter 4 p. 73

Any other criterion that is used for solving problem (4.2) or (4.19) must yield a solution with a lower entropy value—a more informed one where the number of minimal yes/no questions must be lower. (See later chapters and the Classical Dual ME Example2.gms and Classical ME Dice Example.gms in the "Codes and Examples" section for examples of other criteria and different measures of comparing the resulting distributions with the uninformed uniform one.)

For a visual representation of the entropy function, check entropy function.py in the "Codes and Examples" section.

Chapter 4 p. 90

One to one correspondence between Hessians of the primal and of the dual models: "This corresponde is evident when the numerical solutions are coded. But information theory and relative entropy offer a more intuitive perspective."

Please see the code centropy hessian.gms, dual_classical.py and Primal Taylor.gms for an exploration of these concepts.

For more information on the unique relationship between the Covariances of the primal and dual, see, for example, Appendix C of Chapter 3 in Golan, Judge and Miller (1996).

Chapter 4 p. 93

The exact formulation of the Hessian and the covariance for all formulations discussed are included in centropy hessian.gms and Primal Taylor.gms

Chapter 4 p. 104

Exercise 16 (Computer Practice II: Maximizing Entropy Versus Maximizing Other Objective Functions): Use any one of the codes referred to in Chapter 4 and use (or add) any criterion (objective function) you like.

Chapter 4 p. 104

Exercise 17: different examples and graphical analyses of comparing the maximum entropy method with other information-theoretic methods that use the same constraints but different entropies as objective functions can be found in Classical ME Dice Example.gms in the codes and examples section. This file generates graphs in Excel.

Chapter 5 p. 116

The codes for generating the data and solving the firm size problems are available in firm size distribution and firm size distribution (GAMS) in the "Codes and Examples" section. Similar exploration of income distribution is found in income distribution.

Chapter 5 p. 131

A simpler version of the code (for aggregating networks with a small number of nodes) is code is under revision; to be added

Chapter 5 p. 132

Exercise 6 (Multidimensional Size Distribution): see firm size distribution and firm size distributiin (GAMS)in the codes and examples section for further exploration of the multidimensional firm size example. Please see income distribution for exploration of a similar topic.

Chapter 6 p. 140
p. 142

This code is for transformation of the original data (daily minimum and maximum temperature) into n and k as is shown in Appendix 6A. As an example it presents the transformation of the NYC, March 2013 data. The results are also shown in the table. The LA example is available in LA example of the "Codes and Examples" section of the website.

Chapter 6 p. 143

The estimated T and w in this case are -11.11 and 0.29 respectively for 1931, and -142.86 and 5.29 respectively for 2013. The signs of the w’s in this case are also consistent with our earlier argument (Section 2.4) since the integer n is both negative and positive.

Chapter 6 p. 145
p. 159

The code for generating these data and that for other related experiments, as well as detailed solution and diagnostics, is available in Appendix 6B. An example of step 1 in Mathematica is below:

            nobs=100  (* number of individuals *)
            (* Symptom 1: hyperactivity *)
            X1=RandomVariate[NormalDistribution[0,2],nobs];
            (* Symptom 2: dislexia *)
            u2=RandomVariate[UniformDistribution[{0,1}],nobs];
            X2=Map[If[#>0.8,1,0]&,u2];
            (* Symptom 3: attention deficit *)
            X3=Map[If[#>1,1,0]&,X1];
            (* Environmental Impact: single parent household *)
            u4=RandomVariate[UniformDistribution[{0,1}],nobs];
            X4=Map[If[#>0.8,1,0]&,u4]
            (* Other: age in years *)
            X5=RandomVariate[UniformDistribution[{3,18}],nobs];
                            
Chapter 6 p. 146

Marginal effects -- to be added

Chapter 6 p. 155

Though this is not a precise number, the lack of precision does not bias the inferred probabilities (in a significant way) in this case, as long as that value is in the neighborhood of the correct magnitude. (See exercises, and chapter 6 examples in the codes and examples section.)

Chapter 7 p. 171 (Box 7.1)
p. 188

For the direct relationships among these covariances, or for the transformation between the covariance in parameter space to the probability space, see, for example, appendix 3C in Golan, Judge and Miller (1996).

Please see the code centropy hessian.gms, dual_classical.py and Primal Taylor.gms for an exploration of these concepts.

Chapter 8 p. 226

Exercise 3 (Relative Entropy and Maximum Entropy—Computer Practice): Codes for Cross Entropy are available in all commonly used software packages (Stata, SAS < Matlab, GAMS, Python). Please see the codes and examples section. Codes of interest include Dual-Info.gms, primal classical.py, dual_classical.py, centropy hessian.gms, Classical_ME_Example1 gams code, Classical Dual ME Example2.gms, Classical ME Dice Example.gms

Chapter 8 p. 227

Exercise 5 (Grouping: Size Distribution): User needs to modify the code in Firm Size Distribution to add priors (documented in the code).

Chapter 11 p. 324 (Box 11.2)

See the codes related to matrix balancing ( Simple Matrix Balancing.gms, Matrix1.gms, Matrix2.gms, Matrix3.gms, matrix balancing.py.)

Chapter 12  

See the smoking example (Smoker stata.do, Smoker limdep.lim, SAS smoker.sas) as an example of the discrete choice problem.

Chapter 12 pp. 345-346

Data related to the example is available in the "Data for Examples" section. Please refer to the binomial (and multinomial) choice examples in the "Codes and Examples" section (specifically, Smoker stata.do, Smoker limdep.lim, SAS smoker.sas, gmentropylogit.ado).

Chapter 13 p. 401

Please refer to the "linear" examples in the "Codes and Examples" section (specifically Linear auto.gms, Linear random.gms, GME random.gms, GME auto example.do and gmentropylinear.ado).

Chapter 14 p. 412

For option pricing, please refer to Option.zip. For Improved election prediction, refer to Election.zip

Chapter 14 p. 423 (Box 11.2)
p. 424

Predicting Coronary Artery—The analysis can be performed in Stata, SAS or GAMS. Users should remember to transform the original data as described in the text. Also refer to heart.zip

Chapter 14 p. 434
p. 437

Please refer to liver.zip