A (logarithmic) leap of faith

I intended to relax tonight, but somehow, post-Starcraft, my mind ended up wandering back onto research. Funny how that works! Here's something random that I was mulling over: how do you know what form of constraint you should be using in a maximum entropy calculation? The standard variational principle for thermodynamics is maximum entropy: maximizing the entropy, S = ∑_i p_i ln p_i, subject to an average constraint on an experimental observable tells you what your probability distribution will look like at equilibrium. For example, a constraint on the system's average energy would be set as

⟨E⟩= ∑_i E_i p_i

where the angle brackets denote an average.

"Free energy" is defined as the sum of entropy and energy, or F = S +⟨E⟩, and to maximize the entropy subject to the constraint on the average energy, we set the total derivative dF = dS + β d⟨E⟩= 0, where β is the Lagrange multiplier for the constraint and d⟨E⟩= ∑_i E_i dp_i. The total derivative is

dF = ∑_i ( ln p_i + 1 + α + β E_i ) dp_i = 0

where α is the normalization multiplier, ∑_i p_i = 1, so that α ∑_i dp_i = 0. The resulting probability distribution is exponential:

p_i = e^-1-α e^{-β E_i} = Q^-1 e^{-β E_i}

where Q = ∑_i e^{-β E_i} is the partition function.

One assumption built into this calculation is that an average observed quantity implies a linear constraint on the underlying variable, E. For observed average energy, this is empirically true, but clearly for other quantities an experimental average would imply a nonlinear constraint on the variable: observed average momentum, P, for example, would constrain E = P² / 2m. This quadratic constraint results in a Gaussian probability distribution, rather than exponential:

p_i = Q^-1 e^{-β P_i2 / 2m}

This raises an interesting question: for an arbitrary observed quantity, how do we know, a priori, what the form of the constraint on the underlying variable should be? For a multiplicative process, a logarithmic constraint might be the most natural choice. (Ok, so that's not much of an argument. Well, it's late and let's face it, if you couldn't make half-baked arguments on your blog, there wouldn't be too many blogs around, would there?) In this case, for an observed value ⟨k⟩, the constraint would be

⟨k⟩= ∑_i ln (k_i) p_i

This results in a probability distribution which follows a power law,

p_i = Q^-1 k_i^-β

The challenge, then, is to decide what the most justifiable form is for the constraint, both in terms of curve fitting, as well as the underlying microscopic explanation. In the case of a logarithmic constraint, is there a solid argument, for example based on the principle that the constrained quantity must be extensive, that certain constraints should be logarithmic, rather than linear?