Properties of the logarithm - AI Alignment Forum

With a solid interpretation of logarithms under our belt, we are now in a position to look at the basic properties of the logarithm and understand what they are saying. The defining characteristic of logarithm functions is that they are real_valued functions such that

Property 1: Multiplying inputs adds outputs. $\begin{matrix} f (x \cdot y) = f (x) + f (y) \\ (1) \end{matrix}$ for all $x, y \in$ $R^{+}$ . This says that whenever the input grows (or shrinks) by a factor of $y$ , the output goes up (or down) by only a fixed amount, which depends on $y$ . In fact, equation (1) alone tells us quite a bit about the behavior of $f,$ and from it, we can almost guarantee that $f$ is a logarithm function. First, let's see how far we can get using equation (1) all by itself:

Property 2: 1 is mapped to 0. $\begin{matrix} f (1) = 0. \\ (2) \end{matrix}$

This says that the amount the output changes if the input grows by a factor of 1 is zero — i.e., the output does not change if the input changes by a factor of 1. This is obvious, as "the input changed by a factor of 1" means "the input did not change."

Exercise: Prove (2) from (1).

Proof

$f (x) = f (x \cdot 1) = f (x) + f (1), so f (1) = 0.$

Property 3: Reciprocating the input negates the output. $\begin{matrix} f (x) = - f (\frac{1}{x}) . \\ (3) \end{matrix}$

This says that the way that growing the input by a factor of $x$ changes the output is exactly the opposite from the way that shrinking the input by a factor of $x$ changes the output. In terms of the "communication cost" interpretation, if doubling (or tripling, or $n$ -times-ing) the possibility space increases costs by $c$ , then halving (or thirding, or $n$ -parts-ing) the space decreases costs by $c .$

Exercise: Prove (3) from (2) and (1).

Proof

$x \cdot \frac{1}{x} = 1, so f (1) = f (x \cdot \frac{1}{x}) = f (x) + f (\frac{1}{x}) .$

$f (1) = 0, so f (x) and f (\frac{1}{x}) must be opposites.$

Property 4: Dividing inputs subtracts outputs.

$\begin{matrix} f (\frac{x}{y}) = f (x) - f (y) . \\ (4) \end{matrix}$

This follows immediately from (1) and (3).

Exercise: Give an interpretation of (4).

Interpretation

There are at least two good interpretations:

$f (x \cdot \frac{1}{y}) = f (x) - f (y),$ i.e., shrinking the input by a factor of $y$ is the opposite of growing the input by a factor of $y .$

$f (z \cdot \frac{x}{y}) = f (z) + f (x) - f (y),$ i.e., growing the input by a factor of $\frac{x}{y}$ affects the output just like growing the input by $x$ and then shrinking it by $y .$

Try translating these into the communication cost interpretation if it is not clear why they're true.

Property 5: Exponentiating the input multiplies the output.

$\begin{matrix} f (x^{n}) = n \cdot f (x) . \\ (5) \end{matrix}$

This says that multiplying the input by $x$ , $n$ times incurs $n$ identical changes to the output. In terms of the communication cost metaphor, this is saying that you can emulate an $x^{n}$ digit using $n$ different $x$ -digits.

Exercise: Prove (5).

Proof

This is easy to prove when $n \in N :$ $f (x^{n}) = f (x \cdot x \cdot \dots x      n times) = f (x) + f (x) + \dots f (x)      n times = n \cdot f (x) .$

For $n \in Q,$ this is a bit more difficult; we leave it as an exercise to the reader. Hint: Use the proof of (6) below, for $n \in N,$ to bootstrap up to the case where $n \in Q .$

For $n \in R,$ this is actually not provable from (1) alone; we need an additional assumption (such as continuity) on $f$ .

Property 5 is actually false, in full generality — it's possible to create a function $f$ that obeys (1), and obeys (5) for $n \in Q,$ but which exhibits pathological behavior on irrational numbers. For more on this, see pathological near-logarithms.

This is the first place that property (1) fails us: 5 is true for $n \in Q,$ but if we want to guarantee that it's true for $n \in R,$ we need $f$ to be continuous, i.e. we need to ensure that if $f$ follows 5 on the rationals it's not allowed to do anything insane on irrational numbers only.

Property 6: Rooting the input divides the output.

$\begin{matrix} f (\sqrt[n]{x}) = \frac{f (x)}{n} . \\ (6) \end{matrix}$

This says that, to change the output one $n$ th as much as you would if you multiplied the input by $x$ , multiply the input by the $n$ th root of $x$ . See Fractional digits for a physical interpretation of this fact.

Exercise: Prove (6).

Proof

$(\sqrt[n]{x})^{n} = x, so f ((\sqrt[n]{x})^{n}) has to equal f (x) .$

$f ((\sqrt[n]{x})^{n}) = n \cdot f (\sqrt[n]{x}), so f (\sqrt[n]{x}) = \frac{f (x)}{n} .$

As with (5), (6) is always true if $n \in Q,$ but not necessarily always true if $n \in R .$ To prove (6) in full generality, we additionally require that $f$ be continuous.

Property 7: The function is either trivial, or sends some input to 1.

$\begin{matrix} Either f sends all inputs to 0, or there exists a b \neq 1 such that f (b) = 1. \\ (7) \end{matrix}$

This says that either $f$ is very boring (and does nothing regardless of its inputs), or there is some particular factor $b$ such that when the input changes by a factor of $b$ , the output changes by exactly $1$ . In the communication cost interpretation, this says that if you're measuring communication costs, you've got to pick some unit (such as $b$ -digits) with which to measure.

Exercise: Prove (7).

Proof

Suppose $f$ does not send all inputs to $0$ , and let $x$ be an input that $f$ sends to some $y \neq 0.$ Then $f (\sqrt[y]{x}) = \frac{f (x)}{y} = 1.$ ^[1]

$b$ is $\sqrt[y]{x} .$ We know that $b \neq 1$ because $f (b) = 1$ whereas, by (2), $f (1) = 0$ .

Property 8: If the function is continuous, it is either trivial or a logarithm.

$\begin{matrix} If f (b) = 1 then f (b^{x}) = x . \\ (8) \end{matrix}$

This property follows immediately from (5). Thus, (8) is always true if $x$ is a rational, and if $f$ is continuous then it's also true when $x$ is irrational.

Property (8) states that if $f$ is non-trivial, then it inverts exponentials with base $b .$ In other words, $f$ counts the number of $b$ -factors in $x$ . In other words, $f$ counts how many times you need to multiply $1$ by $b$ to get $x$ . In other words, $f = {log}_{b}$ !

Many texts take (8) to be the defining characteristic of the logarithm. As we just demonstrated, one can also define logarithms by (1) as continuous non-trivial functions whose outputs grow by a constant (that depends on $y$ ) whenever their inputs grow by a factor of $y$ . All other properties of the logarithm follow from that.

If you want to remove the "continuous" qualifier, you're still fine as long as you stick to rational inputs. If you want to remove the "non-trivial" qualifier, you can interpret the function $z$ that sends everything to zero as ${log}_{\infty}$ . Allowing ${log}_{\infty}$ and restricting ourselves to rational inputs, every function $f$ that satisfies equation (1) is isomorphic to a logarithm function.

In other words, if you find a function whose output changes by a constant (that depends on $y$ ) whenever its input grows by a factor of $y$ , there is basically only one way it can behave. Furthermore, that function only has one degree of freedom — the choice of $b$ such that $f (b) = 1.$ As we will see next, even that degree of freedom is rather paltry: All logarithm functions behave in essentially the same way. As such, if we find any $f$ such that $f (x \cdot y) = f (x) + f (y)$ (or any physical process well-modeled by such an $f$ ), then we immediately know quite a bit about how $f$ behaves.

^{^︎}
You may be wondering, "what if $y$ is negative, or a fraction?" If so, see Strange roots. Short version: $\sqrt[- 3 / 4]{x}$ is perfectly well-defined.