I talk here about how a mathematician mindset can be useful for AI alignment. But first, a puzzle:

Given $m$ , what is the least number $n \geq 2$ such that for $2 \leq k \leq m$ , the base $k$ representation of $n$ consists entirely of 0s and 1s?

If you want to think about it yourself, stop reading.

For $m$ =2, $n$ =2.

For $m$ =3, $n$ =3.

For $m$ =4, $n$ =4.

For $m$ =5, $n$ =82,000.

Indeed, 82,000 is 10100000001010000 in binary, 11011111001 in ternary, 110001100 in base 4, and 10111000 in base 5.

What about when $m$ =6?

So, a mathematician might tell you that this is an open problem. It is not known if there is any $n \geq 2$ which consists of 0s and 1s in bases 2 through 6.

A scientist, on the other hand, might just tell you that clearly no such number exists. There are $2^{k - 1}$ numbers that consist of $k$ 0s and 1s in base 6. Each of these has roughly ${log}_{5} (6) \cdot k$ digits in base 5, and assuming things are roughly evenly distributed, each of these digits is a 0 or a 1 with "probability" $2 / 5$ . The "probability" that there is any number of length $k$ that has the property is thus less than $2^{k} \cdot (2 / 5)^{k} = (4 / 5)^{k}$ . This means that as you increase $k$ , the "probability" that you find a number with the property drops off exponentially, and this is not even considering bases 3 and 4. Also, we have checked all numbers up to 2000 digits. No number with this property exists.

Who is right?

Well, they are both right. If you want to have fun playing games with proofs, you can consider it an open problem and try to prove it. If you want to get the right answer, just listen to the scientist. If you have to choose between destroying the world with a 1% probability and destroying the world if a number greater than 2 which consists of 0s and 1s in bases 2 through 6 exists, go with the latter.

It is tempting to say that we might be in a situation similar to this. We need to figure out how to make safe AI, and we maybe don't have that much time. Maybe we need to run experiments, and figure out what is true about what we should do and not waste our time with math. Then why are the folks at MIRI doing all this pure math stuff, and why does CHAI talk about "proofs" of desired AI properties? It would seem that if the end of the world is at stake, we need scientists, not mathematicians.

I would agree with the above sentiment if we were averting an astroid, or a plague, or global warming, but I think it fails to apply to AI al... ,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,

Nominated Posts for the 2018 Review

2018 Review Discussion