This is a special post for quick takes by Donald Hobson. Only they can create top-level comments. Comments here also appear on the Quick Takes page and All Posts page.
Incredibly crude alignment technique. Rot13 all text and training data, except for a small corpus of unusually good data.
What's the hoped-for benefit of this plan? I think this post would have been a lot better if you explained what the plan is supposed to do and why you think it can work.
Imagine you have a small corpus of data from good ethical people, and a large corpus of data that's full of 4chan trolls or reward hacking or some other bad thing. (The large corpus isn't just trolls, there is useful data in there too)
The hoped for benefit is that the AI displays all the useful knowledge that can be extracted from the large corpus, but only displays the behavior patterns found in the small corpus.
So suppose the 4chan data contains a few useful maths ideas, and also a lot of slurs. And suppose the good data consists entirely of maths proofs. Then the LLM can store proofs from both sources in the same common format. And "non-rot13 text is never a slur" is an easy to learn rule.
This links in with this post. https://www.lesswrong.com/posts/AfoGGrJfuNzofpzWL/models-may-behave-differently-in-graded-episodes-a-tirade Which says, amongst other things, that current models already mostly know whether they are in a graded episode or not, and this is good.
Another rough alignment idea.
The non-overlap principle. The basic idea, identify the convergent instrumentally useful cognition, and avoid it.
Suppose you ask the AI to cure cancer, and to find a room temperature superconductor. This principle says that there should be 0 overlap between the thought processes used in these 2 ideas. The cancer curing AI should avoid doing any thinking that would be useful in general. Figuring out how to hack it's way out and access the internet is useful in general. Sure, there are things that are generally useful that might be ok actually, like redesigning it's algorithms or building nanotech. But neither of these things are Safe.
A cancer curing AI that's restricted to only thinking about the details of cancer cells is a reasonable balance between safe and useful.
How similar is it to various proposals of narrow AI unable to write code, if not outright to attempts to train the AI not to write anything undesirable in its CoT? The latter option is one of the worst ideas with which one can come up.
How similar is it to various proposals of narrow AI unable to write code, if not outright to attempts to train the AI not to write anything undesirable in its CoT?
"Don't write code" is a blacklist, and blacklists have the major downside of permitting anything you didn't think of. It's hard to define exactly what you mean by "code", is DNA a kind of code? Is pseudocode instructions that are manually executed by a human "code"?
And training on COT is probably a bad idea. This sort of stuff only works if you can make the AI stop thinking about a topic, not just hide it's thoughts.
I mean, narrow AIs which have no way to, say, hack their way from the lab because they never knew how to write code
I mean an AI that doesn't write code is safer than one that does. But if you set an AI to cure cancer, you probably don't want it thinking about say Volcanology in too much detail. If the AI doesn't find anything, thinking was a waste of compute. And if the AI does find a means of "curing cancer" that involves the details of volcanoes, I expect it won't be a solution I like. You want to make the AI focus on just biochemistry and nothing else.
Alignment crazy idea. Only run optimization power through channels that have been optimized to convey it.
Like water that flows through pipes, but doesn't escape from leaks.
Suppose the AI is connected to a robot body. The AI can optimize along the wires, and through the motors. Optimization power can flow along these channels because humans deliberately optimized them to be good at conveying optimization power. But the AI can't use row-hammer. Humans didn't deliberately optimize memory modules to be susceptible. They just happen to be because of physics. Thus the electric interference between memory locations is a channel that optimization power can flow through, but it was not itself optimized to be good at transmitting optimization power. Thus the AI isn't allowed to use it.
I am trying to write something that would make sense if I had as solid and mathy idea of "optimization here" as I do with "information here".
Viruses are optimizing their own spread, not killing all humans. This seems to be further optimizing an already highly optimized artifact, not flowing optimization through an optimized channel.
I am not sure, I think it depends on why the AI wants the shockwave. Again, all I have is a fuzzy intuition that says yes in some cases, no in others, and shrugs in a lot of cases. I am trying to figure out if I can get this into formal maths. And if I succeed, I will (probably, unless infohazard or something) describe the formal maths.
Viruses are optimizing their own spread, not killing all humans. This seems to be further optimizing an > already highly optimized artifact, not flowing optimization through an optimized channel.
Well I'm saying that the virus's ability to penetrate the organism, penetrate cells and nuclei, and hijack the DNA transcription machinery, is a channel. It already exists and was optimized to transmit optimization power: selection on the viral genome is optimization, and it passes through this channel, in that this channel allows the viral genome (when outside of another organism) to modify the behavior of an organism's cells.
(For the record I didn't downvote your original post and don't know why anyone would.)
Yeah, probably. However, note that it can only use this channel if a human has deliberately made an optimization channel that connects in to this process. Ie the AI isn't allowed to invent DNA printers itself.
I think a bigger flaw is where one human decided to make a channel from A to B, another human made a channel from B to C ... until in total there is a channel from A to Z that no human wants and no human knows exists, built entirely out of parts that humans build.
Ie person 1 decides the AI should be able to access the internet. Person 2 decided that anyone on the internet should be able to run arbitrary code on their programming website, and the AI puts those together, even when no human did. Is that a failure of this design? Not sure. Can't get a clear picture until I have actual maths.
rough stop button problem ideas.
You want an AI that believes its actions can't effect the button. You could use causal counterfactuals. An imaginary button that presses itself at random. You can scale the likelihood of worlds up and down, to ensure the button is equally likely to be pressed in each world. (Wierd behaviour, not recomended) You can put the AI in the logical counterfactual of "my actions don't influence the chance the button is pressed." if you can figure out logical counterfactuals.
Or you can get the AI to simulate what it would do if it were an X maximizer. If it thinks the button won't be pressed, it does that, otherwise it does nothing. (not clear how to generalize to uncertain AI)
Here is a potential solution to stop button type problems, how does this go wrong?
Taking into account uncertainty, the algorithm is.
Calculate the X maximizing best action in a world where the stop button does nothing.
Calculate the X maximizing best action in a world where the stop button works.
If they are the same, do that. Otherwise shutdown.
Yes. If you have an AI that has been given a small, easily completable task, like putting one block on top of another with a robot arm, that is probably just going to do your simple task. The idea is that you build a fairly secure box, and give the AI a task it can fairly easily achieve in that box. (With you having no intention of pressing the button so long as the AI seems to be acting normally. ) We want to make "just do your task" the best strategy. If the box is less secure than we thought, or various other things go wrong, the AI will just shut down.
This approach could fail if the easiest way to cause trouble is nothing to do with the stop button. (Say it can send its code elsewhere over the internet. )
A generalization. You have some real world variable X. You run a bunch of AI's that are identical, except for different dogmatic beliefs about X. If they ever disagree on which action to take, the AI's are using the value of X in the optimization. Shutdown.
This acts as a kind of fuse. Have one AI confidant the door password is 1234, a copy that is confidant it is 5678. If these AI's ever disagree on what to do next then at least one was trying to open the door. Shutdown. A slightly counter-intuitive situation here is you want to make sure that if you did know the password, the easiest way to leave would be the door. Otherwise both AI will agree to climb out a window.