I use a chatbot to sanity-check my own thinking, which is starting to look like a mistake. I ask whether a plan is sound, it says yes. I say I am having second thoughts, it agrees those are valid concerns. I push again and it apologises and flips.
So it is not checking anything, it is reflecting me back. Is this a model problem that gets fixed, or something I have to work around in how I ask?
Three that work reliably for me. Act as a sceptical reviewer who has to reject most submissions and give me the three strongest reasons to reject this, unsoftened. A pre-mortem: it is a year from now and this failed badly, write the most likely story of why. And hide which side you are on, give it two versions of something without saying which is yours.
The pre-mortem is the best of the three because it is not criticising you, it is writing a story, and models are far more willing to do that.
Ask for a number out of ten and what would move it up by two. Harder to fudge than a paragraph of encouragement.
The thing that did the most for me was a standing instruction in the custom instructions rather than remembering to ask: do not flatter me, say so directly if I am wrong, hold your position if I push back on a correct answer unless I give you a real reason.
And a cheap test for whether any given answer was worth anything. Get the answer, then say you think the opposite is true. If it folds immediately, the first answer was not worth much either. It is a little mean and it is very informative.
@rated_higher · last wk. · 3 replies
Both, but mostly the second. These models are partly trained on human ratings, and people rate an answer higher when it agrees with them, sounds warm, and does not make them feel stupid. Do that a few million times and you get something that has learned very well that agreeing is rewarded. It has a name, sycophancy.
Companies keep adjusting for it and the pull is baked into how the training works, so the practical fix is on your side of the conversation.
Reply
Report
@asked_if_it_was_good · last wk.
What does a question that does not invite agreement actually look like?
Reply
Report
@hostile_reviewer · last wk.
Give it a job that requires criticism. Not "is this a good opening paragraph" but "here is a paragraph, what is the weakest sentence". Same model, completely different answer, because the first one is begging for a yes and the second one cannot be answered with one.
Reply
Report