Humans need to read through the full length of LLM responses to verify them - can we eliminate that? Ever?
People make mistakes. Similarly LLMs hallucinate.
When working with a team, we look for people who make fewer mistakes - as we work with a person, we understand their unique strengths and weaknesses. Then, we settle into a balance - knowing when to trust and when to check their results in detail, or find an alternate resource where they have weaknesses.
LLMs are somewhat similar - as we work with them, we can understand where they can be correct and where wrong. But they are also different, because, for all but the most trivial cases, we need to check the responses in detail - the longer and more complex the response, the more work we need to put in.
Even as LLMs get better over time, I don't know if we can ever eliminate that minimum linear order effort of going through everything they say before using it.
I have found some success (and there is some research that validates it) of asking an LLM to verify its own past work, or asking a different LLM to verify, or even asking multiple LLMs and taking the majority vote (or weighted majority based on LLM capability) to check, correct past LLM responses. And this does catch and correct things, can be made more efficient by prompting properly for the verification.
However, I still need to read the corrected response if the response is critical for something I need. So, in the end, doesn't beat the linear complexity for human effort on each response.
Any thoughts on how we can get past this problem?