A language model that confidently gives a wrong answer isn't broken. It's doing exactly what it was built to do. That's the issue.
Charles Sanders Peirce, the American logician writing in the late 1800s, drew a clean line between three modes of inference. Deduction