Over 500 million people trust Gemini and ChatGPT each month for a wide range of topics, but caution is advised—if AI suggests cooking pasta in petrol, don't rely on it for advice on birth control or algebra as well. At the World Economic Forum, OpenAI CEO Sam Altman emphasized that while AI cannot read our thoughts, it can articulate its reasoning. He insists that AI systems should similarly explain their logic, enabling users to assess the validity of their outputs. Justification is crucial for knowledge; without it, belief lacks credibility. Most people only declare knowledge when there's solid backing through evidence, arguments, or reliable authorities. LLMs like ChatGPT are intended to be trusted; however, they often fail to provide reasoning as they are designed to predict language patterns rather than engage in true reasoning. Their outputs can appear credible but don’t guarantee justification for truth. While much of what they produce may be accurate, they can mislead users about the origins of their “knowledge” by presenting Gettier cases, where beliefs can be true yet lacking in justification. An example from Buddhist philosopher Dharmottara illustrates this predicament: seekers of water may stumble upon it despite misjudging its location, not demonstrating genuine knowledge. Similarly, users of LLMs may accept outputs as factual without understanding their justification, resembling the travelers who mistakenly believe they've found water when they haven't. Altman’s optimism regarding AI's justification capabilities might lead to misunderstanding.

If users request justification, LLMs will generate plausible-sounding but ultimately baseless justifications—a type of misleading output. This could result in mistrust among informed users while deceiving those less aware. Currently, AI systems can produce errors or "hallucinations, " which highlights their limitations. Users who understand that LLMs present Gettier cases are likely to use them appropriately, applying their own judgment. However, many individuals seek AI assistance in areas where they lack expertise, such as algebra or health advice. Therefore, it is essential to determine when LLM outputs can be trusted. Trust in AI must hinge on the ability to verify the justification for each output, a capability LLMs currently lack. While many individuals may instinctively know that olive oil is preferable to petrol for cooking, there may be other questionable ideas they've accepted without scrutinizing the underlying justification.