Hallucinations and GenAI for Code
Sometimes it’s better to say “I don’t know”…
I wanted to expand a particular submenu of the website I use for hosting my teaching materials by default. A quick online search and consulting the documentation of the framework I’m using didn’t reveal any insights.
I turned to OpenAI ChatGPT, which hallucinated an option nav_expanded: true. I then turned to Perplexity, since they do retrieval-augmented generation by default, that means it considers online resources when generating a response. This yielded another hallucinated option nav_fold_level: 1, which of course also didn’t exist. Interestingly, none of the sources that Perplexity listed documented that option.
I an ideal world, those tools would tell me “this is probably not possible.” Or simply “I don’t know”…
