Hallucinations and GenAI for Code
Sometimes it’s better to say “I don’t know”…
I wanted to expand a particular submenu of the website I use for hosting my teaching materials by default. A quick online search and consulting the documentation of the framework I’m using didn’t reveal any insights.
I turned to OpenAI’s ChatGPT, which hallucinated an option nav_expanded: true.
I then turned to Perplexity, since they do retrieval-augmented generation by default (meaning the tool considers online resources when generating a response).
This yielded another hallucinated option nav_fold_level: 1, which of course also didn’t exist.
Interestingly, none of the sources that Perplexity listed documented that option.
In an ideal world, those tools would tell me “this is probably not possible.” Or simply “I don’t know”…
