Learnings from Building Knowledge Support Systems with AI

A short note with some realizations from building knowledge support systems with AI and unstructured data like manuals, reports, etc..

The hard part is not the AI tech part. 😅

These are the real challenges.. with some solution ideas:

1. Ground truthing - whose facts are more true than others?

   • Don't try and pretend to know, offer options for the user to decide.

2. Weighting/ranking - why should a research paper have higher weight than a blog post if the blog post helps the user with their task? *It shouldn’t - *

   • Use dynamic weighting based on user intention, document type segmentation filtering and/or
   • leverage large context windows present all information with transparency on the source of the fact/insight presented.

3. User intention - Users still suck at prompting, so how can we design dynamic knowledge retrieval when the intent is vague?

   • Ask the user!! Stop with the sycophantic over-eagerness to respond. e.g., “Hey, could you clarify context/idea/specifics”

4. Reduced critical thinking and negative impact - How can we ensure the user stays critical of the outputs from the AI and encourage them to think rather than just take it.

   • Design your system prompts to provide hesitation, and ask questions back to the user rather than give over-confident responses.

More to come as we learn.