AI for Knowledge Extraction
Learnings from Building Knowledge Support Systems with AI
The hard part in building AI-powered knowledge retrieval systems is not really the AI part.
Learn about AI and how it can help your work, from basics to expert advice.
The hard part in building AI-powered knowledge retrieval systems is not really the AI part.
Exploring an innovative approach combining AI and human expertise to analyze development programs. The framework uses large language models to systematically assess program documentation, demonstrated through a case study of locally-led development initiatives, ensuring traceable and consistent analysis.
Olivier
Founder, Baobab Tech
Proof-of-concept of a novel methodology for extracting rich, temporal-aware knowledge graphs from humanitarian sector reports, focusing specifically on a single sub-set of documentation: Ukraine
Olivier
Founder, Baobab Tech
The hard part in building AI-powered knowledge retrieval systems is not really the AI part.
ChatGPT reached 700 million users by 2025, with 73% using it for personal tasks rather than work. Three main uses dominate: practical guidance, information seeking, and writing assistance, primarily supporting decision-making over task automation.
Are you ready to push the boundaries of AI for good? Baobab Tech is on the hunt for a part-time/contract AI Developer. We're looking for someone who can think outside the box, learn at lightning speed, and isn't afraid to get their hands dirty in the world of applied AI.
Addressing the AI industry's diversity imbalance, committing to concrete actions to support women and underrepresented groups, and calling for collective effort to shape a more equal future.
Exploring an innovative approach combining AI and human expertise to analyze development programs. The framework uses large language models to systematically assess program documentation, demonstrated through a case study of locally-led development initiatives, ensuring traceable and consistent analysis.
Proof-of-concept of a novel methodology for extracting rich, temporal-aware knowledge graphs from humanitarian sector reports, focusing specifically on a single sub-set of documentation: Ukraine
A practical discussion for development organizations and funders implementing AI solutions, offering a staged approach to gathering meaningful evidence beyond traditional metrics. Emphasizes locally-led evaluation, ethical considerations, and the importance of demonstrating genuine social impact.
Multi-vector embeddings enhance knowledge retrieval by separately encoding content and document/information structural information, enabling more nuanced search across diverse document types. This approach combines semantic understanding with contextual awareness, improving retrieval accuracy for complex queries while maintaining structural relevance.
Adopting a human-centric approach to AI-powered knowledge retrieval prioritizes usability and decision-making support. By focusing on tailored information presentation, speed, adaptive interactions, and seamless workflow integration, these systems enhance user effectiveness and align with diverse human needs.
How can we make development and humanitarian data more useful? Transform the data architecture with embedded intelligence and edge processing. Move beyond static storage to enable AI-powered development tools through vector embeddings, smart caching, and natural language APIs while reducing computational redundancy and environmental impact
Evaluating knowledge retrieval systems using LLMs can be intimidating for developers, but it’s essential for success. This guide simplifies the process, breaking down key evaluation steps for retrieval and generation to ensure accurate and relevant results in your applications.
Are you ready to push the boundaries of AI for good? Baobab Tech is on the hunt for a part-time/contract AI Developer. We're looking for someone who can think outside the box, learn at lightning speed, and isn't afraid to get their hands dirty in the world of applied AI.
Baobab Tech's "Rapid Research" process leverages AI through a compact 300-line script transforming information synthesis. This innovative approach saves hundreds of research hours, enabling quick, comprehensive topic scans and informed decision-making for development and humanitarian projects.
We advocate for a shift towards accessible fine-tuning tools, specialized AI models, and agentic frameworks with smaller actors. It addresses the limitations of large foundation models while improving efficiency, flexibility, and context-awareness in artificial intelligence development and deployment.
Integrating advanced strategies in building LLM applications, focusing on optimized web data preprocessing, domain-specific customizations, and UI/UX enhancements. Techniques like smart web crawling, effective data parsing, and employing advanced embedding models improve responsiveness and efficiency. The continuous evolution in tools and methods promises significant impacts across various domains.
We introduce a straightforward method to transform log frame data into a standardized, LM-friendly format, enhancing project analysis, comparison, and reporting. This practical approach simplifies understanding and managing diverse project frameworks.
The future and relevancy of knowledge lies in the localization of knowledge exchange. We are building localization into WASH AI and a country-level focus of WASH knowledge sharing.
Idea of a LangGraph powered AI-assistant that searches, filters, summarizes, and classifies resources based on user requirements, providing a curated dataset and a detailed log of its actions.
Exploring two novel approaches to enriching a domain knowledge base: an Expert Knowledge Capture Interface for direct insights and a series of transcript-captured Expert Webinars, blending in-depth discussions with interactive community engagement to refine AI-driven recommendations.
Many great tools are being built such as Custom GPTs, Perplexity AI and Consensus. WASH AI distinguishes itself in the AI knowledge tool landscape, offering updated, sector-specific insights and inclusive knowledge management, surpassing limitations of traditional AI tools with its user-centric, customizable approach tailored for the Water, Sanitation, and Hygiene sector.
Large Language Models revolutionize knowledge dissemination, providing tailored insights for professionals, especially in the WASH sector, through advanced information processing. Their effectiveness, however, requires careful, ethical use to ensure data integrity and relevant, reliable knowledge translation.
The LLM judge technique uses large language models to evaluate AI assistant responses, mimicking human assessments. Applied thoughtfully, it offers a scalable way to ensure AI outputs are aligned with human values, enhancing trust and engagement.
Looking at two recently research papers on Machine Translation, this article synthesizes key advancements in machine translation as applied to Large Language Models (LLMs). Combining insights from cross-lingual supervision with techniques for improving translation faithfulness, we explore the significance of these developments for organizations aiming to democratize knowledge.
This article examines the challenges and opportunities in the humanitarian and development sector's knowledge platforms. Highlighting biases and limitations in conventional knowledge bases, it emphasizes the importance of incorporating diverse perspectives, capturing tacit wisdom, and leveraging AI. The piece also outlines strategies for making knowledge platforms more inclusive, responsive, and locally relevant.
This opinion piece explores the impact of AI beyond its conventional perception as a tool, focusing on its transformative effect on jobs and education. It calls for a revised educational approach to prepare for an AI-driven future, emphasizing creativity and problem-solving over routine tasks
Generative AI brings immense possibilities along with risks. This post explores how organizations in the development and humanitarian sector can thoughtfully apply generative AI to maximize benefits and mitigate harms.
In an article by The Economist, "Sapiens" author Yuval Noah Harari explores the profound potential impacts of AI on human civilization, touching on themes of storytelling, intimacy, cultural dominance, illusions, and the urgent need for regulation and transparency.
Exploring how AI technologies can transform knowledge management by unhiding valuable insights from vast data, facilitating sector-wide knowledge and support systems.
A novel concept in machine translation - using embeddings as a translation memory. It explores the creation of the Translation Vector Database, the process of building semantic bridges, the advent of larger context windows in language models, and the implications of these innovations on the future of machine translation.
LongMem framework's potential to break the memory barrier in Large Language Models (LLMs), enhancing their ability to utilize long-form content.
AI tools can enhance the coordination of emergency response by processing unstructured data from meeting notes and conversations, and generating structured data and maps on-the-fly.
Exploring the integration of Large Language Models (LLMs) into the Geographic Information System (GIS), leading to the development of Autonomous GIS. It discusses the potential of this integration to revolutionize spatial analysis, making it faster, easier, and more accessible.
Transformers have revolutionized AI, specifically in Natural Language Processing (NLP). Their adaptability has led to their use in multimodal tasks, far beyond NLP. With further advancements in tooling and accessibility, coupled with the popularization through chatbots, Transformers continue to dominate. Alongside, Diffusion models emerge as a new force in image generation.
AI models and approaches like RAG, SPABERT, GPT-4, and multimodal foundation models can harness geospatial data to enhance decision-making in humanitarian and development contexts.
AI can be used to translate technical content for users who are not native English speakers, enhancing the accessibility and efficiency of technical support. It uses the example of translating a technical manual from English to Swahili using Transformer models.
Exploring the transformative impact of multimodal models on AI applications, particularly in technical support. It discusses how these models, which can process and integrate multiple types of data, can provide more comprehensive and intuitive support to users. The post also highlights a case study of the MultiModal-GPT model, which demonstrates the potential of multimodal models in maintaining continuous dialogues with humans and effectively understanding and describing images.
Discussing MIT professor Jacob Andreas' insights on large language models (LLMs) and their implications for domain-specific Q&A systems. It covers topics such as contextual understanding, in-context learning, factuality and coherence in LLMs, and the future of these models.
Emphasizing the importance of negative data in the humanitarian and development sectors, this article explains the impact of biased information on AI systems. It calls for balanced documentation of successes and failures for a more comprehensive implementation of AI support tools.
Strategies for reducing costs associated with the use of Large Language Models (LLMs) in AI-powered applications, based on a recent research paper. It introduces the concept of FrugalGPT, a model that optimizes the use of different LLMs to reduce cost and improve accuracy.
Explore the role of entailment in AI and its application in enhancing natural language understanding models, with insights from a recent research paper and implications for organizations leveraging AI.
Processing long text has never been easier with SLED, a simple yet effective approach for understanding long sequences.
DPO: Simplifying the alignment of AI models with human preferences.
Explore the top challenges and opportunities non-profit organizations face in adopting AI tools to enhance their impact in the humanitarian and development sectors.
AI-driven research explores how curricula shape future thinking skills. Data analysis and AI modeling inform the design of transformative education systems.
Unlock the power of AI with Zero-Shot Learning for efficient text and data analysis from reports, enabling precise predictions and swift decision-making.
Understand the key differences between Base and Instruction Tuned Language Learning Models to harness the true potential of AI for your business.
Explore the power of Reinforcement Learning from Human Feedback (RLHF) in transforming data into knowledge, with practical examples in text summarization.
Discover how to extract valuable insights from support conversations using methods like Topic Modeling, Text Classification, and more.
Unveil the transformative role of AI in language translation, bridging the gap and breaking down language barriers.
Uncover how AI contributes to knowledge synthesis, unlocking the potential to turn vast amounts of information into valuable insights.
WASH AI is new initiative that uses AI to enhance global WASH sector knowledge, education, and technical support, bridging language gaps and augmenting human capabilities for better decision-making.
Discover the fascinating world of Generative Adversarial Networks (GANs) and their artistic capabilities in AI.
Embark on an exciting journey to understand Language Models, the linguistic masterpieces that revolutionize natural language understanding and generation in AI.
Discover the pivotal role of AI in data analysis and how it empowers businesses to uncover hidden patterns and make data-driven decisions.
Discover the history and significance of Artificial Intelligence (AI) as we delve into its development and transformative impact.