Building an Enterprise-Level Knowledge Base with Dify

1. Project Introduction

Project Background
Our company's overseas sales team regularly writes bids for clients, accumulating a vast amount of proprietary knowledge documents. These documents cover various aspects, including company profiles, financial statements, product manuals, maintenance handbooks, environmental protection, and cultural considerations. However, these files were not archived in a knowledge base, leading to significant inefficiencies. New team members required at least three months to familiarize themselves with the bidding process. Additionally, the quality and accuracy of responses to client questions varied with experience, affecting our bid success rate. Therefore, we needed a system to standardize and improve efficiency.

Project Objectives

• Establish an enterprise-level knowledge base
• Standardize the bidding process
• Increase bid response efficiency by 100%
• Achieve an AI question-answering accuracy rate of over 80%

Project Timeline
March 2024 - September 2024

Project Location
Internal enterprise project, no public address

2. Project Solution

Overall Approach
We first used AI to streamline the business process and then gradually built the enterprise-level knowledge base.

Product Iterations

• Demo
• MVP
• Productization 1.0

2.1 Demo

Solution Overview
Dify + Feishu Integration + Feishu Multi-Dimensional Table

Solution Description
Dify is an open-source LLM + RAG project that solved the issues of knowledge base file uploads and knowledge extraction, and provided ready-to-use APIs, significantly speeding up the demo development process. Feishu Integration, a low-code platform, quickly connected Dify's API with Feishu Multi-Dimensional Table to achieve batch AI responses, completing the basic business process demo.

Outcome
AI response rate: 40%
AI accuracy rate: 70%
Given that this was a demo and the Q&A pairs were not manually optimized, we deemed this result acceptable.

2.2 MVP

User Feedback
Business users needed to access original files along with AI responses, as clients required proof documents. Additionally, the accuracy of answers needed improvement.

Solution Optimization
We replaced the Feishu Multi-Dimensional Table with a Feishu Robot, allowing users to consult the robot directly in Feishu groups without disrupting their existing workflow. The robot also provided file links along with answers. Moreover, we used Python scripts to optimize product keywords in question titles, enhancing question matching and response rates.

Outcome
Sales team AI usage coverage exceeded 50%
AI response rate exceeded 80%

2.3 Productization 1.0

User Feedback
Some users knew where the files were but found it inconvenient to query them, especially PDF files. Also, many product parameters were stored in the PDM system, which AI could not access.

Solution Optimization
We implemented Meilisearch to build a file search engine and optimized PDF parsing logic to improve text extraction. We also established connectors between the knowledge base and internal systems to dynamically update Dify's knowledge base Q&A pairs, addressing the issue of knowledge timeliness.

Outcome
AI response rate and accuracy rate both exceeded 80%
We established a general enterprise knowledge base management framework and implemented permission grading to enhance knowledge security

3. Deliverables

• Enterprise-level knowledge base + RAG + file search engine
• AI response rate and accuracy rate both exceeded 80%
• Reduced dependence on bid experts, increasing bid response efficiency by 80%

4. Summary & Reflection

• Deliver products early to collect user feedback
• Listen to user needs and focus on the most important features
• Implement data tracking and analysis to improve product metrics
  Repeat steps 1-3