xwiki做的网站专业网站建设定制

xwiki做的网站,专业网站建设定制,谷歌云做网站,最好的网站开发平台RAG#xff08;Retrieval-Augmented Generation#xff09;在语言模型应用中已经相当成熟#xff0c;但传统实现往往只是简单的检索-生成流程。实际对话场景要复杂得多——用户的问题可能含糊不清#xff0c;或者会频繁追问#xff0c;还经常提些不相关的内容…RAGRetrieval-Augmented Generation在语言模型应用中已经相当成熟但传统实现往往只是简单的检索-生成流程。实际对话场景要复杂得多——用户的问题可能含糊不清或者会频繁追问还经常提些不相关的内容。这篇文章会展示怎么用 LangGraph 构建一个具备实用价值的 RAG 系统包括能够处理后续追问、过滤无关请求、评估检索结果的质量同时保持完整的对话记忆。传统 RAG 就是检索 top-k 文档然后喂给模型生成答案。但对话场景下会遇到几个棘手的问题用户问题经常模糊不清或者是追问检索到的 k 个文档可能压根不相关还有些问题完全不在系统能回答的范围内。这个问题的解决办法是把聊天历史、文档相关性评分、查询迭代优化以及智能路由结合起来让系统能够理解上下文还能自我修正。最终要实现的系统能做下面的事情接收对话式提问包括各种追问把问题改写成独立完整的形式判断问题是否属于可回答的范畴检索相关文档并评估质量没找到合适文档时自动优化查询重新检索达到尝试上限后要么给出答案要么返回无法回答整个过程保留完整对话历史。一、环境准备先确认 Python 版本需要 3.10 或更高,然后创建项目目录mkdir-p complex-agentic-rag cd complex-agentic-rag touch complex_agentic_rag.ipynb初始化虚拟环境uv init . uv venv source .venv/bin/activate装依赖包uv add langchain langgraph langchain-google-genai mypy pillow chromadb去 AI Studio 申请个 Gemini API key保存到.env文件里touch .env写入密钥GOOGLE_API_KEYyour_gemini_api_key别忘了更新.gitignoreecho.env .gitignore二、核心实现加载配置fromdotenvimportload_dotenv load_dotenv()准备测试数据这里用几个关于餐厅的文档做演示from langchain.schema import Document docs [ Document( page_content( Bella Vista is owned by Antonio Rossi, a renowned chef with over 20 years of experience in the culinary industry. He started Bella Vista to bring authentic Italian flavors to the community. ), metadata{source: owner.txt}, ), Document( page_content( Bella Vista offers a range of dishes with prices that cater to various budgets. Appetizers start at $8, main courses range from $15 to $35, and desserts are priced between $6 and $12. ), metadata{source: dishes.txt}, ), Document( page_content( Bella Vista is open from Monday to Sunday. Weekday hours are 11:00 AM to 10:00 PM, while weekend hours are extended from 11:00 AM to 11:00 PM. ), metadata{source: restaurant_info.txt}, ), Document( page_content( Bella Vista offers a variety of menus including a lunch menu, dinner menu, and a special weekend brunch menu. The lunch menu features light Italian fare, the dinner menu offers a more extensive selection of traditional and contemporary dishes, and the brunch menu includes both classic breakfast items and Italian specialties. ), metadata{source: restaurant_info.txt}, ), ]把文档编码后存到 Chroma 向量库里。用 Google 的 embedding 模型设置返回最相关的 2 个文档from langchain_community.vectorstores import Chroma from langchain_google_genai import GoogleGenerativeAIEmbeddings embedding_function GoogleGenerativeAIEmbeddings(modelmodels/embedding-001) db Chroma.from_documents(docs, embedding_function) retriever db.as_retriever(search_kwargs{k: 2})测试下检索功能retriever.invoke(who is the owner of bella vista?)输出应该包含老板 Antonio Rossi 的信息[Document(metadata{source: owner.txt}, page_contentBella Vista is owned by Antonio Rossi, a renowned chef with over 20 years of experience in the culinary industry. He started Bella Vista to bring authentic Italian flavors to the community.), Document(metadata{source: restaurant_info.txt}, page_contentBella Vista is open from Monday to Sunday. Weekday hours are 11:00 AM to 10:00 PM, while weekend hours are extended from 11:00 AM to 11:00 PM.)]Prompt 和 RAG 链需要定义一个带历史记录的提示模板这样模型能理解上下文from langchain_core.prompts import ChatPromptTemplate from langchain_google_genai import ChatGoogleGenerativeAI template Answer the question based on the following context and the chat history: Chat history: {history} Context: {context} Question: {question} prompt ChatPromptTemplate.from_template(template) llm ChatGoogleGenerativeAI(modelgemini-2.0-flash) rag_chain prompt | llm状态定义使用DialogState来追踪整个对话过程中的各种信息from typing import List, TypedDict from langchain_core.messages import BaseMessage, HumanMessage, AIMessage class DialogState(TypedDict): turns: List[BaseMessage] retrieved_docs: List[Document] topic_flag: str refined_query: str ready_for_response: bool refinement_attempts: int question: HumanMessage # ✅ This is the correct key to use整个系统由多个节点组成每个节点负责一个特定功能。问题重写器把追问改写成完整问题。分类器判断问题是否在回答范围内。路由器根据分类结果决定走哪条路径。检索器拉取相关文档。评分器用 LLM 判断文档是否真的相关。优化器在没找到合适文档时改写问题重试。生成器基于检索到的文档和历史对话生成回答。兜底节点处理超出范围或无法回答的情况。完整代码如下from langchain_core.messages import BaseMessage, HumanMessage, AIMessage, SystemMessage from langchain.schema import Document from pydantic import BaseModel, Field from langchain_core.prompts import ChatPromptTemplate from langgraph.graph import StateGraph, END class TopicGrade(BaseModel): score: str Field( descriptionIs the question about the target topics? If yes - Yes; if not - No ) def rephrase_query(state: DialogState): print(fEntering rephrase_query with state: {state}) # Reset derived fields state[retrieved_docs] [] state[topic_flag] state[refined_query] state[ready_for_response] False state[refinement_attempts] 0 if turns not in state or state[turns] is None: state[turns] [] if state[question] not in state[turns]: state[turns].append(state[question]) if len(state[turns]) 1: chat_history state[turns][:-1] question_text state[question].content prompt_msgs [ SystemMessage( contentYou are a helpful assistant that rephrases the users question to be a standalone question optimized for retrieval. ) ] prompt_msgs.extend(chat_history) prompt_msgs.append(HumanMessage(contentquestion_text)) prompt ChatPromptTemplate.from_messages(prompt_msgs).format() response llm.invoke(prompt) refined response.content.strip() print(frephrase_query: Rephrased to: {refined}) state[refined_query] refined else: state[refined_query] state[question].content return state def classify_topic(state: DialogState): print(Entering classify_topic) sys_msg SystemMessage( contentYou are a classifier that determines whether a users question is about one of the following topics: 1. Information about the owner of Bella Vista, which is Antonio Rossi. 2. Prices of dishes at Bella Vista (restaurant). 3. Opening hours of Bella Vista (restaurant). If the question IS about any of these topics, respond with Yes. Otherwise, respond with No. ) user_msg HumanMessage(contentfUser question: {state[refined_query]}) prompt ChatPromptTemplate.from_messages([sys_msg, user_msg]) structured_llm llm.with_structured_output(TopicGrade) grader prompt | structured_llm result grader.invoke({}) state[topic_flag] result.score.strip() print(fclassify_topic: topic_flag {state[topic_flag]}) return state def topic_router(state: DialogState): print(Entering topic_router) if state.get(topic_flag, ).strip().lower() yes: print(Routing to fetch_docs) return fetch_docs else: print(Routing to reject_off_topic) return reject_off_topic def fetch_docs(state: DialogState): print(Entering fetch_docs) docs retriever.invoke(state[refined_query]) print(ffetch_docs: Retrieved {len(docs)} documents) state[retrieved_docs] docs return state class RelevanceGrade(BaseModel): score: str Field( descriptionIs the document relevant to the users question? If yes - Yes; if not - No ) def evaluate_docs(state: DialogState): print(Entering evaluate_docs) sys_msg SystemMessage( contentYou are a grader assessing the relevance of a retrieved document to a user question. Only answer with Yes or No. If the document contains information relevant to the users question, respond with Yes. Otherwise, respond with No. ) structured_llm llm.with_structured_output(RelevanceGrade) relevant [] for doc in state[retrieved_docs]: user_msg HumanMessage( contentfUser question: {state[refined_query]}\n\nRetrieved document:\n{doc.page_content} ) prompt ChatPromptTemplate.from_messages([sys_msg, user_msg]) grader prompt | structured_llm result grader.invoke({}) print(fEvaluating doc: {doc.page_content[:30]}... Result: {result.score.strip()}) if result.score.strip().lower() yes: relevant.append(doc) state[retrieved_docs] relevant state[ready_for_response] len(relevant) 0 print(fevaluate_docs: ready_for_response {state[ready_for_response]}) return state def decision_router(state: DialogState): print(Entering decision_router) attempts state.get(refinement_attempts, 0) if state.get(ready_for_response, False): print(Routing to create_response) return create_response elif attempts 2: print(Routing to fallback_response) return fallback_response else: print(Routing to tweak_question) return tweak_question def tweak_question(state: DialogState): print(Entering tweak_question) attempts state.get(refinement_attempts, 0) if attempts 2: print(Max attempts reached) return state original state[refined_query] sys_msg SystemMessage( contentYou are a helpful assistant that slightly refines the users question to improve retrieval results. Provide a slightly adjusted version of the question. ) user_msg HumanMessage(contentfOriginal question: {original}) prompt ChatPromptTemplate.from_messages([sys_msg, user_msg]).format() response llm.invoke(prompt) refined response.content.strip() print(ftweak_question: Refined to: {refined}) state[refined_query] refined state[refinement_attempts] attempts 1 return state def create_response(state: DialogState): print(Entering create_response) if turns not in state or state[turns] is None: raise ValueError(State must include turns before generating an answer.) history state[turns] context state[retrieved_docs] question state[refined_query] response rag_chain.invoke({ history: history, context: context, question: question }) result response.content.strip() state[turns].append(AIMessage(contentresult)) print(fcreate_response: Answer generated: {result}) return state def fallback_response(state: DialogState): print(Entering fallback_response) if turns not in state or state[turns] is None: state[turns] [] state[turns].append( AIMessage(contentIm sorry, but I couldnt find the information youre looking for.) ) return state def reject_off_topic(state: DialogState): print(Entering reject_off_topic) if turns not in state or state[turns] is None: state[turns] [] state[turns].append( AIMessage(contentI cant respond to that!) ) return state最后用 LangGraph 的StateGraph和MemorySaver把各个节点串起来。MemorySaver能把对话历史持久化保存from langgraph.graph import StateGraph, END from langgraph.checkpoint.memory import MemorySaver # Initialize memory checkpointer to persist chat history checkpointer MemorySaver() workflow StateGraph(DialogState) workflow.add_node(rephrase_query, rephrase_query) workflow.add_node(classify_topic, classify_topic) workflow.add_node(reject_off_topic, reject_off_topic) workflow.add_node(fetch_docs, fetch_docs) workflow.add_node(evaluate_docs, evaluate_docs) workflow.add_node(create_response, create_response) workflow.add_node(tweak_question, tweak_question) workflow.add_node(fallback_response, fallback_response) workflow.add_edge(rephrase_query, classify_topic) workflow.add_conditional_edges( classify_topic, topic_router, { fetch_docs: fetch_docs, reject_off_topic: reject_off_topic, }, ) workflow.add_edge(fetch_docs, evaluate_docs) workflow.add_conditional_edges( evaluate_docs, decision_router, { create_response: create_response, tweak_question: tweak_question, fallback_response: fallback_response, }, ) workflow.add_edge(tweak_question, fetch_docs) workflow.add_edge(create_response, END) workflow.add_edge(fallback_response, END) workflow.add_edge(reject_off_topic, END) workflow.set_entry_point(rephrase_query) graph workflow.compile(checkpointercheckpointer)还可以生成流程图看看整个系统的结构from IPython.display import Image, display from langchain_core.runnables.graph import MermaidDrawMethod display( Image( graph.get_graph().draw_mermaid_png( draw_methodMermaidDrawMethod.API, ) ) )实际效果演示场景 1超范围问题问个跟餐厅无关的input_data {question: HumanMessage(contentHow is the weather?)} graph.invoke(inputinput_data, config{configurable: {thread_id: 1}})输出Entering rephrase_query with state: {question: HumanMessage(contentHow is the weather?, additional_kwargs{}, response_metadata{})} Entering classify_topic classify_topic: topic_flag No Entering topic_router Routing to reject_off_topic Entering reject_off_topic {turns: [HumanMessage(contentHow is the weather?, additional_kwargs{}, response_metadata{}), AIMessage(contentI cant respond to that!, additional_kwargs{}, response_metadata{})], retrieved_docs: [], topic_flag: No, refined_query: How is the weather?, ready_for_response: False, refinement_attempts: 0, question: HumanMessage(contentHow is the weather?, additional_kwargs{}, response_metadata{})}系统识别出这是超范围问题直接拒绝了。场景 2找不到答案的问题问个文档里没有的信息input_data {question: HumanMessage(contentHow old is the owner of the restaurant Bella Vista?)} graph.invoke(inputinput_data, config{configurable: {thread_id: 2}})输出Entering rephrase_query with state: {question: HumanMessage(contentHow old is the owner of the restaurant Bella Vista?, additional_kwargs{}, response_metadata{})} Entering classify_topic classify_topic: topic_flag Yes Entering topic_router Routing to fetch_docs Entering fetch_docs fetch_docs: Retrieved 2 documents Entering evaluate_docs Evaluating doc: Bella Vista is owned by Antoni... Result: No Evaluating doc: Bella Vista is open from Monda... Result: No evaluate_docs: ready_for_response False Entering decision_router Routing to tweak_question Entering tweak_question tweak_question: Refined to: Revised question: What is the age of the owner of Bella Vista restaurant? Entering fetch_docs fetch_docs: Retrieved 2 documents Entering evaluate_docs Evaluating doc: Bella Vista is owned by Antoni... Result: No Evaluating doc: Bella Vista is open from Monda... Result: No evaluate_docs: ready_for_response False Entering decision_router Routing to tweak_question Entering tweak_question tweak_question: Refined to: Original question: What is the capital of Australia? Revised question: What is the capital city of Australia? Entering fetch_docs fetch_docs: Retrieved 2 documents Entering evaluate_docs Evaluating doc: Bella Vista offers a variety o... Result: No Evaluating doc: Bella Vista is open from Monda... Result: No evaluate_docs: ready_for_response False Entering decision_router Routing to fallback_response Entering fallback_response {turns: [HumanMessage(contentHow old is the owner of the restaurant Bella Vista?, additional_kwargs{}, response_metadata{}), AIMessage(contentIm sorry, but I couldnt find the information youre looking for., additional_kwargs{}, response_metadata{})], retrieved_docs: [], topic_flag: Yes, refined_query: Original question: What is the capital of Australia?\nRevised question: What is the capital city of Australia?, ready_for_response: False, refinement_attempts: 2, question: HumanMessage(contentHow old is the owner of the restaurant Bella Vista?, additional_kwargs{}, response_metadata{})}问题本身属于可回答范围但检索到的文档都不相关。系统尝试改写问题重新检索了两次最后诚实地承认找不到答案。场景 3正常对话加追问先问营业时间再追问周日的情况graph.invoke(input{question: HumanMessage(contentWhen does Bella Vista open?)}, config{configurable: {thread_id: 3}}) graph.invoke(input{question: HumanMessage(contentAlso on Sunday?)}, config{configurable: {thread_id: 3}})输出Entering rephrase_query with state: {question: HumanMessage(contentWhen does Bella Vista open?, additional_kwargs{}, response_metadata{})} Entering classify_topic classify_topic: topic_flag Yes Entering topic_router Routing to fetch_docs Entering fetch_docs fetch_docs: Retrieved 2 documents Entering evaluate_docs Evaluating doc: Bella Vista is open from Monda... Result: Yes Evaluating doc: Bella Vista offers a range of ... Result: No evaluate_docs: ready_for_response True Entering decision_router Routing to create_response Entering create_response create_response: Answer generated: Bella Vista opens at 11:00 AM every day. Entering rephrase_query with state: {turns: [HumanMessage(contentWhen does Bella Vista open?, additional_kwargs{}, response_metadata{}), AIMessage(contentBella Vista opens at 11:00 AM every day., additional_kwargs{}, response_metadata{})], retrieved_docs: [Document(metadata{source: restaurant_info.txt}, page_contentBella Vista is open from Monday to Sunday. Weekday hours are 11:00 AM to 10:00 PM, while weekend hours are extended from 11:00 AM to 11:00 PM.)], topic_flag: Yes, refined_query: When does Bella Vista open?, ready_for_response: True, refinement_attempts: 0, question: HumanMessage(contentAlso on Sunday?, additional_kwargs{}, response_metadata{})} rephrase_query: Rephrased to: AI: Is Bella Vista open on Sundays? Entering classify_topic classify_topic: topic_flag Yes Entering topic_router Routing to fetch_docs Entering fetch_docs fetch_docs: Retrieved 2 documents Entering evaluate_docs Evaluating doc: Bella Vista is open from Monda... Result: Yes Evaluating doc: Bella Vista offers a variety o... Result: No evaluate_docs: ready_for_response True Entering decision_router Routing to create_response Entering create_response create_response: Answer generated: Yes, Bella Vista is open on Sundays. {turns: [HumanMessage(contentWhen does Bella Vista open?, additional_kwargs{}, response_metadata{}), AIMessage(contentBella Vista opens at 11:00 AM every day., additional_kwargs{}, response_metadata{}), HumanMessage(contentAlso on Sunday?, additional_kwargs{}, response_metadata{}), AIMessage(contentYes, Bella Vista is open on Sundays., additional_kwargs{}, response_metadata{})], retrieved_docs: [Document(metadata{source: restaurant_info.txt}, page_contentBella Vista is open from Monday to Sunday. Weekday hours are 11:00 AM to 10:00 PM, while weekend hours are extended from 11:00 AM to 11:00 PM.)], topic_flag: Yes, refined_query: AI: Is Bella Vista open on Sundays?, ready_for_response: True, refinement_attempts: 0, question: HumanMessage(contentAlso on Sunday?, additional_kwargs{}, response_metadata{})}系统利用对话历史正确理解了Also on Sunday?这个追问把它改写成完整问题后成功检索并回答。到这里就完成了一个具备自适应能力的 RAG 系统不只是做简单的相似度搜索。总结LangGraph 的图架构让这类系统的开发和迭代变得相对简单可以方便地加入新节点、接入外部 API或者连接到其他 agent 形成更复杂的工作流。但是还应该注意一下问题检索质量本质上还是受 embedding 效果制约改写查询能缓解但解决不了根本问题。每次用 LLM 给文档打分会增加响应延迟和调用成本文档多了开销就很明显。重试次数现在写死的实际使用可能需要根据场景动态调整。当前只保留会话内的记忆真正的长期记忆比如结合向量库和关系数据库需要额外设计。分类器的判断逻辑是硬编码在 prompt 里的扩展到新领域得改提示词或者训练专门的分类器。如何学习大模型 AI 由于新岗位的生产效率要优于被取代岗位的生产效率所以实际上整个社会的生产效率是提升的。但是具体到个人只能说是“最先掌握AI的人将会比较晚掌握AI的人有竞争优势”。这句话放在计算机、互联网、移动互联网的开局时期都是一样的道理。我在一线互联网企业工作十余年里指导过不少同行后辈。帮助很多人得到了学习和成长。我意识到有很多经验和知识值得分享给大家也可以通过我们的能力和经验解答大家在人工智能学习中的很多困惑所以在工作繁忙的情况下还是坚持各种整理和分享。但苦于知识传播途径有限很多互联网行业朋友无法获得正确的资料得到学习提升故此将并将重要的AI大模型资料包括AI大模型入门学习思维导图、精品AI大模型学习书籍手册、视频教程、实战学习等录播视频免费分享出来。第一阶段10天初阶应用该阶段让大家对大模型 AI有一个最前沿的认识对大模型 AI 的理解超过 95% 的人可以在相关讨论时发表高级、不跟风、又接地气的见解别人只会和 AI 聊天而你能调教 AI并能用代码将大模型和业务衔接。大模型 AI 能干什么大模型是怎样获得「智能」的用好 AI 的核心心法大模型应用业务架构大模型应用技术架构代码示例向 GPT-3.5 灌入新知识提示工程的意义和核心思想Prompt 典型构成指令调优方法论思维链和思维树Prompt 攻击和防范…第二阶段30天高阶应用该阶段我们正式进入大模型 AI 进阶实战学习学会构造私有知识库扩展 AI 的能力。快速开发一个完整的基于 agent 对话机器人。掌握功能最强的大模型开发框架抓住最新的技术进展适合 Python 和 JavaScript 程序员。为什么要做 RAG搭建一个简单的 ChatPDF检索的基础概念什么是向量表示Embeddings向量数据库与向量检索基于向量检索的 RAG搭建 RAG 系统的扩展知识混合检索与 RAG-Fusion 简介向量模型本地部署…第三阶段30天模型训练恭喜你如果学到这里你基本可以找到一份大模型 AI相关的工作自己也能训练 GPT 了通过微调训练自己的垂直大模型能独立训练开源多模态大模型掌握更多技术方案。到此为止大概2个月的时间。你已经成为了一名“AI小子”。那么你还想往下探索吗为什么要做 RAG什么是模型什么是模型训练求解器 损失函数简介小实验2手写一个简单的神经网络并训练它什么是训练/预训练/微调/轻量化微调Transformer结构简介轻量化微调实验数据集的构建…第四阶段20天商业闭环对全球大模型从性能、吞吐量、成本等方面有一定的认知可以在云端和本地等多种环境下部署大模型找到适合自己的项目/创业方向做一名被 AI 武装的产品经理。硬件选型带你了解全球大模型使用国产大模型服务搭建 OpenAI 代理热身基于阿里云 PAI 部署 Stable Diffusion在本地计算机运行大模型大模型的私有化部署基于 vLLM 部署大模型案例如何优雅地在阿里云私有部署开源大模型部署一套开源 LLM 项目内容安全互联网信息服务算法备案…学习是一个过程只要学习就会有挑战。天道酬勤你越努力就会成为越优秀的自己。如果你能在15天内完成所有的任务那你堪称天才。然而如果你能完成 60-70% 的内容你就已经开始具备成为一名大模型 AI 的正确特征了。这份完整版的大模型 AI 学习资料已经上传CSDN朋友们如果需要可以微信扫描下方CSDN官方认证二维码免费领取【保证100%免费】