我用LangChain搭了个代码审查Agent，踩了3个坑

我想让Agent自动Review代码，省点人力。

一开始觉得很简单：调用GitHub API获取PR diff，扔给GPT-4，拿到意见返回。

结果踩了3个坑，浪费了一周时间。

坑1：Token超限，Review只看了一半

现象：PR有2000行代码，Agent只看了前500行就给意见了。

原因：GPT-4的上下文窗口是8K tokens，2000行代码加上diff格式，早就超了。

第一次尝试：

from langchain_openai import ChatOpenAI from langchain.chains import LLMChain from langchain.prompts import ChatPromptTemplate llm = ChatOpenAI(model="gpt-4", temperature=0) prompt = ChatPromptTemplate.from_template( "Review this PR:\n\n{diff}\n\nProvide suggestions." ) chain = LLMChain(llm=llm, prompt=prompt) result = chain.run(diff=pr_diff)

结果：只看了前300行。

解决：分块Review，每块500行，最后汇总。

from langchain.text_splitter import RecursiveCharacterTextSplitter splitter = RecursiveCharacterTextSplitter( chunk_size=4000, chunk_overlap=0, separators=["\n\n", "\n", " "] ) chunks = splitter.split_text(pr_diff) reviews = [] for chunk in chunks: review = chain.run(diff=chunk) reviews.append(review) # 汇总所有意见 summary_prompt = ChatPromptTemplate.from_template( "Summarize these reviews:\n\n{reviews}" ) summary_chain = LLMChain(llm=llm, prompt=summary_prompt) final_review = summary_chain.run(reviews="\n".join(reviews))

坑2：给了一堆"建议"但没一个能执行的

现象：Agent说"建议优化性能"、"建议加错误处理"，但没说具体改哪行。

原因：Prompt太泛，没有强制要求具体行号和代码示例。

第一次Prompt：

Review this PR and provide suggestions.

结果：输出一堆泛泛而谈的建议。

解决：结构化输出，要求具体行号和代码示例。

from langchain.output_parsers import PydanticOutputParser from pydantic import BaseModel, Field from typing import List class CodeReviewIssue(BaseModel): line_number: int = Field(description="Line number of the issue") severity: str = Field(description="Issue severity: high/medium/low") description: str = Field(description="Issue description") suggestion: str = Field(description="How to fix") class CodeReview(BaseModel): issues: List[CodeReviewIssue] = Field(description="List of issues") parser = PydanticOutputParser(pydantic_object=CodeReview) prompt = ChatPromptTemplate.from_template( "Review this PR:\n\n{diff}\n\n" "{format_instructions}\n\n" "Return ONLY the structured output." ) format_instructions = parser.get_format_instructions() chain = LLMChain( llm=llm, prompt=prompt, output_parser=parser ) review = chain.run(diff=chunk, format_instructions=format_instructions) for issue in review.issues: print(f"Line {issue.line_number} [{issue.severity}]: {issue.description}") print(f"Suggestion: {issue.suggestion}\n")

输出示例：

Line 42 [high]: Potential SQL injection vulnerability Suggestion: Use parameterized queries: cursor.execute("SELECT * FROM users WHERE id = %s", (user_id,)) Line 87 [medium]: Missing error handling Suggestion: Add try-except block: try: response = requests.get(url) response.raise_for_status() except RequestException as e: logger.error(f"Request failed: {e}")

坑3：成本爆炸，Review一个PR花了30美元

现象：用了GPT-4，Review一个1000行的PR花了200K tokens，30美元。

原因：GPT-4太贵，而且每次都要把完整diff发过去。

第一次成本计算：

• 1000行diff ≈ 10K tokens
• 分块20块，每块5K tokens
• GPT-4 input 0.03/1K tokens，output 0.06/1K tokens
• 总成本：(20 * 5K * 0.03) + (20 * 1K * 0.06) = 3 + 1.2 =

但实际用了200K tokens，因为Agent反复尝试不同的Review策略。

解决：用GPT-3.5-turbo做初筛，GPT-4做深度Review。

# 先用GPT-3.5快速扫描 llm_fast = ChatOpenAI(model="gpt-3.5-turbo", temperature=0) quick_scan_prompt = ChatPromptTemplate.from_template( "Quick scan this PR and identify files with potential issues:\n\n{diff}\n\n" "Return ONLY the list of files that need detailed review." ) scan_chain = LLMChain(llm=llm_fast, prompt=quick_scan_prompt) files_to_review = scan_chain.run(diff=pr_diff) # 只对问题文件用GPT-4深度Review llm_deep = ChatOpenAI(model="gpt-4", temperature=0) for file in files_to_review: file_diff = extract_file_diff(pr_diff, file) review = deep_review_chain.run(diff=file_diff)

成本对比：

• 方案1：纯GPT-4，200K tokens，$30
• 方案2：GPT-3.5初筛 + GPT-4深度，20K tokens GPT-3.5 + 50K tokens GPT-4，0.06 + 1.5 =
• 节省：95%成本

最终方案：完整代码

import os from typing import List from github import Github from langchain_openai import ChatOpenAI from langchain.chains import LLMChain from langchain.prompts import ChatPromptTemplate from langchain.text_splitter import RecursiveCharacterTextSplitter from langchain.output_parsers import PydanticOutputParser from pydantic import BaseModel, Field # GitHub配置 GITHUB_TOKEN = os.getenv("GITHUB_TOKEN") github = Github(GITHUB_TOKEN) # LangChain配置 llm_fast = ChatOpenAI(model="gpt-3.5-turbo", temperature=0) llm_deep = ChatOpenAI(model="gpt-4", temperature=0) class CodeReviewIssue(BaseModel): line_number: int severity: str description: str suggestion: str class CodeReview(BaseModel): issues: List[CodeReviewIssue] parser = PydanticOutputParser(pydantic_object=CodeReview) def get_pr_diff(repo_name: str, pr_number: int) -> str: """获取PR的diff""" repo = github.get_repo(repo_name) pr = repo.get_pull(pr_number) return pr.get_files()[0].patch def quick_scan(diff: str) -> List[str]: """快速扫描，识别需要Review的文件""" prompt = ChatPromptTemplate.from_template( "Quick scan this PR and identify files with potential issues:\n\n{diff}\n\n" "Return ONLY the list of file paths, one per line." ) chain = LLMChain(llm=llm_fast, prompt=prompt) result = chain.run(diff=diff) return [f.strip() for f in result.split("\n") if f.strip()] def deep_review(diff: str) -> CodeReview: """深度Review""" splitter = RecursiveCharacterTextSplitter( chunk_size=4000, chunk_overlap=0, separators=["\n\n", "\n", " "] ) chunks = splitter.split_text(diff) all_issues = [] for chunk in chunks: prompt = ChatPromptTemplate.from_template( "Review this code:\n\n{diff}\n\n" "{format_instructions}\n\n" "Return ONLY the structured output." ) chain = LLMChain( llm=llm_deep, prompt=prompt, output_parser=parser ) review = chain.run( diff=chunk, format_instructions=parser.get_format_instructions() ) all_issues.extend(review.issues) return CodeReview(issues=all_issues) def post_review_comment(repo_name: str, pr_number: int, review: CodeReview): """在PR上发布Review评论""" repo = github.get_repo(repo_name) pr = repo.get_pull(pr_number) for issue in review.issues: pr.create_review_comment( body=f"**[{issue.severity.upper()}]** {issue.description}\n\n" f"Suggestion: {issue.suggestion}", commit=pr.get_commits().reversed[0], path="src/main/java/example/Example.java", position=issue.line_number ) def review_pr(repo_name: str, pr_number: int): """完整的Review流程""" diff = get_pr_diff(repo_name, pr_number) files_to_review = quick_scan(diff) for file in files_to_review: file_diff = extract_file_diff(diff, file) review = deep_review(file_diff) post_review_comment(repo_name, pr_number, review) # 使用示例 if __name__ == "__main__": review_pr("helloworldtang/mq-tutorial", 123)

GitHub项目

完整代码：https://github.com/helloworldtang/ai-code-review-agent

https://github.com/helloworldtang/mq-tutorial

一句话总结： 分块处理避免Token超限，结构化输出保证可执行，GPT-3.5+GPT-4组合降低成本。