图数据库这么流行的赛道说没就没了?
申明一下,绝对不是危言耸听啊,且听我严密分析。
以前图数据库还是个比较小众的数据库分支, 随着AI应用的发展, 图数据库正在崛起。
BUT 你以为图数据库赛道将大放异彩,那就大错特错了。
因为关系数据库都将开始内置图数据库的功能, 只因这个图相关SQL标准(SQL:2023 Property-Graph Queries).
不好意思,图SQL功能将大放异彩,但图赛道将不复存在。
为什么说AI应用推动了图的发展呢?
RAG是AI领域常见的应用之一, 用于将未经模型训练的知识、记忆整理存放, 在AI进行内容生成时, 通过向量搜索、模糊搜索、全文检索等手段召回与问题相关的内容, 提升AI的回复质量.
但传统RAG没有对内容进行深层次的整理, 召回效果较差(准确率低、覆盖率低或噪音明显), 最终导致AI回复效果差.
通过对内容重新整理, 可能是提升召回质量的路径之一, 所以有了GraphRAG, KAG等. 底层则需要图数据库的支持.
有兴趣可阅读相关论文如下( 本文链接文章全部来自 https://github.com/digoal/blog )
图查询的应用有哪些?
图数据库查询中众人皆知的请求类型包括: N度关系, 最短距离.
这两种请求直接使用递归语法就能实现. 有兴趣可以阅读如下文章(来自 https://github.com/digoal/blog )
结果集爆减、性能飙升、内存爆省》实际上图的查询远不止于此, 例如还有鲜为人知的: 图的PageRank, 图的局部聚类情况, 图的连通性等。
Page Rank 的应用场景
弱连通分量 (Weakly Connected Components, WCC) 的应用场景
局部聚类系数 (Local Clustering Coefficient, LCC) 的应用场景
有兴趣可阅读相关论文如下:
图数据库SQL到底怎么用?
PGQ到底怎么写?和普通SQL有何区别?
先别慌, 前面提到了一些关系数据库中就支持图SQL标准(SQL:2023 Property-Graph Queries)了.
例如 Oracle 23i, DuckDB, PostgreSQL.
虽然PostgreSQL之前有一款图数据库插件AGE停更了, 参考下文
但好消息是, 图数据库SQL语法标准可能要直接进入PG内核了, 参考下文
甚至于, 还有更牛x的. 以线性代数为基础, 将图映射到矩阵, 应用GPU实现并行图计算加速, 每秒处理数十亿条边. 参考下文
DuckDB 也推出了相关的PGQ插件, 并不是简单的递归,我这里放一张论文中的图吧,CSR技术很关键。

相关图论文如下。
可以这么说, 学PGQ, 用DuckDB最方便.
1、基表
每张图至少需要2类基表, 1类存储顶点数据(包括PK, 其他属性), 另1类存储顶点和顶点联通的路径. 在同一张图中, 顶点和路径都可以有很多张.
顶点需要PK. 除了PK, 也可以存储其他属性字段, 查询时可以作为过滤条件.
create table a (id int64 primary key, info text, ts timestamp);
create table b (id int64 primary key, info text, ts timestamp);
路径表也需要PK, 并且还需要2个外键, references顶点表的PK, 分别表达源和目标ID. 除了PK, 也可以存储其他属性字段, 查询时可以作为过滤条件.
create table c (id int64 primary key, ids int64 references a(id), idd int64 references b(id), weight float);
构图
CREATE PROPERTY GRAPH g1
VERTEX TABLES (
a,b
)
EDGE TABLES (
c
SOURCE KEY ( ids ) REFERENCES a ( id )
DESTINATION KEY ( idd ) REFERENCES b ( id )
LABEL knows
);
注意顶点可以只用1张表. 例如表达人与人的关系, 人只需存储1张表. 表达人与商品的关系时, 通常人定义1张表, 商品定义另1张表.
在DuckDB中可以创建多张图, 只要业务需要.
2、顶点
表达图中的点. 就像前面基表中提到的a.id, b.id.
3、边
表达图中的边, 就像前面基表中提到的c.id.
4、源
表达路径(某个点a 通过 某条边x 连接到 某个点b)中的 起点a.
5、目标
表达路径(某个点a 通过 某条边x 连接到 某个点b)中的 目标点b.
6、图
每一张图都包含了顶点、边的全部定义.
列出DuckDB中创建了哪些图?
PRAGMA show_property_graphs;
7、标签(别名)
在定义图时, 可以给顶点表、边表分别定义标签(LABEL), 未来在进行图查询时, 可以使用标签(LABEL)代替表名.
可能是为了方便开发者写SQL?
8、缩写
在写图查询SQL语句时, 还可以给顶点表、边表、甚至LABEL再定义缩写. 使用冒号(:)即可, 冒号左边是缩写, 冒号右边是原始名.
更加方便开发者写SQL?
或者当顶点为单张表时, 用来区分源和目标点?
例如
FROM GRAPH_TABLE (snb
MATCH p = ANY SHORTEST (a:Person WHERE a.firstName = 'Jan')-[k:knows]->+(b:Person)
COLUMNS (path_length(p), b.firstName) -- 这个查询返回什么字端?
)
ORDER BY firstName
LIMIT 5;
这里的
a:Person
k:knows
b:Person
a代表Person, 左边的顶点
k代表knows, 边
b代表Person, 右边的顶点
圆括号表示顶点
方括号表示边
()-[]->()
箭头表示联通方向, 可以不限制联通方向, 也可以限制单向或双向联通性
()-[]-()
()<-[]-()
()<-[]->()
如果要限制路径跳数(即经过了多少条边), 可以加在右边的顶点边上.
+ : Lower bound of 1, no upper bound.
* : Lower bound of 0, no upper bound.
{n, m}: Lower bound of n (where n > 0) and upper bound of m (where m ≥ n).
{,m}: Lower bound of 0, upper bound of m .
{n,} : Lower bound of n , no upper bound.
例如
()-[]->{1,}()
()-[]->+()
()-[]->*()
9、返回路径、顶点ID、边ID、距离
COLUMNS 内表示返回字端
FROM GRAPH_TABLE (snb
MATCH p = ANY SHORTEST (a:Person WHERE a.firstName = 'Jan')-[k:knows]->+(b:Person)
COLUMNS (element_id(p), vertices(p), edges(p), path_length(p), b.firstName)
)
ORDER BY firstName
LIMIT 5;
p可能是一个缩写, 表示 ANY SHORTEST (a:Person WHERE a.firstName = 'Jan')-[k:knows]->+(b:Person) 匹配到的结果
element_id(p) 表示路径
vertices(p) 表示顶点
edges(p) 表示边
path_length(p) 表示路径长度, 其实就是边的条数
b.firstName 在这里表示目标点的某个属性字段firstName的值
┌───────────────────────────┬────────────────┬─────────────┬────────────────┬─────────────┐
│ element_id(p) │ vertices(p) │ edges(p) │ path_length(p) │ firstName │
│ int64[] │ int64[] │ int64[] │ int64 │ varchar │
├───────────────────────────┼────────────────┼─────────────┼────────────────┼─────────────┤
│ [1, 3, 5, 22, 26, 66, 44] │ [1, 5, 26, 44] │ [3, 22, 66] │ 3 │ Abdul Haris │
│ [1, 5, 33, 79, 39] │ [1, 33, 39] │ [5, 79] │ 2 │ Aleksandr │
│ [1, 3, 5, 24, 32] │ [1, 5, 32] │ [3, 24] │ 2 │ Alexei │
│ [1, 3, 5, 21, 21] │ [1, 5, 21] │ [3, 21] │ 2 │ Ali │
│ [1, 3, 5] │ [1, 5] │ [3] │ 1 │ Ali │
└───────────────────────────┴────────────────┴─────────────┴────────────────┴─────────────┘
三个例子分别包括社交网络、航班数据、金融数据.
1、安装DuckDB
curl https://install.duckdb.org | sh
PS: 如果你的环境无法访问github, 恭喜你, 可参考下文:
假设socks5代理为127.0.0.1:11112, 可以这么安装DuckDB
https_proxy=socks5h://127.0.0.1:11112 curl https://install.duckdb.org | https_proxy=socks5h://127.0.0.1:11112 sh
2、安装PGQ插件, 这个插件来自DuckDB社区, 非核心repo, 需要指定repo.
https://duckdb.org/docs/stable/extensions/installing_extensions
$ duckdb
DuckDB v1.3.2 (Ossivalis) 0b83e5d2f6
Enter ".help" for usage hints.
Connected to a transient in-memory database.
Use ".open FILENAME" to reopen on a persistent database.
D
D INSTALL 'duckpgq' FROM 'http://community-extensions.duckdb.org';
100% ▕████████████████████████████████████████████████████████████▏
D LOAD duckpgq;
3、下面的例子参考duckpgq插件官网手册
https://duckpgq.org/documentation/graph_functions/
DuckDB可以直接通过http协议加载远程数据, 练习中的例子都是已经准备好的数据, 都在远端.
直接通过http协议加载即可测试, 非常方便.
为避免duckdb shell中无法访问github, 先下载数据文件
https_proxy=socks5h://127.0.0.1:11112 curl -L -o /Users/digoal/Downloads/snb.duckdb https://github.com/Dtenwolde/duckpgq-docs/raw/refs/heads/main/datasets/snb.duckdb
加载数据文件, 建图
ATTACH '/Users/digoal/Downloads/snb.duckdb';
use snb;
-- install duckpgq from community;
load duckpgq;
CREATE or replace PROPERTY GRAPH snb
VERTEX TABLES (
Person, Forum
)
EDGE TABLES (
Person_knows_person SOURCE KEY (Person1Id) REFERENCES Person (id)
DESTINATION KEY (Person2Id) REFERENCES Person (id)
LABEL knows,
Forum_hasMember_Person SOURCE KEY (ForumId) REFERENCES Forum (id)
DESTINATION KEY (PersonId) REFERENCES Person (id)
LABEL hasMember
);
Shortest Path Query
-- find the shortest path from one person to all other persons
FROM GRAPH_TABLE (snb
MATCH p = ANY SHORTEST (p1:person WHERE p1.id = 14)-[k:knows]->*(p2:person)
COLUMNS (p1.id, p2.id as other_person_id, element_id(p), path_length(p))
);
┌───────┬─────────────────┬────────────────────────────┬────────────────┐
│ id │ other_person_id │ element_id(p) │ path_length(p) │
│ int64 │ int64 │ int64[] │ int64 │
├───────┼─────────────────┼────────────────────────────┼────────────────┤
│ 14 │ 14 │ [0] │ 0 │
│ 14 │ 10995116277782 │ [0, 0, 13] │ 1 │
│ 14 │ 24189255811081 │ [0, 1, 26] │ 1 │
│ 14 │ 24189255811109 │ [0, 1, 26, 61, 27] │ 2 │
│ 14 │ 26388279066641 │ [0, 0, 13, 42, 29] │ 2 │
│ 14 │ 26388279066658 │ [0, 0, 13, 43, 31] │ 2 │
│ 14 │ 26388279066668 │ [0, 2, 32] │ 1 │
│ 14 │ 28587302322180 │ [0, 0, 13, 44, 33] │ 2 │
│ 14 │ 28587302322196 │ [0, 1, 26, 65, 35] │ 2 │
│ 14 │ 28587302322204 │ [0, 0, 13, 45, 36] │ 2 │
│ 14 │ 28587302322223 │ [0, 0, 13, 44, 33, 78, 38] │ 3 │
│ 14 │ 30786325577731 │ [0, 0, 13, 44, 33, 79, 39] │ 3 │
│ 14 │ 30786325577740 │ [0, 0, 13, 43, 31, 72, 40] │ 3 │
│ 14 │ 32985348833329 │ [0, 0, 13, 44, 33, 80, 43] │ 3 │
│ 14 │ 35184372088834 │ [0, 1, 26, 66, 44] │ 2 │
│ 14 │ 35184372088850 │ [0, 0, 13, 43, 31, 73, 45] │ 3 │
│ 14 │ 35184372088856 │ [0, 0, 13, 46, 46] │ 2 │
├───────┴─────────────────┴────────────────────────────┴────────────────┤
│ 17 rows 4 columns │
└───────────────────────────────────────────────────────────────────────┘
Find Mutual Friends
-- Find mutual friends between two users
FROM GRAPH_TABLE (snb
MATCH (p1:Person WHERE p1.id = 16)-[k:knows]->(p2:Person)<-[k2:knows]-(p3:Person WHERE p3.id = 32)
COLUMNS (p2.firstName)
);
┌───────────┐
│ firstName │
│ varchar │
├───────────┤
│ Ali │
└───────────┘
Most Popular People
-- Find the 3 most popular people
FROM GRAPH_TABLE (snb
MATCH (follower:Person)-[follows:knows]->(person:Person)
COLUMNS (person.id AS personID, person.firstname, person.lastname, follower.id AS followerID)
)
SELECT personID, firstname, lastname, COUNT(followerID) AS numFollowers
GROUP BY ALL
ORDER BY numFollowers DESC
LIMIT 3;
┌────────────────┬───────────┬──────────┬──────────────┐
│ personID │ firstName │ lastName │ numFollowers │
│ int64 │ varchar │ varchar │ int64 │
├────────────────┼───────────┼──────────┼──────────────┤
│ 24189255811081 │ Alim │ Guliyev │ 10 │
│ 28587302322180 │ Bryn │ Davies │ 9 │
│ 26388279066658 │ Roberto │ Diaz │ 9 │
└────────────────┴───────────┴──────────┴──────────────┘
Forum count of the most-followed person
-- Number of forums posted on by the most followed person
WITH
mfp AS (
FROM GRAPH_TABLE (snb
MATCH (follower:Person)-[follows:knows]->(person:Person)
COLUMNS (person.id AS personID, person.firstname, follower.id AS followerID)
)
SELECT personID, firstname, COUNT(followerID) AS numFollowers
GROUP BY ALL ORDER BY numFollowers DESC LIMIT 1
)
FROM
mfp,
GRAPH_TABLE (snb
MATCH (person:Person)<-[fhm:hasMember]-(f:Forum)
COLUMNS (person.id AS personID, f.id as forumId)
) mem
SELECT mfp.personID, mfp.firstname, mfp.numFollowers, count(mem.forumId) forumCount
WHERE mfp.personID = mem.personID
GROUP BY ALL;
┌────────────────┬───────────┬──────────────┬────────────┐
│ personID │ firstName │ numFollowers │ forumCount │
│ int64 │ varchar │ int64 │ int64 │
├────────────────┼───────────┼──────────────┼────────────┤
│ 24189255811081 │ Alim │ 10 │ 138 │
└────────────────┴───────────┴──────────────┴────────────┘
下载数据文件
https_proxy=socks5h://127.0.0.1:11112 curl -L -o /Users/digoal/Downloads/airline-data-small.duckdb https://github.com/Dtenwolde/duckpgq-docs/raw/refs/heads/airline-data/datasets/airline-data-small.duckdb
装载数据文件, 建图
ATTACH '/Users/digoal/Downloads/airline-data-small.duckdb' as airline;
use airline;
-- install duckpgq from community;
load duckpgq;
CREATE PROPERTY GRAPH flight_graph
VERTEX TABLES (
aircrafts_data, airports_data,
bookings, flights,
tickets, seats
)
EDGE TABLES (
route
SOURCE KEY (departure_airport) REFERENCES airports_data(airport_code)
DESTINATION KEY (arrival_airport) REFERENCES airports_data(airport_code),
ticket_flights
SOURCE KEY (ticket_no) REFERENCES tickets(ticket_no)
DESTINATION KEY (flight_id) REFERENCES flights(flight_id),
bookings_tickets
SOURCE KEY (book_ref) REFERENCES bookings(book_ref)
DESTINATION KEY (ticket_no) REFERENCES tickets(ticket_no),
boarding_passes
SOURCE KEY (ticket_no) REFERENCES tickets(ticket_no)
DESTINATION KEY (seat_no) REFERENCES seats(seat_no)
);
Shortest Route Between Airports
FROM (
SELECT unnest(flights) AS flights
FROM GRAPH_TABLE (
flight_graph
MATCH o = ANY SHORTEST (a:airports_data WHERE a.airport_code = 'UKX')
-[fr:route]->*
(a2:airports_data WHERE a2.airport_code = 'CNN')
COLUMNS (edges(o) AS flights)
)
)
JOIN route f
ON f.rowid = flights;
┌─────────┬───────────────────┬─────────────────┐
│ flights │ departure_airport │ arrival_airport │
│ int64 │ varchar │ varchar │
├─────────┼───────────────────┼─────────────────┤
│ 3 │ MJZ │ CNN │
│ 58 │ UKX │ KJA │
│ 313 │ SVO │ MJZ │
│ 493 │ KJA │ SVO │
└─────────┴───────────────────┴─────────────────┘
Most Expensive Seats on Average
FROM GRAPH_TABLE (
flight_graph
MATCH (b:bookings)-[bt:bookings_tickets]->(t:tickets)-[bp:boarding_passes]->(s:seats)
)
SELECT round(avg(total_amount), 2) avg_amount, seat_no
GROUP BY seat_no
ORDER BY avg_amount DESC;
┌────────────┬─────────┐
│ avg_amount │ seat_no │
│ double │ varchar │
├────────────┼─────────┤
│ 165002.34 │ 5H │
│ 162638.52 │ 3H │
│ 161094.62 │ 1H │
│ 159982.75 │ 2H │
│ 159357.03 │ 4H │
│ 157285.04 │ 5G │
│ 156692.37 │ 4G │
│ 153729.37 │ 1G │
│ 151677.06 │ 2G │
│ 150673.17 │ 3G │
│ 137574.63 │ 4K │
│ 131681.02 │ 24B │
│ 131023.19 │ 2F │
│ 130373.44 │ 1F │
│ 129315.14 │ 38D │
│ 129178.79 │ 3F │
│ 128531.07 │ 2K │
│ 127953.63 │ 15G │
│ 127886.73 │ 5K │
│ 127745.15 │ 3K │
│ · │ · │
│ · │ · │
│ · │ · │
│ 87545.89 │ 44H │
│ 87502.67 │ 7B │
│ 87414.22 │ 51D │
│ 87117.02 │ 40A │
│ 87025.12 │ 43A │
│ 86942.65 │ 47C │
│ 86872.91 │ 43F │
│ 86686.42 │ 4B │
│ 86658.08 │ 45F │
│ 86603.0 │ 39J │
│ 86315.35 │ 41D │
│ 86232.64 │ 4E │
│ 85916.28 │ 45A │
│ 85891.1 │ 22C │
│ 85413.07 │ 39G │
│ 85194.06 │ 36K │
│ 85015.46 │ 49H │
│ 85012.69 │ 6B │
│ 84873.21 │ 42F │
│ 83132.0 │ 26J │
├────────────┴─────────┤
│ 461 rows (40 shown) │
└──────────────────────┘
下载数据文件
https_proxy=socks5h://127.0.0.1:11112 curl -L -o /Users/digoal/Downloads/finbench.duckdb https://github.com/Dtenwolde/duckpgq-docs/raw/refs/heads/main/datasets/finbench.duckdb
装载数据文件, 建图
ATTACH '/Users/digoal/Downloads/finbench.duckdb' as finbench;
use finbench;
-- INSTALL duckpgq FROM community;
LOAD duckpgq;
CREATE OR REPLACE PROPERTY GRAPH finbench
VERTEX TABLES (
Account, Company, Loan,
Medium, Person
)
EDGE TABLES (
AccountRepayLoan SOURCE KEY (accountId) REFERENCES Account (accountId)
DESTINATION KEY (loanId) REFERENCES Loan (loanId)
LABEL repay,
AccountTransferAccount SOURCE KEY (fromId) REFERENCES Account (accountId)
DESTINATION KEY (toId) REFERENCES Account (AccountId)
LABEL transfer,
AccountWithdrawAccount SOURCE KEY (fromId) REFERENCES Account (accountId)
DESTINATION KEY (toId) REFERENCES Account (AccountId)
LABEL withdraw,
CompanyApplyLoan SOURCE KEY (companyId) REFERENCES Company (companyId)
DESTINATION KEY (loanId) REFERENCES Loan (loanId)
LABEL companyApply,
CompanyGuaranteeCompany SOURCE KEY (fromId) REFERENCES Company (companyId)
DESTINATION KEY (toId) REFERENCES Company (companyId)
LABEL companyGuarantee,
CompanyInvestCompany SOURCE KEY (investorId) REFERENCES Company (companyId)
DESTINATION KEY (companyId) REFERENCES Company (companyId)
LABEL companyInvest,
CompanyOwnAccount SOURCE KEY (companyId) REFERENCES Company (companyId)
DESTINATION KEY (accountId) REFERENCES Account (accountId)
LABEL companyOwn,
LoanDepositAccount SOURCE KEY (loanId) REFERENCES Loan (loanId)
DESTINATION KEY (accountId) REFERENCES Account (accountId)
LABEL deposit,
MediumSignInAccount SOURCE KEY (mediumId) REFERENCES Medium (mediumId)
DESTINATION KEY (accountId) REFERENCES Account (accountId)
LABEL signIn,
PersonApplyLoan SOURCE KEY (personId) REFERENCES Person (personId)
DESTINATION KEY (loanId) REFERENCES Loan (loanId)
LABEL personApply,
PersonGuaranteePerson SOURCE KEY (fromId) REFERENCES Person (personId)
DESTINATION KEY (toId) REFERENCES Person (personId)
LABEL personGuarantee,
PersonInvestCompany SOURCE KEY (investorId) REFERENCES Person (personId)
DESTINATION KEY (companyId) REFERENCES Company (companyId)
LABEL personInvest,
PersonOwnAccount SOURCE KEY (personId) REFERENCES Person (personId)
DESTINATION KEY (accountId) REFERENCES Account (accountId)
LABEL personOwn
);
Find blocked accounts via transfers
FROM GRAPH_TABLE (
finbench
MATCH (src:Account where src.accountId = 16607023625929101)
<-[e1:transfer]-(mid:Account)
-[e2:transfer]->(dst:Account where dst.isBlocked = true)
COLUMNS (src.accountId as src_id, dst.accountId as dst_id)
)
SELECT src_id, dst_id
WHERE src_Id <> dst_id;
┌───────────────────┬────────────────────┐
│ src_id │ dst_id │
│ int64 │ int64 │
├───────────────────┼────────────────────┤
│ 16607023625929101 │ 112027040730841418 │
│ 16607023625929101 │ 164662861375734142 │
│ 16607023625929101 │ 182395784908505830 │
│ 16607023625929101 │ 190840034209825196 │
│ 16607023625929101 │ 195343633837195608 │
│ 16607023625929101 │ 233061280716423788 │
│ 16607023625929101 │ 236720455413662450 │
│ 16607023625929101 │ 260645828434067974 │
│ 16607023625929101 │ 290200700988686944 │
│ 16607023625929101 │ 291045125918818679 │
│ 16607023625929101 │ 182395784908505830 │
│ 16607023625929101 │ 233061280716423788 │
│ 16607023625929101 │ 236720455413662450 │
│ 16607023625929101 │ 260645828434067974 │
│ 16607023625929101 │ 290200700988686944 │
│ 16607023625929101 │ 291045125918818679 │
│ 16607023625929101 │ 182395784908505830 │
│ 16607023625929101 │ 233061280716423788 │
│ 16607023625929101 │ 260645828434067974 │
│ 16607023625929101 │ 290200700988686944 │
│ 16607023625929101 │ 291045125918818679 │
│ 16607023625929101 │ 260645828434067974 │
│ 16607023625929101 │ 260645828434067974 │
│ 16607023625929101 │ 260645828434067974 │
│ 16607023625929101 │ 260645828434067974 │
├───────────────────┴────────────────────┤
│ 25 rows 2 columns │
└────────────────────────────────────────┘
Filter high-value transfers by time
FROM GRAPH_TABLE (
finbench
MATCH (src:Account)-[e:Transfer]->(dst:Account)
WHERE '2022-07-13 09:18:33.137' < e.createtime
AND e.createtime < '2022-09-03 02:31:47.812'
AND e.amount > 4829783
);
┌─────────────────────┬────────────────┬──────────────────────┬──────────────────────┬────────────────┬───┬─────────────┬─────────────────┬──────────────────┬────────────┐
│ accountId │ accountLevel │ accoutType │ createTime │ email │ … │ isBlocked_1 │ lastLoginTime_1 │ nickname_1 │ phonenum_1 │
│ int64 │ varchar │ varchar │ timestamp │ varchar │ │ boolean │ int64 │ varchar │ varchar │
├─────────────────────┼────────────────┼──────────────────────┼──────────────────────┼────────────────┼───┼─────────────┼─────────────────┼──────────────────┼────────────┤
│ 4871487421931323467 │ Silver level │ foreign currency │ 2022-07-13 16:39:2… │ gmail.com │ … │ false │ 1607559769318 │ Von Kwiecinski │ 093-2605 │
│ 4859946947886188028 │ Elite level │ trust account │ 2022-06-02 06:31:1… │ hotmail.com │ … │ false │ 1637915991894 │ Quincy Palevic │ 993-7856 │
│ 4874865121651852757 │ Silver level │ foreign currency │ 2022-07-25 06:32:1… │ yahoo.com │ … │ false │ 1707138596631 │ Randolph Gica │ 019-6516 │
│ 224617031415104267 │ Basic level │ debit card │ 2022-03-10 08:16:0… │ gmail.com │ … │ false │ 1821244840150 │ Mee Litehiser │ 634-7538 │
│ 4868391197187506662 │ Basic level │ merchant account │ 2022-07-02 16:29:3… │ gmail.com │ … │ false │ 1870998197095 │ Laverna Model │ 817-9166 │
│ 4865576447420401096 │ Platinum level │ custodial account │ 2022-06-22 14:50:4… │ yahoo.com │ … │ false │ 1645937738814 │ Paula Kreiman │ 058-9085 │
│ 260927303410778194 │ Basic level │ trust account │ 2022-07-17 03:12:4… │ gmail.com │ … │ false │ 1701567052278 │ Eve Pusateri │ 185-5297 │
│ 4877679871418958311 │ Silver level │ trust account │ 2022-08-04 06:58:3… │ gmx.com │ … │ false │ 1858743258786 │ Mimi Voelkel │ 392-1577 │
│ 4840525174493152518 │ Silver level │ escrow account │ 2022-03-25 10:44:1… │ gmail.com │ … │ false │ 1854023613134 │ Mohammad Creitz │ 074-0610 │
│ 4881057571139487097 │ Silver level │ debit card │ 2022-08-16 03:31:0… │ gmail.com │ … │ false │ 1816448033193 │ Leo Astudillo │ 064-0237 │
│ 4850376798678024353 │ Basic level │ brokerage account │ 2022-04-28 20:49:5… │ yahoo.com │ … │ false │ 1677236764209 │ Rogelio Thaut │ 713-2481 │
│ 4807311127241295552 │ Basic level │ retirement account │ 2021-11-27 09:11:2… │ gmx.com │ … │ false │ 1869571881089 │ Jimmie Poldrack │ 881-3626 │
│ 4803651952544055659 │ Gold level │ credit card │ 2021-11-14 08:28:0… │ gmail.com │ … │ false │ 1705349033849 │ Theresia Wight │ 788-0613 │
│ 4821947826030247975 │ Silver level │ prepaid card │ 2022-01-18 07:18:4… │ gmail.com │ … │ false │ 1779950672568 │ Crissy Rehmann │ 839-8629 │
│ 4852347123515000212 │ Basic level │ foreign currency │ 2022-05-06 05:30:5… │ gmx.com │ … │ false │ 1848972719586 │ Alesia Dragovich │ 205-0421 │
│ 4868391197187506662 │ Basic level │ merchant account │ 2022-07-02 16:29:3… │ gmail.com │ … │ false │ 1884801105807 │ Andre Craigwell │ 148-1591 │
│ 4835177149935649794 │ Basic level │ brokerage account │ 2022-03-06 07:45:3… │ zoho.com │ … │ false │ 1666407376558 │ Karie Naji │ 930-3471 │
│ 4865576447420401096 │ Platinum level │ custodial account │ 2022-06-22 14:50:4… │ yahoo.com │ … │ false │ 1684982774740 │ Stephan Esqueda │ 582-2215 │
│ 215609832160363230 │ Basic level │ merchant account │ 2022-02-06 16:22:5… │ gmail.com │ … │ false │ 1660006878289 │ Jacob Sheild │ 473-0001 │
│ 181269885001663159 │ Basic level │ debit card │ 2021-10-06 17:19:0… │ hotmail.com │ … │ false │ 1764480655226 │ Damian Warsing │ 787-9979 │
│ · │ · │ · │ · │ · │ · │ · │ · │ · │ · │
│ · │ · │ · │ · │ · │ · │ · │ · │ · │ · │
│ · │ · │ · │ · │ · │ · │ · │ · │ · │ · │
│ 99079191802151398 │ Basic level │ merchant account │ 2020-12-18 13:42:5… │ gmail.com │ … │ false │ 1796558405647 │ Mervin Scammon │ 811-6740 │
│ 41939771529888100 │ Platinum level │ certificate of dep… │ 2020-05-29 11:12:3… │ gmail.com │ … │ false │ 1796558405647 │ Mervin Scammon │ 811-6740 │
│ 165788761282576994 │ Basic level │ retirement account │ 2021-08-12 13:34:2… │ hotmail.com │ … │ false │ 1796558405647 │ Mervin Scammon │ 811-6740 │
│ 151152062493622619 │ Elite level │ escrow account │ 2021-06-22 02:48:3… │ gmail.com │ … │ false │ 1796558405647 │ Mervin Scammon │ 811-6740 │
│ 131448814123876944 │ Basic level │ trust account │ 2021-04-13 01:47:5… │ gmx.com │ … │ false │ 1796558405647 │ Mervin Scammon │ 811-6740 │
│ 229402106019184975 │ Basic level │ merchant account │ 2022-03-26 22:11:3… │ gmail.com │ … │ false │ 1796558405647 │ Mervin Scammon │ 811-6740 │
│ 237846355320504906 │ Basic level │ prepaid card │ 2022-04-26 07:31:2… │ hotmail.com │ … │ false │ 1796558405647 │ Mervin Scammon │ 811-6740 │
│ 261771728340910437 │ Basic level │ debit card │ 2022-07-20 05:06:1… │ gmx.com │ … │ false │ 1796558405647 │ Mervin Scammon │ 811-6740 │
│ 84161018036486476 │ Platinum level │ corporate account │ 2020-10-26 14:33:2… │ yemeni.cc │ … │ false │ 1796558405647 │ Mervin Scammon │ 811-6740 │
│ 99079191802151398 │ Basic level │ merchant account │ 2020-12-18 13:42:5… │ gmail.com │ … │ false │ 1796558405647 │ Mervin Scammon │ 811-6740 │
│ 256423703783408196 │ Basic level │ internet account │ 2022-06-30 20:58:4… │ gmx.com │ … │ false │ 1796558405647 │ Mervin Scammon │ 811-6740 │
│ 90353467524121172 │ Silver level │ escrow account │ 2020-11-17 12:01:1… │ hotmail.com │ … │ false │ 1796558405647 │ Mervin Scammon │ 811-6740 │
│ 224617031415104066 │ Silver level │ certificate of dep… │ 2022-03-10 06:36:0… │ gmail.com │ … │ false │ 1796558405647 │ Mervin Scammon │ 811-6740 │
│ 163536961468891568 │ Basic level │ merchant account │ 2021-08-05 05:40:2… │ programmer.net │ … │ false │ 1796558405647 │ Mervin Scammon │ 811-6740 │
│ 63894819713319622 │ Basic level │ foreign currency │ 2020-08-15 08:39:4… │ hotmail.com │ … │ false │ 1796558405647 │ Mervin Scammon │ 811-6740 │
│ 131448814123876944 │ Basic level │ trust account │ 2021-04-13 01:47:5… │ gmx.com │ … │ false │ 1796558405647 │ Mervin Scammon │ 811-6740 │
│ 233061280716423788 │ Platinum level │ brokerage account │ 2022-04-09 04:40:5… │ zoho.com │ … │ false │ 1796558405647 │ Mervin Scammon │ 811-6740 │
│ 172825635700343512 │ Basic level │ credit card │ 2021-09-07 10:17:5… │ gmx.com │ … │ false │ 1796558405647 │ Mervin Scammon │ 811-6740 │
│ 162129586585338190 │ Basic level │ prepaid card │ 2021-07-31 09:30:4… │ gmail.com │ … │ false │ 1796558405647 │ Mervin Scammon │ 811-6740 │
│ 172825635700343512 │ Basic level │ credit card │ 2021-09-07 10:17:5… │ gmx.com │ … │ false │ 1796558405647 │ Mervin Scammon │ 811-6740 │
├─────────────────────┴────────────────┴──────────────────────┴──────────────────────┴────────────────┴───┴─────────────┴─────────────────┴──────────────────┴────────────┤
│ 638 rows (40 shown) 28 columns (9 shown) │
└─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘
DuckDB提供了几个内置函数, 获取图局部聚集性、弱联通性、pagerank等.
建表, 构图
-- Create the Person table
CREATE TABLE Person AS
SELECT * FROM 'https://gist.githubusercontent.com/Dtenwolde/2b02aebbed3c9638a06fda8ee0088a36/raw/8c4dc551f7344b12eaff2d1438c9da08649d00ec/person-sf0.003.csv';
-- Create the Person_knows_person table
CREATE TABLE Person_knows_person AS
SELECT * FROM 'https://gist.githubusercontent.com/Dtenwolde/81c32c9002d4059c2c3073dbca155275/raw/8b440e810a48dcaa08c07086e493ec0e2ec6b3cb/person_knows_person-sf0.003.csv';
-- Create the property graph
CREATE PROPERTY GRAPH snb
VERTEX TABLES (
Person
)
EDGE TABLES (
Person_knows_person
SOURCE KEY (Person1Id) REFERENCES Person (id)
DESTINATION KEY (Person2Id) REFERENCES Person (id)
LABEL Knows
);
Local Clustering Coefficient
SELECT *
FROM local_clustering_coefficient(<property graph>, <vertex label>, <edge label>);
SELECT *
FROM local_clustering_coefficient(snb, Person, Knows);
┌────────────────┬──────────────────────────────┐
│ id │ local_clustering_coefficient │
│ int64 │ float │
├────────────────┼──────────────────────────────┤
│ 14 │ 0.33333334 │
│ 16 │ 0.5 │
│ 32 │ 0.8333333 │
│ 2199023255557 │ 0.6666667 │
│ 2199023255573 │ 1.0 │
│ 2199023255594 │ 0.1904762 │
│ 4398046511139 │ 0.0 │
│ 6597069766702 │ 0.0 │
│ 8796093022234 │ 0.0 │
│ 8796093022237 │ 0.6666667 │
│ 8796093022244 │ 0.0 │
│ 8796093022249 │ 0.4 │
│ 10995116277761 │ 0.3 │
│ 10995116277782 │ 0.23809524 │
│ 10995116277783 │ 0.0 │
│ 10995116277808 │ 0.0 │
│ 13194139533342 │ 1.0 │
│ 13194139533352 │ 0.31111112 │
│ 13194139533355 │ 0.2 │
│ 15393162788877 │ 0.0 │
│ · │ · │
│ · │ · │
│ · │ · │
│ 26388279066655 │ 0.33333334 │
│ 26388279066658 │ 0.21794872 │
│ 26388279066668 │ 0.5 │
│ 28587302322180 │ 0.16666667 │
│ 28587302322191 │ 0.0 │
│ 28587302322196 │ 0.8333333 │
│ 28587302322204 │ 0.2857143 │
│ 28587302322209 │ 0.0 │
│ 28587302322223 │ 0.0 │
│ 30786325577731 │ 0.0 │
│ 30786325577740 │ 1.0 │
│ 32985348833291 │ 0.0 │
│ 32985348833318 │ 0.0 │
│ 32985348833329 │ 0.0 │
│ 35184372088834 │ 0.0 │
│ 35184372088850 │ 0.6666667 │
│ 35184372088856 │ 0.33333334 │
│ 35184372088871 │ 0.0 │
│ 37383395344394 │ 0.0 │
│ 37383395344409 │ 0.0 │
├────────────────┴──────────────────────────────┤
│ 50 rows (40 shown) 2 columns │
└───────────────────────────────────────────────┘
Weakly Connected Component
SELECT *
FROM weakly_connected_component(<property graph>, <vertex label>, <edge label>);
SELECT *
FROM weakly_connected_component(snb, Person, Knows);
┌────────────────┬─────────────┐
│ id │ componentId │
│ int64 │ int64 │
├────────────────┼─────────────┤
│ 14 │ 39 │
│ 16 │ 39 │
│ 32 │ 39 │
│ 2199023255557 │ 39 │
│ 2199023255573 │ 39 │
│ 2199023255594 │ 39 │
│ 4398046511139 │ 6 │
│ 6597069766702 │ 39 │
│ 8796093022234 │ 8 │
│ 8796093022237 │ 39 │
│ 8796093022244 │ 39 │
│ 8796093022249 │ 39 │
│ 10995116277761 │ 39 │
│ 10995116277782 │ 39 │
│ 10995116277783 │ 14 │
│ 10995116277808 │ 15 │
│ 13194139533342 │ 39 │
│ 13194139533352 │ 39 │
│ 13194139533355 │ 39 │
│ 15393162788877 │ 39 │
│ · │ · │
│ · │ · │
│ · │ · │
│ 26388279066655 │ 39 │
│ 26388279066658 │ 39 │
│ 26388279066668 │ 39 │
│ 28587302322180 │ 39 │
│ 28587302322191 │ 39 │
│ 28587302322196 │ 39 │
│ 28587302322204 │ 39 │
│ 28587302322209 │ 37 │
│ 28587302322223 │ 39 │
│ 30786325577731 │ 39 │
│ 30786325577740 │ 39 │
│ 32985348833291 │ 41 │
│ 32985348833318 │ 42 │
│ 32985348833329 │ 39 │
│ 35184372088834 │ 39 │
│ 35184372088850 │ 39 │
│ 35184372088856 │ 39 │
│ 35184372088871 │ 47 │
│ 37383395344394 │ 48 │
│ 37383395344409 │ 49 │
├────────────────┴─────────────┤
│ 50 rows (40 shown) │
└──────────────────────────────┘
PageRank
SELECT *
FROM pagerank(<property graph>, <vertex label>, <edge label>);
┌────────────────┬──────────────────────┐
│ id │ pagerank │
│ int64 │ double │
├────────────────┼──────────────────────┤
│ 14 │ 0.010160948892729645 │
│ 16 │ 0.010160948892729645 │
│ 32 │ 0.010160948892729645 │
│ 2199023255557 │ 0.010160948892729645 │
│ 2199023255573 │ 0.010160948892729645 │
│ 2199023255594 │ 0.014479362809180427 │
│ 4398046511139 │ 0.010160948892729645 │
│ 6597069766702 │ 0.010160948892729645 │
│ 8796093022234 │ 0.010160948892729645 │
│ 8796093022237 │ 0.013039891503696832 │
│ 8796093022244 │ 0.011107680480004004 │
│ 8796093022249 │ 0.010160948892729645 │
│ 10995116277761 │ 0.011107680480004004 │
│ 10995116277782 │ 0.014767257070277144 │
│ 10995116277783 │ 0.010160948892729645 │
│ 10995116277808 │ 0.010160948892729645 │
│ 13194139533342 │ 0.011107680480004004 │
│ 13194139533352 │ 0.015627275199203507 │
│ 13194139533355 │ 0.028527222030624376 │
│ 15393162788877 │ 0.011107680480004004 │
│ · │ · │
│ · │ · │
│ · │ · │
│ 26388279066655 │ 0.014680543611929147 │
│ 26388279066658 │ 0.05141757730669825 │
│ 26388279066668 │ 0.02372779009907439 │
│ 28587302322180 │ 0.06681388098901132 │
│ 28587302322191 │ 0.010160948892729645 │
│ 28587302322196 │ 0.03367268951085164 │
│ 28587302322204 │ 0.05583904542516861 │
│ 28587302322209 │ 0.010160948892729645 │
│ 28587302322223 │ 0.024358861941305765 │
│ 30786325577731 │ 0.024358861941305765 │
│ 30786325577740 │ 0.022033917811836636 │
│ 32985348833291 │ 0.010160948892729645 │
│ 32985348833318 │ 0.010160948892729645 │
│ 32985348833329 │ 0.03648299553611766 │
│ 35184372088834 │ 0.019902115900832847 │
│ 35184372088850 │ 0.07139459174390198 │
│ 35184372088856 │ 0.03223445862173271 │
│ 35184372088871 │ 0.010160948892729645 │
│ 37383395344394 │ 0.010160948892729645 │
│ 37383395344409 │ 0.010160948892729645 │
├────────────────┴──────────────────────┤
│ 50 rows (40 shown) 2 columns │
└───────────────────────────────────────┘
https://github.com/szarnyasg/graph-data-management
https://duckpgq.org/documentation/sql_pgq/
https://duckpgq.org/
论文解读可关注后续文章。DuckPGQ真的非常强。