Research Presentation · 2026
Attributed Graph Clustering From Contrastive Learning to Foundation Models
Ph.D. research on structured data learning, from contrastive clustering to a one-for-all foundation model.
Yunhui Liu
3rd-year Ph.D. Student · Software Engineering
State Key Laboratory for Novel Software Technology · Nanjing University
- Advisors · Prof. Tieke He, Prof. Jianhua Zhao
- Internship · Research intern at Ant Group, mentored by Dr. Yongchao Liu
- Focus · Graph ML (clustering, anomaly detection, efficient inference) and knowledge-enhanced LLMs
Attributed Graph Clustering
Graph Foundation Model
Graph Anomaly Detection
LLM-Graph Synergy
10+
Publications
KDD · ICLR · WWW
Top venues
3.1M+
HF model downloads
111M+
Nodes supported
About
About Me
🎓 Education
Nanjing University · Ph.D. Student
Software Engineering · Sep 2024 – Present (Year 3)
Advisors: Prof. Tieke He, Prof. Jianhua Zhao
Nanjing University · B.Eng.
Software Engineering · Sep 2020 – Jun 2024
💼 Internship
Ant Group · Research Intern
Jul 2025 – Feb 2026 · Hangzhou
Graph/tabular learning · GraphRAG · Buy-Now-Pay-Later risk control
Mentor: Dr. Yongchao Liu
Mentor: Dr. Yongchao Liu
🏆 Honors
- China National Scholarship, 2025
- Jiangsu Postgraduate Research & Practice Innovation Program, 2026
- Insta360 Think Bold Scholarship (Grand Prize), 2026
🔬 Research Interests
- Structured graph learning: graph clustering, graph anomaly detection, link prediction
- Foundation models: graph foundation models, tabular foundation models
- Large language models: LLM + knowledge graphs, text-attributed graphs
- Industrial applications: fraud detection, user segmentation, GraphRAG
📦 Open Source
Hugging Face sentiment model · 3,108,129+ downloads
Heterogeneous text-attributed graph datasets · WWW 2025
✍️ Academic Service
Reviewer: NeurIPS 2026, KDD 2025/26/27, ICANN
Journal reviewer: PR, TKDE
TA: Big Data Analytics (2024), Data Management Fundamentals (2025)
Journal reviewer: PR, TKDE
TA: Big Data Analytics (2024), Data Management Fundamentals (2025)
Background
What Is a Graph?
Mathematical definition
$\mathcal{G} = (\mathcal{V}, \mathcal{E}, X)$
$\mathcal{V}$: node set (entities), $|\mathcal{V}| = N$
$\mathcal{E} \subseteq \mathcal{V} \times \mathcal{V}$: edge set (relations)
$X \in \mathbb{R}^{N \times F}$: node feature matrix
$A \in \{0,1\}^{N \times N}$: adjacency matrix
$\mathcal{V}$: node set (entities), $|\mathcal{V}| = N$
$\mathcal{E} \subseteq \mathcal{V} \times \mathcal{V}$: edge set (relations)
$X \in \mathbb{R}^{N \times F}$: node feature matrix
$A \in \{0,1\}^{N \times N}$: adjacency matrix
Topology
Connectivity among nodes (adjacency matrix $A$)
Node attributes
Feature vector of each node (feature matrix $X$)
🌍 Graphs in the real world
🧑🤝🧑
Social networks
nodes = users, edges = follows
🛒
E-commerce graphs
nodes = users/items, edges = purchases
📚
Citation networks
nodes = papers, edges = citations
🧬
Molecular graphs
nodes = atoms, edges = bonds
Why graph neural networks (GNNs)?
Conventional deep learning handles sequences or grids (text/images),
while graphs have irregular topology. GNNs aggregate neighbor information via message passing to learn node representations on the graph:
Conventional deep learning handles sequences or grids (text/images),
while graphs have irregular topology. GNNs aggregate neighbor information via message passing to learn node representations on the graph:
$h_v^{(l+1)} = \text{UPDATE}\!\left(h_v^{(l)},\; \text{AGG}\!\left(\{h_u^{(l)} : u \in \mathcal{N}(v)\}\right)\right)$
Interactive demo: a real graph structure (drag nodes)
Citation network sketch · nodes = papers · directed edges = citations
|V| = 16
|E| = 28
2014–16
2017–19
2020+
size ∝ citation count
Drag to pin · click a pinned node to unpin
Core Task
What Is Graph Clustering?
Task definition
Partition nodes into semantically similar groups (clusters),
using both node attributes and topology,
without label supervision.
Given $\mathcal{G}=(A, X)$, find an assignment
$f: \mathcal{V} \to \{1, 2, \ldots, K\}$
such that intra-cluster similarity is high and inter-cluster similarity is low
$f: \mathcal{V} \to \{1, 2, \ldots, K\}$
such that intra-cluster similarity is high and inter-cluster similarity is low
📊 Evaluation metrics
NMI
Normalized Mutual Information
ACC
Clustering accuracy
ARI
Adjusted Rand Index
F1
F1 score
Visualization: clustering paints nodes into distinct groups
Dense intra-cluster links · sparse inter-cluster links · unsupervised
Cluster 1
Cluster 2
Cluster 3
Cluster 4
Hover to highlight a cluster
🌐 Applications
- Community detection: user groups in social networks
- Fraud detection: identifying anomalous subgraphs
- User segmentation: personalized e-commerce recommendations
- Knowledge integration: mining entity relations
⚠️ Key challenges
- Unsupervised: no label guidance
- Attribute–structure conflict: homophily vs. heterophily
- Scalability: million-scale graphs
- Model selection: no validation set for tuning
📈 Landscape
- GNN + contrastive learning (e.g., NS4GC)
- Graph autoencoders (e.g., DAEGC)
- Spectral methods (e.g., SASE)
- Foundation models (OFAG) ← our work
Papers
Selected Publications
Graph Clustering
Under Review @ KDD 2027
Yunhui Liu, Xudong Jin, Kang Zhang, et al.
Foundation ModelZero-shot
arXiv 2026 (Survey)
Yunhui Liu, Yue Liu, Yongchao Liu, et al.
Survey
arXiv 2026 (Benchmark)
Yunhui Liu, Pengyu Qiu, Yu Xing, Yongchao Liu, et al.
Benchmark
TKDE 2024
Yunhui Liu, Xinyi Gao, Tieke He, et al.
Contrastive
CIKM 2024
Yunhui Liu, Tieke He, Qing Wu, et al.
Scalable
Under Review @ NeurIPS 2026
Uncovering Semantic Hierarchies in Text-Attributed Graphs via Variational EM-based LLM–GNN Synergy
SHiFTYunhui Liu, Xudong Jin, Qizhuo Xie, et al.
HierarchicalLLM+GNN
Graph Anomaly Detection
ICLR 2026
Yunhui Liu, Jiashun Cheng, et al.
WWW 2026
Yunhui Liu, Tieke He, Yongchao Liu, et al.
WWW 2026
Yunhui Liu, Qizhuo Xie, et al.
Other Directions
Journey
Research Trajectory in Graph Clustering
1
Contrastive
NS4GC · TKDE 2024
Contrastive clustering guided by a node similarity matrix
→
2
Scalability
SASE · CIKM 2024
Linear-complexity spectral embedding for large graphs
→
3
Hierarchy
SHiFT · UR @ NeurIPS 2026
LLM–GNN synergy for semantic hierarchies
→
4
Industry benchmark
PyAGC · arXiv 2026
Unified framework of 20+ methods, tested on industrial graphs
→
5
Foundation model ⭐
OFAG · UR @ KDD 2027
One frozen model, zero-shot clustering on any graph
Key insight from this trajectory:
Existing methods have a fundamental limitation — every new graph needs retraining and retuning.
In unlabeled clustering, there is not even a validation set to guide model selection.
Can we train one general model and apply it to any attributed graph? → OFAG
Existing methods have a fundamental limitation — every new graph needs retraining and retuning.
In unlabeled clustering, there is not even a validation set to guide model selection.
Can we train one general model and apply it to any attributed graph? → OFAG
NS4GC idea
Contrastive learning from a node-similarity-matrix view: the ideal matrix is an enhanced adjacency that keeps high-similarity intra-cluster edges.
Best mean and average rank on 8 datasets.
SASE idea
Parameter-free spectral embedding: $k$-order graph convolution + random Fourier features + adaptive order.
Linear complexity; ArXiv ACC +6.9%, 5.87× speedup.
SHiFT idea
An EM framework pairing an LLM (taxonomy expert) with a GNN: E-step induces a semantic tree; M-step aligns the hierarchy.
Outputs hierarchy-aware embeddings and a human-readable taxonomy.
PyAGC contribution
Encode–Cluster–Optimize unified framework, first mini-batch implementation,
2.7K–111M nodes, industrial graphs. Validated at Ant Group.
OFAG
Motivation: A Fundamental Limitation
⚠️ Pain points of the “one graph, one model” paradigm
💸
High compute cost
Every new graph needs training from scratch and retuning
🎯
No labeled validation set
Unsupervised clustering cannot support hyperparameter selection
📐
Incompatible feature spaces
Feature dimensions and semantics differ across graphs
Three core challenges
(i) Cluster-label permutation variability
$K$ varies across graphs, and cluster identities have no consistent cross-graph meaning — a shared output head is infeasible
(ii) No supervision at inference
No labels, validation set, or few-shot context at test time — domain adaptation is unavailable
(iii) Unsupervised feature-space alignment
Supervised GFMs solve this via labeled fine-tuning; clustering cannot use downstream supervision
Our question:
Can we train a frozen single model that, without graph-specific training,
fine-tuning, or hyperparameter search, clusters any attributed graph?
Can we train a frozen single model that, without graph-specific training,
fine-tuning, or hyperparameter search, clusters any attributed graph?
Limits of existing GFMs
- Text-space GFMs (OFA, GraphGPT): require textual node attributes and dataset-specific templates
- Feature-alignment GFMs (GCOPE, UniGraph): require target-graph fine-tuning
- Fully inductive GFMs (GraphAny, NodePFN): need a labeled context set at inference — clustering cannot provide this
OFAG’s core idea
Following the Prior-data Fitted Networks (PFN) paradigm,
we learn a reusable clustering-inference strategy on synthetic graphs,
then produce cluster-friendly embeddings with a single forward pass at inference.
Pre-train once on $p(\mathcal{D})$ → Freeze → Apply anywhere
OFAG
OFAG Framework
📦
CAGM Prior
Synthetic attributed-graph generator
covering diverse graph structures
covering diverse graph structures
→
🔄
SwiFRT Encoder
Dimension-agnostic graph encoder
each feature channel processed independently
each feature channel processed independently
→
🎯
Hyperspherical Obj.
Hyperspherical clustering objective
driven by a vMF mixture
driven by a vMF mixture
→
⚡
Zero-Shot Inference
A single forward pass
+ k-means clustering
+ k-means clustering
Training (pre-training)
Optimize on synthetic attributed graphs $(A,X,y) \sim p(\mathcal{D})$:
$\theta^* = \arg\min_\theta\; \mathbb{E}_{(A,X,y)\sim p(\mathcal{D})}\bigl[\mathcal{L}\bigl(y,\, f_\theta(A,X)\bigr)\bigr]$
$y$ is a synthetic label (visible only at training). The model learns a reusable representation geometry, not a dataset-specific solution.
Inference (zero-shot)
$Z = \operatorname{SwiFRT}_{\theta^*}(X, A)$
$z_i = Z_{i,:}/\|Z_{i,:}\|_2$
→ $k\text{-means}(Z)$
$z_i = Z_{i,:}/\|Z_{i,:}\|_2$
→ $k\text{-means}(Z)$
All parameters are frozen — no gradient updates, graph-specific training, or validation labels.
Relation to PFNs (theory)
Prior-data Fitted Networks learn on synthetic data sampled from a prior,
amortizing Bayesian inference — no parameter updates at test time.
Minimizing $\mathbb{E}[\mathcal{L}_\text{comp}]$ ≡ minimizing the empty-context vMF-mixture PFN
Prior-Data Negative Log-Likelihood
Prior-Data Negative Log-Likelihood
What we learn: representation geometry
Same-cluster nodes gather around one prototype · prototypes spread uniformly on the hypersphere
Node $z_i$
Prototype $\hat{\mu}_k$
Dashed = uniform prototype spread
Auto-rotate · hover to inspect a cluster
OFAG
CAGM: A Synthetic Attributed-Graph Prior
Core idea: One-for-all generalization depends entirely on the breadth of the prior $p(\mathcal{D})$.
A too-narrow prior lets the model learn shortcuts that fail on out-of-distribution graphs.
CAGM is a latent-variable generative model that treats cluster membership as the common cause of node attributes and graph structure.
Generative process
Sample sizes: $N, F, K$ are drawn from wide distributions;
cluster proportions $\pi \sim \text{Dirichlet}(\eta \mathbf{1}_K)$ (including highly imbalanced cases)
cluster proportions $\pi \sim \text{Dirichlet}(\eta \mathbf{1}_K)$ (including highly imbalanced cases)
Node-attribute generation:
$h_i^X \mid y_i=k \sim t_\nu(\mu_k, \Sigma_k)$ (cluster-conditional latent)
→ a random transform $T$ decodes continuous / categorical / count / bag-of-words features
$h_i^X \mid y_i=k \sim t_\nu(\mu_k, \Sigma_k)$ (cluster-conditional latent)
→ a random transform $T$ decodes continuous / categorical / count / bag-of-words features
Edge generation:
$A_{ij} \sim \text{Bernoulli}(\sigma(s_{ij}))$
$s_{ij} = \rho + \alpha(\theta_i+\theta_j) + \lambda_B B_{y_i,y_j} + \lambda_H\kappa(h_i^A,h_j^A) + \lambda_X\text{sim}(x_i,x_j)$
$A_{ij} \sim \text{Bernoulli}(\sigma(s_{ij}))$
$s_{ij} = \rho + \alpha(\theta_i+\theta_j) + \lambda_B B_{y_i,y_j} + \lambda_H\kappa(h_i^A,h_j^A) + \lambda_X\text{sim}(x_i,x_j)$
Homophily control:
$h \sim \text{Uniform}(0.1, 0.9)$, smoothly interpolating homophilic → heterophilic structure
$h \sim \text{Uniform}(0.1, 0.9)$, smoothly interpolating homophilic → heterophilic structure
Design principles
- If we only generate homophilic graphs → the model collapses to a feature-smoothing heuristic
- If attributes are always separable → the model ignores topology
- Independently randomize attribute separability, edge–cluster coupling, and structural patterns,
forcing the model to learn an adaptive strategy that weights attribute and structural evidence by reliability
Theorem: synthetic-to-real generalization
Theorem (Synthetic-to-Real Generalization):
Suppose $p_\text{real}(\mathcal{G})$ satisfies:
(i) the conditional feature distribution $p(X|y)$ is a finite mixture of continuous distributions;
(ii) the edge distribution $p(A|y,X)$ follows a pairwise probability model.
Then there exist a graph encoder $\Phi$ and a CAGM prior $p_\text{syn}$ such that $\Phi_\# p_\text{syn}(\mathcal{G})$ approximates $p_\text{real}(\mathcal{G})$ to arbitrarily small total variation distance.
Suppose $p_\text{real}(\mathcal{G})$ satisfies:
(i) the conditional feature distribution $p(X|y)$ is a finite mixture of continuous distributions;
(ii) the edge distribution $p(A|y,X)$ follows a pairwise probability model.
Then there exist a graph encoder $\Phi$ and a CAGM prior $p_\text{syn}$ such that $\Phi_\# p_\text{syn}(\mathcal{G})$ approximates $p_\text{real}(\mathcal{G})$ to arbitrarily small total variation distance.
Diversity of graphs generated by CAGM
Low/mid: interleaved layout to show edges; high homophily: same-cluster nodes gather
Cluster A
Cluster B
Cluster C
Colored edges = intra-cluster · gray = inter-cluster · lower-right $A$ is cluster-sorted
OFAG
SwiFRT: A Dimension-Agnostic Graph Encoder
Core problem: a dimension-specific transform $XW$ ($W \in \mathbb{R}^{F \times d}$)
is tied to a fixed feature dimension $F$ and cannot transfer zero-shot across graphs.
SwiFRT treats each feature channel $X_{:,f}$ as a scalar signal on the graph, encoding its structural response under a bank of graph filters rather than the raw feature values.
SwiFRT treats each feature channel $X_{:,f}$ as a scalar signal on the graph, encoding its structural response under a bank of graph filters rather than the raw feature values.
Step 1: Filter-response profile
Let $\hat{A} = \tilde{D}^{-1/2}(A+I)\tilde{D}^{-1/2}$ be the symmetrically normalized adjacency.
For node $i$ and channel $f$, collect the signal values at each diffusion order:
$r_{i,f} = \bigl[(\hat{A}^0 X)_{i,f},\; (\hat{A}^1 X)_{i,f},\; \ldots,\; (\hat{A}^L X)_{i,f}\bigr] \in \mathbb{R}^{L+1}$
The profile captures whether the signal is locally stable, smoothed by neighbors, or amplified/attenuated across hops —
independent of semantic content and transferable across graphs.
Step 2: Transformer encoding
Embed each order’s response as a $d$-dimensional token:
$e_{i,f,\ell} = \phi_\text{val}(r_{i,f,\ell}) + p_\ell$
$h_{i,f} = \text{Transformer}([c_\text{cls}, e_{i,f,0}, \ldots, e_{i,f,L}])_{[0]}$
$Z_{i,f} = \phi_\text{out}(h_{i,f})$
$h_{i,f} = \text{Transformer}([c_\text{cls}, e_{i,f,0}, \ldots, e_{i,f,L}])_{[0]}$
$Z_{i,f} = \phi_\text{out}(h_{i,f})$
Self-attention is applied only among filter-order tokens of the same $(i,f)$ pair,
learning interaction patterns across diffusion orders.
Step 3: Node embedding
$z_i = Z_{i,:} / \|Z_{i,:}\|_2 \in S^{F-1}$
SwiFRT signal-response sketch
Same node $i$: filter profiles $r_{i,f}\in\mathbb{R}^{L+1}$ of three channels $f$
Smooth diffusion
Local oscillation
Cross-hop amplification
Hover to compare a single channel
Proposition: dimension-agnostic & feature-permutation equivariant
Let $\sigma$ be any permutation of feature indices and $X_\sigma$ the correspondingly permuted feature matrix. SwiFRT satisfies:
(i) Dimension-agnostic: the number of trainable parameters is independent of $F$;
(ii) Permutation-equivariant: $\operatorname{SwiFRT}(X_\sigma, A) = \operatorname{SwiFRT}(X, A)_\sigma$
(i) Dimension-agnostic: the number of trainable parameters is independent of $F$;
(ii) Permutation-equivariant: $\operatorname{SwiFRT}(X_\sigma, A) = \operatorname{SwiFRT}(X, A)_\sigma$
Why are structural responses transferable?
- A smoothly diffused signal ≠ a locally oscillating one — independent of semantic content
- The same response pattern has similar clustering meaning across graphs (even domains)
- Channel-independent processing → any $F$ can share the same parameters
OFAG
Hyperspherical Clustering Objective
Prototype computation (episode-specific)
For each synthetic graph, cluster prototypes are computed on the fly from current embeddings, supporting variable $K$:
$\hat{\mu}_k = \dfrac{\sum_{i:\,y_i=k} z_i}{\bigl\|\sum_{i:\,y_i=k} z_i\bigr\|_2 + \varepsilon} \in S^{F-1}$
Computed per episode: no global prototype parameters and no fixed cluster-identity structure.
Compactness loss $\mathcal{L}_\text{comp}$ (vMF MLE)
$q_\theta(y_i{=}k \mid z_i) = \dfrac{\exp(\hat{\mu}_k^\top z_i/\tau)}{\sum_{k'} \exp(\hat{\mu}_{k'}^\top z_i/\tau)}$
$\mathcal{L}_\text{comp} = -\dfrac{1}{N}\sum_{i=1}^N \log\dfrac{\exp(\hat{\mu}_{y_i}^\top z_i/\tau)}{\sum_{k=1}^K \exp(\hat{\mu}_k^\top z_i/\tau)}$
$\mathcal{L}_\text{comp} = -\dfrac{1}{N}\sum_{i=1}^N \log\dfrac{\exp(\hat{\mu}_{y_i}^\top z_i/\tau)}{\sum_{k=1}^K \exp(\hat{\mu}_k^\top z_i/\tau)}$
Pull node representations toward their cluster prototype and push them away from others.
Equivalent to MLE of a $K$-component vMF mixture with concentration $\kappa=1/\tau$.
Dispersion loss $\mathcal{L}_\text{disp}$
$\mathcal{L}_\text{disp} = \dfrac{1}{K}\sum_{k=1}^K \log\dfrac{1}{K-1}\sum_{j\neq k} \exp\!\bigl(\hat{\mu}_k^\top\hat{\mu}_j/\tau\bigr)$
Minimize cosine similarity among distinct prototypes so they spread uniformly on the hypersphere.
The optimum approaches an equiangular tight frame (ETF) — maximizing angular separation among clusters.
Clustering geometry on the hypersphere
Nodes gather at prototypes $\mathcal{L}_\text{comp}$ · prototypes spread uniformly $\mathcal{L}_\text{disp}$
Node $z_i$
Prototype $\hat{\mu}_k$
Dashed = angular separation among prototypes
Auto-rotate · hover to inspect a cluster
Overall training objective
$\mathcal{L}(\theta) = \mathbb{E}_{(A,X,y)\sim p(\mathcal{D})}\bigl[\mathcal{L}_\text{comp} + \lambda\,\mathcal{L}_\text{disp}\bigr]$
Monte Carlo approximation: sample a batch of synthetic graphs → encode → compute on-the-fly prototypes → backprop
Complementarity
$\mathcal{L}_\text{comp}$
Nodes gather at their cluster prototype and stay away from others (intra-cluster compactness)
Nodes gather at their cluster prototype and stay away from others (intra-cluster compactness)
$\mathcal{L}_\text{disp}$
Prototypes spread uniformly on the hypersphere (inter-cluster separation)
Prototypes spread uniformly on the hypersphere (inter-cluster separation)
Results
OFAG: Leading in Quality and Efficiency
1st
Average rank
NMI / ACC / ARI / F1
NMI / ACC / ARI / F1
39.49
Mean NMI (10 datasets)
highest mean
highest mean
12.43min
Total runtime on 10 datasets
fastest method
fastest method
28×
Speedup vs. the next-best-quality
method in runtime
method in runtime
Table: NMI (↑) comparison on homophilic and heterophilic graphs (representative methods)
| Method | Cora | Photo | ArXiv | Products | HM (het.) | Flickr (het.) | MAG | Mean | Avg.Rank | |
|---|---|---|---|---|---|---|---|---|---|---|
| KMeans | 13.89 | 31.70 | 22.59 | 11.08 | 29.24 | 10.17 | 1.21 | 28.32 | 15.21 | 16.30 |
| S3GC | 55.45 | 68.27 | 47.11 | 83.45 | 53.43 | 11.57 | 7.84 | 39.80 | 38.08 | 5.10 |
| MAGI | 58.94 | 68.65 | 46.53 | 72.53 | 44.58 | 11.24 | 6.31 | 41.34 | 36.82 | 5.75 |
| NS4GC | 59.40 | 72.62 | 48.39 | 56.71 | 54.63 | 15.28 | 6.19 | 41.64 | 37.21 | 3.40 |
| OFAG ⭐ | 57.06 | 71.45 | 45.08 | 85.94 | 52.77 | 11.61 | 9.27 | 41.78 | 39.49 | 3.05 |
⚡ Efficiency (total runtime)
💡 Key takeaways
- A single frozen model achieves the best mean performance and average rank on all 10 datasets
- OFAG finishes all 10 datasets in 12.43 minutes; the second-fastest method takes 76+ minutes
- Existing methods still need blind per-graph tuning, so the practical time saving is an order of magnitude
- Pre-train once, freeze forever, and apply directly to any new graph
Results
Why Pre-training Matters: Prior Breadth + Dynamics
Take-home: one-for-all performance is capped by the coverage of the prior $p(\mathcal{D})$; a transferable clustering strategy is learned rapidly during pre-training.
One run covers 80,000 synthetic graphs (10,000 steps, ~1.5 h / A800), then the model is frozen forever and applied zero-shot from the lowest-validation-loss checkpoint.
Prior ablation: homophily coverage $h$
Restrict the sampled $h$ range at pre-training → zero-shot All NMI (mean over 10 graphs)
Full coverage (default)
Restricted prior
Heterophily-only hurts most: All NMI −7.12
Component ablation (All NMI / ACC)
| Variant | Homo | Hetero | All NMI | Δ |
|---|---|---|---|---|
| OFAG (full) | 62.46 | 16.51 | 39.49 | — |
| Continuous features only | 62.02 | 16.12 | 39.07 | −0.42 |
| Homophilous only $h\in[0.7,0.9]$ | 61.46 | 15.65 | 38.56 | −0.93 |
| $g_\theta$: Transformer → MLP | 60.24 | 13.80 | 37.02 | −2.47 |
| w/o $\mathcal{L}_\text{disp}$ | 61.40 | 14.50 | 37.95 | −1.54 |
| w/o $\ell_2$ normalization | 54.72 | 14.10 | 34.41 | −5.08 |
Pre-training dynamics: checkpoint NMI on real graphs
Measured checkpoints: init → step 50 → every 500 steps; steep rise in the first ~1,000 steps, then a plateau
Homophilous
Heterophilous
All 10 graphs
Measured checkpoint NMI (init / step 50 / every 500 steps)
Filter order $L$
SwiFRT multi-scale context; $L{=}2$ is clearly insufficient
| $L$ | 2 | 3 | 5 | 8 | 10 | 12 |
|---|---|---|---|---|---|---|
| All NMI | 37.22 | 38.75 | 39.06 | 38.91 | 39.49 | 39.16 |
Dispersion weight $\lambda$
$\lambda{=}0$ is worst; stable as long as $\lambda{>}0$
| $\lambda$ | 0 | 0.1 | 0.3 | 0.5 | 1.0 |
|---|---|---|---|---|---|
| All NMI | 37.95 | 39.49 | 38.81 | 39.38 | 39.28 |
The prior must be broad
Full-spectrum $h\in[0.1,0.9]$ dominates every split. Homophily-only over-smooths; heterophily-only routes information across clusters, dropping homophilous NMI by −10.84.
The strategy is learned fast
Homo 51.0→62.7, Hetero 12.0→16.5, All 31.5→39.6: a large jump by step 50, then a low-amplitude plateau.
Geometry and the objective matter
Removing $\ell_2$ normalization hurts most (All NMI −5.08); $\mathcal{L}_\text{disp}$ is especially important on heterophilous graphs (Hetero NMI $+2.01$).
Summary
Summary and Outlook
📌 Research contributions
- NS4GC (TKDE 2024): contrastive graph clustering via a node similarity matrix; SOTA
- SASE (CIKM 2024): linear-complexity, parameter-free spectral embedding for AGC
- SHiFT (arXiv 2026): LLM–GNN hierarchical clustering in an EM paradigm
- PyAGC (arXiv 2026): industrial AGC benchmark and unified framework
- OFAG (KDD 2027): first foundation model for AGC with zero-shot transfer
- Survey (arXiv 2026): systematic survey with an Encode–Cluster–Optimize taxonomy
🔬 OFAG’s core innovations
- CAGM prior: coupled attributed-graph generation covering diverse structures
- SwiFRT encoder: signal-response space for dimension-agnostic zero-shot transfer
- Hyperspherical objective: vMF-mixture MLE with ETF-optimal prototype geometry
- Theory: PFN framing and a synthetic-to-real generalization theorem
🚀 Future directions
- Automatic $K$: cluster-number estimation without prior knowledge
- Larger-scale training: pre-training on billion-node synthetic graphs
- Hierarchical clustering: layered semantics with taxonomies
- Anomaly-aware clustering: integrating GAD (APF, TFM4GAD)
- LLM enhancement: injecting semantic priors for text-attributed graphs
💼 Industrial potential
- Zero-shot user segmentation: no retraining per graph
- Fraud-ring discovery: clustering on heterophilic graphs
- GraphRAG: community mining on graphs to assist RAG
- Already validated on Ant Group’s Buy-Now-Pay-Later risk-control setting
Take-home claim: a “pre-train once, apply anywhere” paradigm for graph clustering
is not only conceptually compelling, but in practice achieves both the highest accuracy and the fastest speed.
Yunhui Liu
Ph.D. Student · Nanjing University