FLAIR-XGT: Local visual patch feature aggregation with multi-graph fusion for image-based recommendation

Abstract

Existing multimodal recommender systems discard fine-grained spatial information before recommendation training even starts by representing each product image as a single global vector, either a pooled CNN output or a Vision Transformer’s CLS token. We propose FLAIR-XGT, an image-based recommendation framework that uses three parallel graph networks (LightGCN, VisualGCN, and TextGCN) to jointly train  representative local DINOv2 ViT-B/14 patch tokens (each covering a  pixel region) end-to-end under a combined BPR + InfoNCE objective. Our primary scoring component, GlobalVis, combines the local patch tokens using a projector that is optimized in tandem with the recommender. In comparison to all fixed pre-extracted global-feature baselines, this joint training of local patch aggregation with multi-graph collaborative fusion produces NDCG@20   on Amazon Fashion (  over MMGCN) and NDCG@20   on Sports & Outdoors (  over MMGCN). We also study XGT, a user-conditioned cross-attention module that focuses spatial attention on user-relevant patches. A Graph Transformer block with a CF residual connection fuses the attended output. One important empirical finding is that XGT cross-attention performs worse than GlobalVis on both datasets, which reflects a cold-start limitation: user embeddings are too sparse to form discriminative attention queries, making learned mean-pooling of local patches the stronger configuration in practice. Sensitivity analysis verifies resilience across the critical hyperparameters, and ablation studies link each performance gain to particular design decisions.

Conflict of Interest

The authors declare that there is no conflict of interest regarding the publication of this paper. The authors have no financial, professional, or personal relationships that could have influenced the research results or the interpretation of the findings.