Architecture and Objective Design for Robust Recommendation under Sparse Implicit Feedback

Abstract:

Recommender systems (RS) are central to many online services, yet they remain challenged by implicit-feedback sparsity, cold-start users and items, multimodal noise, and the need to capture both short- and long-term preference dynamics. This thesis develops a unified view of multimodal, sequential, and optimisation advances for RS, contributing two model architectures and a training objective that jointly improve accuracy, robustness, and behaviour under distribution shift.

First, we propose a novel recommender framework, MuSTRec (Multimodal and Sequential Transformer-based Recommendation), that unifies multimodal and sequential recommendation paradigms. MuSTRec captures cross-item similarities and collaborative filtering signals by building item–item graphs from extracted text and visual features, while a frequency-based self-attention module additionally captures short- and long-term user preferences. Across multiple Amazon datasets, MuSTRec demonstrates superior performance, with improvements of up to 33.5% over multimodal and sequential state-of-the-art baselines. We further highlight practical facets of this paradigm, including the need for a new data-partitioning regime, and show that integrating user embeddings into sequential recommendation can drastically increase short-term metrics, with improvements of up to 200% on smaller datasets.

Second, to address the limitations of augmentation-heavy training—where multimodal, sequential, and contrastive techniques can add noise and disrupt useful semantics—we propose MuSICRec (Multimodal Sequence–Item Contrastive Recommender), a multi-view graph-based recommender that combines collaborative, sequential, and multimodal signals. MuSICRec builds a sequence–item view by attention pooling over users’ interacted items to form sequence nodes and propagates over the sequence–item graph to obtain a second view organically, as an alternative to artificial data augmentation, while simultaneously injecting sequential context signals. To mitigate modality noise and align multimodal information, the contribution of text and visual features is modulated using an ID-guided gate. Evaluated under a strict leave-two-out split against a broad range of sequential, multimodal, and contrastive baselines on Amazon Baby, Sports, and Electronics, MuSICRec outperforms state-of-the-art methods across model types, with the largest gains for short-history users—directly mitigating sparsity and cold-start effects.

Finally, we tackle brittleness in implicit-feedback training objectives. Softmax Loss is increasingly adopted in recommender systems due to its strong performance, robustness, and fairness, yet a single global temperature and equal treatment of uniformly sampled negatives can be destabilising because sampled competitor sets vary widely in informativeness. We introduce Dual-scale Softmax Loss (DSL), which infers effective sharpness from the sampled competition itself by adding two complementary branches to the log-sum-exp backbone: first, reweighting negatives within each instance using hardness and item–item similarity; and second, adapting a per-example temperature from the competition intensity over a constructed competitor slate. Across representative benchmarks and backbones, DSL yields substantial gains over strong baselines, exceeding 10% improvements over Softmax Loss in several settings and averaging 6.22% across datasets, metrics, and backbones. Under out-of-distribution popularity shift, the gains are larger, averaging 9.31%. We further provide a distributionally robust optimisation analysis illustrating how DSL reshapes the robust payoff and Kullback–Leibler deviation for ambiguous instances, helping explain the observed improvements in accuracy and robustness.

Collectively, these contributions advance principled architectures and objectives for integrating multimodal content and sequential structure under realistic implicit-feedback competition.