Skip to main content
Back

KiseKloset Combines Visual Search and Virtual Try-On to Fix Fashion Shopping

A Vietnamese research team built an end-to-end fashion shopping system pairing CLIP-based retrieval, a novel outfit transformer, and real-time virtual try-on.

Pro

Paper, Full Access
  • KiseKloset unifies outfit retrieval, complementary recommendation, and virtual try-on in one end-to-end fashion shopping system.
  • A novel transformer treats outfits as sequences, filling in variable numbers of missing items across ten categories.
  • Text-feedback retrieval uses CLIP4Cir to combine reference images with natural language edits like color or sleeve changes.
  • HNSW indexing hits 1.80ms queries at 98.11% recall, a roughly 20x speedup over exhaustive KNN.
  • Real-time try-on uses DM-VTON, a teacher-student network with a MobileNetV2-based student for speed.
  • User study: 84.4% found it highly useful, 65.6% would recommend, but try-on quality scored just 3.3 out of 5.

Online fashion shopping has a trust problem. Shoppers can browse thousands of items but cannot feel the fabric, ask a salesperson for outfit suggestions, or try things on before their card gets charged. A team from the University of Science in Ho Chi Minh City and the University of Dayton has stitched together a system called KiseKloset that attacks all three problems at once, combining visual search, complementary outfit recommendation, and real-time virtual try-on into a single pipeline.

The paper is less about a single algorithmic breakthrough and more about assembling known components into something that actually ships. The authors claim KiseKloset is the first system supporting both outfit retrieval and recommendation (ORR) and virtual try-on (VTON) for real-world applications, and they backed the claim with a deployed user study.

Why a search bar isn't enough

The economic stakes are large. Global fashion e-commerce is expected to top 820 billion USD in 2023 and reach 1.2 trillion by 2027, with recommendation systems capable of boosting conversion rates by up to 300% and revenue by up to 31%. Research in the space has been fragmented, though. Outfit retrieval, complementary item recommendation, and virtual try-on have been studied largely as separate problems, with different datasets, different architectures, and no cohesive user experience.

Three pipelines, one CLIP backbone

KiseKloset supports two retrieval modes. The first is standard similar-item search using CLIP image embeddings. The second is text-feedback retrieval, where a user provides a reference image plus natural language like "same style but green with longer sleeves." For this, the team adopts the CLIP4Cir architecture, which extracts embeddings from both the reference image and the accompanying text, then merges them via a combiner network trained with symmetric contrastive loss.