The visual language of fabrics

Deschaintre, Valentin ; Guerrero-Viu, Julia (Universidad de Zaragoza) ; Gutierrez, Diego (Universidad de Zaragoza) ; Boubekeur, Tamy ; Masia, Belen (Universidad de Zaragoza)

Resumen: We introduce text2fabric, a novel dataset that links free-text descriptions to various fabric materials. The dataset comprises 15,000 natural language descriptions associated to 3,000 corresponding images of fabric materials. Traditionally, material descriptions come in the form of tags/keywords, which limits their expressivity, induces pre-existing knowledge of the appropriate vocabulary, and ultimately leads to a chopped description system. Therefore, we study the use of free-text as a more appropriate way to describe material appearance, taking the use case of fabrics as a common item that non-experts may often deal with. Based on the analysis of the dataset, we identify a compact lexicon, set of attributes and key structure that emerge from the descriptions. This allows us to accurately understand how people describe fabrics and draw directions for generalization to other types of materials. We also show that our dataset enables specializing large vision-language models such as CLIP, creating a meaningful latent space for fabric appearance, and significantly improving applications such as fine-grained material retrieval and automatic captioning.

Idioma: Inglés

DOI: 10.1145/3592391

Año: 2023

Publicado en: ACM TRANSACTIONS ON GRAPHICS 42, 4 (2023), [15 pp]
ISSN: 0730-0301

Factor impacto JCR: 7.8 (2023)

Categ. JCR: COMPUTER SCIENCE, SOFTWARE ENGINEERING rank: 5 / 131 = 0.038 (2023) - Q1 - T1

Factor impacto CITESCORE: 14.3 - Computer Graphics and Computer-Aided Design (Q1)

Factor impacto SCIMAGO: 7.766 - Computer Graphics and Computer-Aided Design (Q1)

Tipo y forma: Article (PostPrint)

Área (Departamento): Área Lenguajes y Sistemas Inf. (Dpto. Informát.Ingenie.Sistms.)