Publication

PoseFix: correcting 3D human poses with natural language

Conference Article

Conference

International Conference on Computer Vision (ICCV)

Edition

2023

Pages

14972-14982

Doc link

https://doi.org/10.1109/ICCV51070.2023.01379

File

Download the digital copy of the doc pdf document

Abstract

Automatically producing instructions to modify one’s posture could open the door to endless applications, such as personalized coaching and in-home physical therapy. Tackling the reverse problem (i.e., refining a 3D pose based on some natural language feedback) could help for assisted 3D character animation or robot teaching, for instance. Although a few recent works explore the connections between natural language and 3D human pose, none focus on describing 3D body pose differences. In this paper, we tackle the problem of correcting 3D human poses with natural language. To this end, we introduce the PoseFix dataset, which consists of several thousand paired 3D poses and their corresponding text feedback, that describe how the source pose needs to be modified to obtain the target pose. We demonstrate the potential of this dataset on two tasks: (1) text-based pose editing, that aims at generating corrected 3D body poses given a query pose and a text modifier; and (2) correctional text generation, where instructions are generated based on the differences between two body poses.

Categories

computer vision.

Scientific reference

G.D. Delmas, P. Weinzaepfel, F. Moreno-Noguer and G. Rogez. PoseFix: correcting 3D human poses with natural language, 2023 International Conference on Computer Vision, 2023, Paris, France, pp. 14972-14982.