Welcome to the new version of CaltechAUTHORS. Login is currently restricted to library staff. If you notice any issues, please email coda@library.caltech.edu
Published March 1, 2022 | Submitted
Report Open

Rayleigh EigenDirections (REDs): GAN latent space traversals for multidimensional features

Abstract

We present a method for finding paths in a deep generative model's latent space that can maximally vary one set of image features while holding others constant. Crucially, unlike past traversal approaches, ours can manipulate multidimensional features of an image such as facial identity and pixels within a specified region. Our method is principled and conceptually simple: optimal traversal directions are chosen by maximizing differential changes to one feature set such that changes to another set are negligible. We show that this problem is nearly equivalent to one of Rayleigh quotient maximization, and provide a closed-form solution to it based on solving a generalized eigenvalue equation. We use repeated computations of the corresponding optimal directions, which we call Rayleigh EigenDirections (REDs), to generate appropriately curved paths in latent space. We empirically evaluate our method using StyleGAN2 on two image domains: faces and living rooms. We show that our method is capable of controlling various multidimensional features out of the scope of previous latent space traversal methods: face identity, spatial frequency bands, pixels within a region, and the appearance and position of an object. Our work suggests that a wealth of opportunities lies in the local analysis of the geometry and semantics of latent spaces.

Additional Information

Attribution 4.0 International (CC BY 4.0).

Attached Files

Submitted - 2201.10423.pdf

Files

2201.10423.pdf
Files (37.3 MB)
Name Size Download all
md5:fbc5cbaeb79b6effba595a650a1dd942
37.3 MB Preview Download

Additional details

Created:
August 20, 2023
Modified:
October 23, 2023