SIGGRAPH 2024 Papers — Page 3
ACM SIGGRAPH (Transactions on Graphics) · 252 papers
Specular Polynomials
Zhimin Fan (Nanjing University), Ling-Qi Yan (University of California Santa Barbara)
Diffusion modelScore-based ModelNeural Radiance FieldContrastive LearningGaussian SplattingOptical FlowImageVideoPhysics Related
🎯 What it does: A method is proposed that transforms the solving of mirror chains into a polynomial system and solves all feasible paths through resultants elimination, achieving deterministic light path search without Newton iterations, and applying it to the rendering of high-frequency effects such as glints and caustics.
Spice·E: Structural Priors in 3D Diffusion using Cross-Entity Attention
Etai Sella (Tel Aviv University), Hadar Averbuch-Elor (Tel Aviv University)
GenerationData SynthesisTransformerPrompt EngineeringDiffusion modelScore-based ModelTextPoint CloudMesh
🎯 What it does: Achieve controllable generation of structural priors by introducing cross-entity attention mechanisms into pre-trained 3D diffusion models;
Spin-It Faster: Quadrics Solve All Topology Optimization Problems That Depend Only On Mass Moments
C. Hafner, C. Wojtan
Optimization
🎯 What it does: Mathematical analysis of restricted topology optimization problems that rely solely on quality moments, proving that the optimal material-vacuum interface is always a quadratic surface, and transforming the problem into a small-scale nonlinear system of equations, which can be solved within seconds to obtain an analytical surface directly usable in CAD.
Spin-Weighted Spherical Harmonics for Polarized Light Transport
Shinyoung Yi (KAIST), Min H. Kim (KAIST)
Diffusion modelNeural Radiance FieldAuto EncoderOptical FlowImagePoint CloudPhysics Related
🎯 What it does: Propose polarization spherical harmonics (PSH) based on spin-weighted spherical harmonics to achieve frequency domain representation and real-time polarization light rendering of Stokes vector fields.
Split-and-Fit: Learning B-Reps via Structure-Aware Voronoi Partitioning
Yilin Liu, Hui Huang
GenerationData SynthesisConvolutional Neural NetworkDiffusion modelContrastive LearningPoint CloudMesh
🎯 What it does: Propose the Split-and-Fit framework, which learns structure-aware Voronoi segmentation to predict the B-Rep model of point clouds.
Split-Aperture 2-in-1 Computational Cameras
Zheng Shi, Arabia
RestorationDepth EstimationSuper ResolutionDiffusion modelAuto EncoderOptical FlowImage
🎯 What it does: Proposed a 2-in-1 computational camera design that splits the traditional camera aperture into two halves, enabling the capture of both encoded images and traditional images;
ST-4DGS: Spatial-Temporally Consistent 4D Gaussian Splatting for Efficient Dynamic Scene Rendering
Deqi Li, Hua Huang
GenerationComputational EfficiencyNeural Radiance FieldGaussian SplattingVideo
🎯 What it does: Proposed a spatial-temporally consistent 4D Gaussian expansion model called ST-4DGS for high-quality and efficient dynamic scene rendering
Stabler Neo-Hookean Simulation: Absolute Eigenvalue Filtering for Projected Newton
Honglin Chen (Columbia University), Alec Jacobson (University of Toronto)
OptimizationMeshPhysics Related
🎯 What it does: Proposes a strategy of using the absolute value filtering of negative eigenvalues in the projected Newton method to stabilize and accelerate numerical optimization for Neo-Hookean elastic materials.
Stochastic Computation of Barycentric Coordinates
Fernando de Goes, Mathieu Desbrun
OptimizationMesh
🎯 What it does: A centroidal coordinate calculation method based on random sampling is proposed. By reformulating the kernel integral as a weighted least squares problem, Monte Carlo integration is achieved while maintaining linear accuracy. This method can directly compute centroidal coordinates at points of interest (both inside and outside), supports various cage representations, and does not require volumetric discretization. Additionally, a dedicated denoising scheme specifically for centroidal coordinates is introduced.
StopThePop: Sorted Gaussian Splatting for View-Consistent Real-time Rendering
L. Radl, M. Steinberger
Computational EfficiencyNeural Radiance FieldGaussian SplattingImagePoint Cloud
🎯 What it does: Proposed a hierarchical 3D Gaussian splatting renderer to achieve view-consistent real-time rendering and eliminate popping artifacts when the camera rotates.
Strategy and Skill Learning for Physics-based Table Tennis Animation
Jiashun Wang (Carnegie Mellon University), Jungdam Won (Seoul National University)
TransformerReinforcement LearningMixture of ExpertsDiffusion modelAuto EncoderGenerative Adversarial NetworkVideoSequentialPhysics Related
🎯 What it does: This paper proposes a physics-driven table tennis animation method based on hierarchical control, integrating a strategy layer and a skill layer to achieve the selection and execution of various striking skills.
Streetscapes: Large-scale Consistent Street View Generation Using Autoregressive Video Diffusion
Boyang Deng (Stanford University), Gordon Wetzstein (Stanford University)
GenerationData SynthesisAutonomous DrivingConvolutional Neural NetworkTransformerDiffusion modelOptical FlowImageVideoText
🎯 What it does: Proposed a system called Streetscapes that can generate long-duration, 3D-consistent street view panoramas at the city level, supporting control over layout, camera trajectory, and attributes such as weather/time through maps, elevation maps, and text prompts.
Stylized Rendering as a Function of Expectation
Rex West, Sayan Mukherjee
Diffusion modelNeural Radiance FieldImagePhysics Related
🎯 What it does: Proposed a general extension of the rendering equation that can simultaneously cover physically based rendering (PBR) and part of non-photorealistic rendering (NPR) stylization, and implemented corresponding sampling algorithms for actual rendering;
Subject-Diffusion: Open Domain Personalized Text-to-Image Generation without Test-time Fine-tuning
Jiancang Ma, H. Lu
GenerationData SynthesisTransformerPrompt EngineeringVision Language ModelDiffusion modelContrastive LearningImageTextMultimodality
🎯 What it does: Propose the Subject-Diffusion model, enabling personalized image generation in open domains with only a single reference image, without requiring fine-tuning during testing, and supporting both single and dual-body scenarios.
Super-Resolution Cloth Animation with Spatial and Temporal Coherence
Jiawang Yu, Zhendong Wang
Super ResolutionDiffusion modelOptical FlowMesh
🎯 What it does: A general framework is proposed to achieve super-resolution fabric animation by alternately using a simulator and a corrector, and using grid-based super-resolution to enhance details.
SuperPADL: Scaling Language-Directed Physics-Based Control with Progressive Supervised Distillation
Jordan Juravsky (NVIDIA), X. Peng
Knowledge DistillationRobotic IntelligenceTransformerSupervised Fine-TuningReinforcement LearningMixture of ExpertsVision-Language-Action ModelDiffusion modelContrastive LearningVideoTextMultimodalityPhysics Related
🎯 What it does: Proposes a framework called SuperPADL, which can train a single physical controller capable of executing natural language instructions in real-time on a large-scale (5000+) text-annotated motion dataset.
Surface-Filling Curve Flows via Implicit Medial Axes
Yuta Noma, Alec Jacobson
OptimizationComputational EfficiencyMesh
🎯 What it does: A fast, robust, and controllable algorithm is proposed for generating surface filling curves.
Taming Diffusion Probabilistic Models for Character Control
Rui Chen (Hong Kong University of Science and Technology), Xuelin Chen (Tencent AI Lab)
GenerationData SynthesisPose EstimationTransformerDiffusion modelAuto EncoderVideoSequential
🎯 What it does: Propose a real-time character control framework that generates diverse and high-quality character animations using a conditional autoregressive motion diffusion model (CAMDM) based on user input coarse-grained control signals.
Target-Aware Image Denoising for Inverse Monte Carlo Rendering
Jeongmin Gu, Bochang Moon
OptimizationImagePhysics Related
🎯 What it does: Developed a target-aware image denoiser for inverse Monte Carlo rendering to improve optimization convergence speed
TEDi: Temporally-Entangled Diffusion for Long-Term Motion Synthesis
Zihan Zhang (University of Chicago), Kfir Aberman (Google Research)
GenerationData SynthesisRecurrent Neural NetworkTransformerDiffusion modelScore-based ModelTime SeriesSequential
🎯 What it does: Proposed a new Time-Entangled Diffusion (TEDi) framework for long-term action generation, which couples the progressive denoising process of diffusion models with the timeline of action sequences to achieve autoregressive-style infinite-length action synthesis;
Tele-Aloha: A Telepresence System with Low-budget and High-authenticity Using Sparse RGB Cameras
Hanzhang Tu (Tsinghua University), Yebin Liu (Tsinghua University)
Data SynthesisRobotic IntelligenceConvolutional Neural NetworkTransformerNeural Radiance FieldAuto EncoderGaussian SplattingOptical FlowImagePoint Cloud
🎯 What it does: This paper designs and implements a low-cost, high-fidelity Tele-Aloha upper-body telecommunication system, achieving real-time interaction with a resolution of 2048×2048, 30fps, and latency below 150ms using only four RGB cameras, a single GPU card, and a self-stereoscopic display.
Temporal acoustic point holography
G. Christopoulos, S. Subramanian
OptimizationPhysics RelatedAudio
🎯 What it does: Developed a time-phase retrieval algorithm, improving the Gerchberg-Saxton algorithm and achieving dynamic multi-particle suspension animation
Temporally Stable Metropolis Light Transport Denoising using Recurrent Transformer Blocks
Chuhao Chen, Tzu-Mao Li
RestorationRecurrent Neural NetworkTransformerImage
🎯 What it does: Designed and implemented a sequence denoiser that combines recursive connections with a visual Transformer architecture to improve image quality and temporal stability in Metropolis Light Transport rendering.
TensoSDF: Roughness-aware Tensorial Representation for Robust Geometry and Material Reconstruction
Jia Li, Beibei Wang
Computational EfficiencyRepresentation LearningNeural Radiance FieldAuto EncoderGaussian SplattingImageMesh
🎯 What it does: Propose a roughness-aware tensorized SDF representation method called TensoSDF, which can simultaneously reconstruct geometry and material from multi-view images.
Terrain Amplification using Multi Scale Erosion
Hugo Schott, A. Peytavie
GenerationData SynthesisDiffusion modelScore-based ModelFlow-based ModelRectified FlowMeshPhysics Related
🎯 What it does: Through multi-scale erosion simulation, low-resolution terrain is upscaled to high-resolution, complex terrain with hydrological consistency.
TexPainter: Generative Mesh Texturing with Multi-view Consistency
Hongkun Zhang (Southeast University), Xifeng Gao (LightSpeed Studios)
GenerationData SynthesisTransformerVision Language ModelDiffusion modelScore-based ModelAuto EncoderGenerative Adversarial NetworkOptical FlowImageTextMesh
🎯 What it does: Generate multi-view consistent 3D model textures
TexSliders: Diffusion-Based Texture Editing in CLIP Space
Julia Guerrero-Viu, V. Deschaintre
Image TranslationRestorationGenerationTransformerPrompt EngineeringDiffusion modelScore-based ModelContrastive LearningImageMultimodality
🎯 What it does: Propose TexSliders, which achieves texture editing based on diffusion models by constructing sliders in the CLIP image embedding space;
Text-Guided Synthesis of Crowd Animation
Xuebo Ji, Jia Pan
GenerationData SynthesisTransformerLarge Language ModelDiffusion modelText
🎯 What it does: A machine learning method based on text description is proposed, which uses conditional diffusion models to generate text-guided agent distribution fields and velocity fields, and combines them with local navigation algorithms to control multiple agents, thereby synthesizing diverse dynamic crowd animation scenes; at the same time, a large language model is used to standardize general scripts into structured sentences to improve training stability and scalability.
Text-to-Vector Generation with Neural Path Representation
Peiying Zhang (City University of Hong Kong), Jing Liao (City University of Hong Kong)
GenerationRepresentation LearningTransformerDiffusion modelScore-based ModelAuto EncoderContrastive LearningImageTextMultimodality
🎯 What it does: Propose a complete process for generating vector graphics (SVG) from text, with the core being the realization of high-quality, editable SVG output through neural path representation learning and two-stage path optimization.
The Chosen One: Consistent Characters in Text-to-Image Diffusion Models
Omri Avrahami (Hebrew University of Jerusalem), D. Lischinski
GenerationData SynthesisTransformerSupervised Fine-TuningPrompt EngineeringDiffusion modelContrastive LearningImageTextMultimodality
🎯 What it does: Propose a fully automated method for generating role consistency in text-to-image diffusion models, which refines a unified identity and generates coherent role images in new contexts through iterative generation, embedding, clustering, and personalized fine-tuning.
ThemeStation: Generating Theme-Aware 3D Assets from Few Exemplars
Zhenwei Wang (City University of Hong Kong), Rynson W. H. Lau
GenerationData SynthesisTransformerSupervised Fine-TuningPrompt EngineeringDiffusion modelScore-based ModelNeural Radiance FieldGenerative Adversarial NetworkContrastive LearningImageTextPoint CloudMesh
🎯 What it does: Proposes ThemeStation, a two-stage theme-aware 3D-to-3D generation framework that can synthesize diverse and theme-consistent 3D assets using only a minimal number of 3D examples.
Theory of Human Tetrachromatic Color Experience and Printing
Jessica Lee, Ren Ng
Review/Survey Paper
🎯 What it does: Proposed a theoretical framework of human tetrachromatic color experience and implemented a prototype for tetrachromatic printing; derived the tetrachromatic color space and hue sphere from multidimensional color theory, designed a tetrachromatic ink system and printing method; extended existing color tests to identify tetrachromatic color perception in the wild.
TIP-Editor: An Accurate 3D Editor Following Both Text-Prompts And Image-Prompts
Jingyu Zhuang (Sun Yat-sen University), Ying Shan (Tencent AI Lab)
GenerationData SynthesisTransformerSupervised Fine-TuningPrompt EngineeringVision Language ModelDiffusion modelScore-based ModelGaussian SplattingImageTextPoint CloudMeshStochastic Differential Equation
🎯 What it does: Propose TIP-Editor, a 3D scene editing framework that supports text and image prompts and specifies editing regions through 3D bounding boxes, capable of performing various editing operations such as object insertion, replacement, re-texturing, and stylization.
Toonify3D: StyleGAN-based 3D Stylized Face Generator
W. Jang, Seungyong Lee
GenerationData SynthesisDiffusion modelGenerative Adversarial NetworkImageMesh
🎯 What it does: Convert Toonify into a 3D stylized face generator, and generate facial normal maps through the feature regression method of StyleGAN, ultimately producing full-head 3D stylized avatars and supporting expression editing.
Towards Motion Metamers for Foveated Rendering
Taimoor Tariq, P. Didyk
Optical Flow
🎯 What it does: This study investigates the impact of visual field rendering on motion perception and proposes a real-time, perception-based technique to compensate for the missing motion perception in the visual field area.
Towards Unstructured Unlabeled Optical Mocap: A Video Helps!
Nicholas Milef (Texas A&M University), Shuqing Kong
Pose EstimationOptimizationConvolutional Neural NetworkTransformerDiffusion modelAuto EncoderGenerative Adversarial NetworkContrastive LearningOptical FlowVideoPoint CloudMesh
🎯 What it does: Studied the problem of unstructured and unlabeled optical motion capture (UUO mocap), proposing to utilize synchronized monocular video to generate body priors and combine them with unlabeled markers to achieve simultaneous reconstruction of full/local body pose and shape.
Training-Free Consistent Text-to-Image Generation
Yoad Tewel (NVIDIA), Y. Atzmon
GenerationTransformerPrompt EngineeringVision Language ModelDiffusion modelImageTextMultimodality
🎯 What it does: Propose a training-free consistency text-to-image generation method called ConsiStory, which maintains visual consistency of the same subject under different prompts by leveraging the internal activations shared within a pre-trained diffusion model;
Transparent Image Layer Diffusion using Latent Transparency
Lvmin Zhang (Stanford University), Maneesh Agrawala (Stanford University)
GenerationData SynthesisTransformerDiffusion modelAuto EncoderImageTextMultimodality
🎯 What it does: Achieved end-to-end generation of transparent images and multi-layer transparent images on large-scale latent diffusion models (e.g., Stable Diffusion XL);
Ultra Inertial Poser: Scalable Motion Capture and Tracking from Sparse Inertial Sensors and Ultra-Wideband Ranging
Rayan Armani (ETH Zürich), Christian Holz (ETH Zürich)
Pose EstimationRecurrent Neural NetworkGraph Neural NetworkContrastive LearningSimultaneous Localization and MappingOptical FlowPoint CloudGraphTime Series
🎯 What it does: Propose the Ultra Inertial Poser method, which combines sparse inertial sensors with UWB ranging to achieve real-time full-body pose estimation.
Universal Facial Encoding of Codec Avatars from VR Headsets
Shaojie Bai (Meta), S. Wei
GenerationPose EstimationRepresentation LearningTransformerDiffusion modelAuto EncoderGenerative Adversarial NetworkContrastive LearningOptical FlowImageVideo
🎯 What it does: A general encoder was developed to real-time capture facial expressions from the head-mounted camera (HMC) of VR headsets and generate realistic avatars.
Variational Feature Extraction in Scientific Visualization
Nico Daßler, Tobias Günther
OptimizationPhysics Related
🎯 What it does: Proposes a mathematical framework based on variational principles, treating feature extraction in scientific visualization as a variational minimization problem, and provides regularization methods that can be used across domains;
Velocity-Based Monte Carlo Fluids
Ryusuke Sugimoto (University of Waterloo), T. Hachisuka
Diffusion modelOptical FlowImageVideoMeshPhysics RelatedStochastic Differential EquationOrdinary Differential Equation
🎯 What it does: This paper proposes a velocity-based Monte Carlo fluid solver that uses operator splitting and point value Monte Carlo estimators to implement four sub-steps of the Navier–Stokes equations (advection, external force, diffusion, projection), thereby overcoming the incorrect physical behavior of previous vorticity-based Monte Carlo methods in multi-obstacle and non-simply connected domains.
Versatile Vision Foundation Model for Image and Video Colorization
Vukasin Bozic, Christopher Schroers
Image TranslationRestorationTransformerSupervised Fine-TuningPrompt EngineeringDiffusion modelImageVideo
🎯 What it does: Achieve image and video colorization by fine-tuning pre-trained latent diffusion models (based on text-to-image synthesis), offering a flexible solution for high-quality direct colorization, user-guided colorization (color prompts, text prompts, or reference images), and video colorization.
Vertex Block Descent
A. Chen (University of Utah), C. Yuksel (University of Utah)
OptimizationDiffusion modelScore-based ModelOptical FlowPoint CloudMeshTabularPhysics RelatedStochastic Differential EquationOrdinary Differential Equation
🎯 What it does: Propose a physics solver called Vertex Block Descent (VBD) based on block coordinate descent, which solves the implicit Euler variational form by performing Gauss–Seidel iterations on individual vertices, achieving unconditionally stable, controllable iteration counts, and high parallelism for elastic body dynamics simulations.
VR-GS: A Physical Dynamics-Aware Interactive Gaussian Splatting System in Virtual Reality
Ying Jiang, Chenfanfu Jiang
OptimizationComputational EfficiencyRobotic IntelligenceNeural Radiance FieldGaussian SplattingImageVideoMesh
🎯 What it does: Built an interactive system VR‑GS based on virtual reality, utilizing 3D Gaussian Splatting to achieve real-time, physics-aware dynamic editing and rendering.
VRMM: A Volumetric Relightable Morphable Head Model
Haotian Yang (Kuaishou Technology), Haibin Huang (Kuaishou Technology)
RestorationGenerationRepresentation LearningConvolutional Neural NetworkTransformerDiffusion modelNeural Radiance FieldAuto EncoderGenerative Adversarial NetworkContrastive LearningGaussian SplattingImageVideoMesh
🎯 What it does: Proposed VRMM (Volumetric Relightable Morphable Model), a voxel-based prior model that can separate identity, expression, and illumination in a low-dimensional space and support continuous relighting, animation, and high-quality avatar reconstruction;
Walkin’ Robin: Walk on Stars with Robin Boundary Conditions
Bailey Miller, Ioannis Gkioulekas
OptimizationPhysics RelatedStochastic Differential Equation
🎯 What it does: A Monte Carlo-based random walk method was developed for solving elliptic partial differential equation boundary value problems with arbitrary first-order linear boundary conditions (Dirichlet, Neumann, Robin).
WalkTheDog: Cross-Morphology Motion Alignment via Phase Manifolds
Peizhuo Li (ETH Zurich), O. Sorkine-Hornung
GenerationPose EstimationRetrievalConvolutional Neural NetworkAuto EncoderContrastive LearningVideoSequential
🎯 What it does: Propose a discrete one-dimensional phase manifold, using a vector-quantized periodic autoencoder (VQ-PAE) to learn a shared phase representation across different morphologies (humans, dogs, monsters, etc.), and utilize this manifold for motion retrieval, transfer, and stylization.
Woven Fabric Capture with a Reflection-Transmission Photo Pair
Yingjie Tang (Nankai University), Beibei Wang (Nanjing University)
RestorationGenerationData SynthesisConvolutional Neural NetworkGraph Neural NetworkTransformerDiffusion modelScore-based ModelAuto EncoderGenerative Adversarial NetworkContrastive LearningOptical FlowImageMeshBenchmark
🎯 What it does: By acquiring reflectance-transmittance images of fabrics and combining them with a new two-layer BSDF model, the inversion and reconstruction of fabric parameters are achieved.
X-Portrait: Expressive Portrait Animation with Hierarchical Motion Attention
You Xie (ByteDance), Linjie Luo (ByteDance)
Image TranslationGenerationConvolutional Neural NetworkTransformerPrompt EngineeringDiffusion modelImageVideo
🎯 What it does: This paper proposes X-Portrait, a zero-shot Stable Diffusion-based portrait animation framework that can generate high-fidelity, expressive, and temporally coherent video from a single static portrait;
X-SLAM: Scalable Dense SLAM for Task-aware Optimization using CSFD
Zhexi Peng (State Key Lab of CAD&CG Zhejiang University), Kun Zhou (State Key Lab of CAD&CG Zhejiang University)
OptimizationRobotic IntelligenceGaussian SplattingSimultaneous Localization and MappingOptical FlowImageVideoPoint Cloud
🎯 What it does: Proposed a real-time differentiable dense SLAM system called X-SLAM, which utilizes the CSFD method to achieve high-order gradient computation, and on this basis, implements task-aware optimization, such as camera relocalization and robot active scanning.
ZeroGrads: Learning Local Surrogates for Non-Differentiable Graphics
Michael Fischer (University College London), Tobias Ritschel (University College London)
OptimizationComputational EfficiencyData-Centric LearningAuto EncoderContrastive LearningGaussian SplattingImagePoint CloudMeshGraphTabularSequential
🎯 What it does: Propose the ZeroGrads framework, which online self-supervisedly learns a locally differentiable surrogate (neural network) and utilizes its gradient for parameter optimization in black-box graphics pipelines where gradients cannot be directly obtained.