SIGGRAPH 2024 Papers — Page 2
ACM SIGGRAPH (Transactions on Graphics) · 252 papers
Interactive Design of Stylized Walking Gaits for Robotic Characters
Michael A. Hopkins, Moritz Bächer
Pose EstimationRobotic IntelligenceDiffusion modelGenerative Adversarial NetworkOptical FlowMultimodality
🎯 What it does: Designed an interactive tool that allows artists to edit and generate stylized bipedal walking gaits in real-time, applicable for physical or simulated robotic execution, and realized real-time motion generation through a model-based control stack.
Interactive Invigoration: Volumetric Modeling of Trees with Strands
Bosheng Li, B. Benes
GenerationData SynthesisDiffusion modelAuto EncoderGenerative Adversarial NetworkMeshAgriculture Related
🎯 What it does: Proposes a volumetric model based on Strands, using fixed-size volumetric tubes to define the branching structure of trees, and combines a new branching development formula with editing operators to achieve interactive and detailed modeling of tree trunks and branches;
Into the Portal: Directable Fractal Self-Similarity
Alexa Schor, Theodore Kim
GenerationDiffusion model
🎯 What it does: Proposes a fractal method that can directly specify self-similar positions, enabling the introduction of self-similarity into arbitrary shapes and the recreation of known fractals such as the Koch snowflake.
IntrinsicDiffusion: Joint Intrinsic Layers from Latent Diffusion Models
Jundan Luo, Tuanfeng Wang
GenerationData SynthesisTransformerVision Language ModelDiffusion modelImageTextMultimodality
🎯 What it does: Propose a conditional mechanism built on a pre-trained large-scale text-to-image generation model to jointly predict multiple intrinsic modalities of an image (such as albedo, illumination, surface geometry)
InvertAvatar: Incremental GAN Inversion for Generalized Head Avatars
Xiaochen Zhao (Tsinghua University), Yebin Liu (Tsinghua University)
RestorationGenerationPose EstimationConvolutional Neural NetworkRecurrent Neural NetworkDiffusion modelAuto EncoderGenerative Adversarial NetworkContrastive LearningImageVideo
🎯 What it does: Propose an incremental GAN inversion framework that quickly generates high-fidelity, animatable 3D head avatars using multi-frame input.
Iterative Motion Editing with Natural Language
Purvi Goel (Stanford University), Kayvon Fatahalian (Stanford University)
GenerationData SynthesisPose EstimationTransformerLarge Language ModelPrompt EngineeringDiffusion modelScore-based ModelVideoTextSequentialRetrieval-Augmented Generation
🎯 What it does: This study proposes an iterative motion editing framework based on natural language, which maps editing intentions into executable motion editing operations (MEOs), and utilizes diffusion models to generate motion, thereby enabling fine-grained and predictable modifications to existing character animations.
Kinetic Simulation of Turbulent Multifluid Flows
Wei Li, Mathieu Desbrun
Diffusion modelOptical FlowPhysics Related
🎯 What it does: A multi-phase flow simulation solver based on the LBM (Lattice Boltzmann Method) is proposed, capable of handling miscible, immiscible, and partially miscible fluids, supporting turbulent high Reynolds numbers and large density ratios.
Lagrangian Covector Fluid with Free Surface
Zhiqi Li (Georgia Institute of Technology), Greg Turk (Georgia Institute of Technology)
Diffusion modelOptical FlowPoint CloudMeshBenchmarkPhysics RelatedStochastic Differential EquationOrdinary Differential Equation
🎯 What it does: Propose a free surface fluid simulation method based on Lagrangian covector flow graphs; decouple long-range flow graph mappings from short-range projections via path integrals to address the incompressibility constraint under free boundaries; implement the Voronoi mesh particle method and validate it in various benchmark experiments;
LayGA: Layered Gaussian Avatars for Animatable Clothing Transfer
Siyou Lin (Tsinghua University), Yebin Liu (Tsinghua University)
Image TranslationGenerationPose EstimationConvolutional Neural NetworkTransformerDiffusion modelContrastive LearningGaussian SplattingOptical FlowImageVideoMesh
🎯 What it does: Propose Layered Gaussian Avatars (LayGA), a hierarchical human avatar model based on 3D Gaussian Splatting, which can achieve pose-driven clothing animation and cross-identity clothing transfer.
Learning a Generalized Physical Face Model From Data
Lingchen Yang (ETH Zurich), Derek Bradley (DisneyResearch|Studios)
GenerationData SynthesisDiffusion modelScore-based ModelNeural Radiance FieldAuto EncoderGenerative Adversarial NetworkContrastive LearningOptical FlowImagePoint CloudMeshPhysics RelatedStochastic Differential EquationOrdinary Differential Equation
🎯 What it does: This paper proposes a general physical facial model that can learn from large-scale 3D facial scan data and quickly fit and generate physically simulated facial animations on a single image or 3D scan;
Learning Images Across Scales Using Adversarial Training
Krzysztof Wolski (Max Planck Institute for Informatics), Thomas Leimkühler (Max Planck Institute for Informatics)
GenerationSuper ResolutionDiffusion modelAuto EncoderGenerative Adversarial NetworkContrastive LearningImage
🎯 What it does: Learn a continuous multi-scale image space from unstructured, low-resolution multi-scale image patches, enabling seamless visualization and synthesis of scenes at up to 256 times magnification.
LGTM: Local-to-Global Text-Driven Human Motion Diffusion Model
Haowen Sun (Shenzhen University), Ruizhen Hu (Shenzhen University)
GenerationPose EstimationTransformerLarge Language ModelPrompt EngineeringDiffusion modelTextMultimodalityRetrieval-Augmented Generation
🎯 What it does: Propose LGTM, a local-global text-driven human motion generation framework that first decomposes textual descriptions into part-level semantics and then gradually generates local motions using diffusion models, ultimately fusing them into a complete motion through a full-body optimizer.
Lifting Directional Fields to Minimal Sections
David R Palmer, J. Solomon
OptimizationMesh
🎯 What it does: Propose a new framework based on the convex relaxation of minimal sections, used to optimize direction fields (including unit vector fields, line fields, and cross fields) within a circle bundle, and explicitly handle their singularities.
LightFormer: Light-Oriented Global Neural Rendering in Dynamic Scene
Haocheng Ren, Hujun Bao
GenerationData SynthesisTransformerNeural Radiance FieldAuto EncoderImageVideo
🎯 What it does: Propose a neural rendering method called LightFormer, which can generate realistic global illumination in real-time within fully dynamic scenes;
Lightning-fast Method of Fundamental Solutions
Jiong Chen, Mathieu Desbrun
OptimizationComputational EfficiencyGaussian SplattingPhysics Related
🎯 What it does: A variational preconditioner was developed to improve the computational scalability and memory usage of the method of fundamental solutions (MFS) and boundary element method (BEM); the preconditioner is implemented via sparse inverse Cholesky decomposition and can efficiently solve problems in large-scale parallel environments.
Lite2Relight: 3D-aware Single Image Portrait Relighting
Pramod Rao (MPI for Informatics), Christian Theobalt (MPI for Informatics)
Image TranslationRestorationGenerationTransformerDiffusion modelNeural Radiance FieldAuto EncoderGenerative Adversarial NetworkContrastive LearningImagePoint CloudMesh
🎯 What it does: Designed a single-image relighting method based on 3D GAN, which can re-render under arbitrary environmental lighting while maintaining 3D-consistent pose and details.
LOOSECONTROL: Lifting ControlNet for Generalized Depth Conditioning
Shariq Farooq Bhat (King Abdullah University of Science and Technology), Peter Wonka (King Abdullah University of Science and Technology)
GenerationDepth EstimationConvolutional Neural NetworkTransformerSupervised Fine-TuningPrompt EngineeringDiffusion modelImageTextPoint Cloud
🎯 What it does: Propose LOOSECONTROL, which allows text-to-image diffusion model generation using loose depth constraints (scene boundaries or 3D boxes), and provides two interactive modification methods: 3D box editing and attribute editing.
MaPa: Text-driven Photorealistic Material Painting for 3D Shapes
Shangzhan Zhang, Xiaowei Zhou (Zhejiang University)
GenerationData SynthesisConvolutional Neural NetworkTransformerPrompt EngineeringVision Language ModelDiffusion modelAuto EncoderGenerative Adversarial NetworkImageTextMesh
🎯 What it does: Propose a text-based 3D mesh material painting framework called MaPa, which generates corresponding 2D images using a segment-controlled diffusion model, and then obtains high-quality editable materials through differentiable rendering and procedural material map optimization.
Matting by Generation
Zhixiang Wang (University of Tokyo), Shin’ichi Satoh
SegmentationGenerationTransformerPrompt EngineeringDiffusion modelImageText
🎯 What it does: This paper proposes a 'Matting by Generation' method, transforming the traditional image matting regression task into a conditional generation task. It generates high-quality alpha matte by utilizing a pre-trained latent diffusion model combined with the latent representation of the input image, supporting various guidance types such as no trimap, prompt mask, sketch, and text.
Media2Face: Co-speech Facial Animation Generation With Multi-Modality Guidance
Qingcheng Zhao (ShanghaiTech University), Lan Xu (ShanghaiTech University)
GenerationData SynthesisTransformerPrompt EngineeringDiffusion modelAuto EncoderContrastive LearningImageVideoTextMultimodalityAudio
🎯 What it does: Propose a multi-modal guided diffusion model called Media2Face, which can generate high-fidelity, controllable 3D talking face animations and head movements based on multi-source information such as speech, text, and images.
Mesh Neural Cellular Automata
Ehsan Pajouheshgar (EPFL), Sabine Süsstrunk (EPFL)
GenerationData SynthesisGraph Neural NetworkDiffusion modelAuto EncoderGenerative Adversarial NetworkContrastive LearningOptical FlowImageVideoTextMesh
🎯 What it does: This paper proposes MeshNCA, a neural cellular automaton (NCA) framework capable of performing real-time, interactive texture synthesis on arbitrary 3D meshes.
Minkowski Penalties: Robust Differentiable Constraint Enforcement for Vector Graphics
Jiří Minarčík, Keenan Crane
OptimizationMesh
🎯 What it does: An optimization-based framework is proposed to achieve the arrangement of 2D vector shapes under multiple pairwise constraints by minimizing the energy penalty derived from the signed distance function of the Minkowski gap of interacting shapes.
Mob-FGSR: Frame Generation and Super Resolution for Mobile Real-Time Rendering
Sipeng Yang, Xiaogang Jin
GenerationSuper ResolutionGaussian SplattingOptical FlowImageVideo
🎯 What it does: Proposes Mob-FGSR, a lightweight upsampling framework for mobile devices, which integrates frame generation with super-resolution to enhance real-time rendering performance.
MoConVQ: Unified Physics-Based Motion Control via Scalable Discrete Representations
Heyuan Yao (Peking University), Libin Liu (Peking University)
Pose EstimationRobotic IntelligenceTransformerLarge Language ModelReinforcement LearningPrompt EngineeringMixture of ExpertsAuto EncoderWorld ModelVideoTextSequential
🎯 What it does: Proposes the MoConVQ framework, which uses discrete VQ-VAE encoding to achieve unified physics-driven motion control, supporting tracking, interactive control, text generation, and integration with LLMs.
Modal Folding: Discovering Smooth Folding Patterns for Sheet Materials using Strain-Space Modes
P. Tang, Stelian Coros (ETH Zürich)
OptimizationDiffusion modelContrastive LearningMeshPhysics Related
🎯 What it does: This paper proposes an automatic method called Modal Folding for discovering smooth folding patterns of sheet materials (such as fabrics, paper, and copper sheets) under a given deformation amplitude.
Modeling Ambient Scene Dynamics for Free-view Synthesis
Meng-Li Shih (University of Washington), Chen Gao (Meta)
GenerationData SynthesisNeural Radiance FieldGaussian SplattingOptical FlowVideo
🎯 What it does: Free-viewpoint rendering for dynamic natural scenes reconstructed and synthesized from monocular videos, achieving high-quality, real-time visualization using 3D Gaussian Splatting.
Modeling Hair Strands with Roving Capsules
A. Reshetov, David Hart
Computational EfficiencyDiffusion modelNeural Radiance FieldGaussian SplattingOptical FlowMesh
🎯 What it does: A significantly accelerated intersection algorithm is proposed by modeling hair strands through sweeping a radius-varying sphere along a Bézier curve and using dynamically defined capsules for ray tracing.
Modelling a Feather as a Strongly Anisotropic Elastic Shell
Jean Jouve, F. Bertails-Descoubes
Diffusion modelScore-based ModelAuto EncoderGenerative Adversarial NetworkContrastive LearningOptical FlowMeshPhysics RelatedStochastic Differential EquationOrdinary Differential Equation
🎯 What it does: Studied and constructed a strong heterogeneous elastic shell model for a rachis, and verified its performance through experiments; meanwhile, proposed methods of grid alignment and inextensibility constraints to address the locking and ill-conditioning problems caused by extreme stiffness ratios.
MonoGaussianAvatar: Monocular Gaussian Point-based Head Avatar
Yufan chen, Yebin Liu (Harbin Institute of Technology)
RestorationGenerationConvolutional Neural NetworkTransformerDiffusion modelNeural Radiance FieldAuto EncoderContrastive LearningGaussian SplattingImageVideoPoint Cloud
🎯 What it does: This paper proposes MonoGaussianAvatar, a method for constructing dynamic head 3D avatars from monocular portrait videos using 3D Gaussian points representation and Gaussian deformation fields.
Motion-I2V: Consistent and Controllable Image-to-Video Generation with Explicit Motion Modeling
Xiaoyu Shi (Chinese University of Hong Kong), Hongsheng Li
GenerationData SynthesisTransformerDiffusion modelAuto EncoderOptical FlowImageVideoText
🎯 What it does: Propose the Motion-I2V framework, which decomposes image-to-video generation into two stages: the first stage predicts a pixel-level motion field using a diffusion model; the second stage generates temporally consistent and detail-preserving videos under the guidance of the predicted motion, leveraging motion-enhanced temporal attention.
MotionCtrl: A Unified and Flexible Motion Controller for Video Generation
Zhouxia Wang (Nanyang Technological University), Ying Shan (Tencent)
GenerationData SynthesisConvolutional Neural NetworkTransformerPrompt EngineeringVision Language ModelDiffusion modelVideoText
🎯 What it does: Propose the MotionCtrl unified model to achieve independent and joint control of camera and object motion
Multi-Material Mesh-Based Surface Tracking with Implicit Topology Changes
P. Heiss-Synak, C. Wojtan
Mesh
🎯 What it does: Proposed a multi-material non-manifold mesh surface tracking algorithm that converts self-intersections into topological changes;
MVD^2: Efficient Multiview 3D Reconstruction for Multiview Diffusion
Xin-Yang Zheng (Tsinghua University), Yang Liu (Microsoft Research Asia)
GenerationData SynthesisComputational EfficiencyRepresentation LearningConvolutional Neural NetworkTransformerDiffusion modelNeural Radiance FieldAuto EncoderContrastive LearningGaussian SplattingImageMultimodalityPoint CloudMesh
🎯 What it does: This study proposes the MVD 2 model to address the problem of efficiently recovering 3D geometry from sparse and inconsistent multi-view diffusion-generated images;
N-BVH: Neural ray queries with bounding volume hierarchies
Philippe Weier (Saarland University), T. Boubekeur
CompressionComputational EfficiencyNeural Radiance FieldAuto EncoderPoint CloudMesh
🎯 What it does: This paper proposes N-BVH, a ray query compression structure that integrates neural networks with traditional BVH, used to efficiently replace traditional triangle mesh queries in ray tracing.
N-Dimensional Gaussians for Fitting of High Dimensional Functions
Stavros Diolatzis (Intel Labs), Anton Kaplanyan (Intel Labs)
OptimizationComputational EfficiencyRepresentation LearningDiffusion modelNeural Radiance FieldContrastive LearningGaussian SplattingPoint CloudMeshTabularBenchmark
🎯 What it does: Construct and train an N-dimensional Gaussian Mixture Model (GMM) to explicitly approximate functions with high-dimensional parameter spaces, achieving fast training and efficient rendering.
Navigation-Driven Approximate Convex Decomposition
James Andrews
🎯 What it does: Proposes the navigable approximate convex decomposition problem, defines navigable space and seeks convex decompositions that do not overlap with this space, and provides corresponding algorithm implementations
NeRF as a Non-Distant Environment Emitter in Physics-based Inverse Rendering
Jingwang Ling (Tsinghua University), Shuang Zhao (University of California, Irvine)
OptimizationDiffusion modelNeural Radiance FieldAuto EncoderGenerative Adversarial NetworkContrastive LearningGaussian SplattingOptical FlowImagePoint CloudPhysics Related
🎯 What it does: This paper proposes using NeRF as a non-distant environment emitter, achieving physically accurate inverse rendering through a hybrid rendering that combines object surfaces and NeRF.
Neural Bounding
Stephanie Liu, Tobias Ritschel (University College London)
OptimizationComputational EfficiencyRepresentation LearningDiffusion modelNeural Radiance FieldAuto EncoderContrastive LearningImagePoint CloudMeshTime Series
🎯 What it does: This paper proposes using neural networks to learn conservative bounding volumes for complex geometries (neural bounding), achieving zero false negatives (FN) and significantly reducing false positives (FP rate);
Neural Control Variates with Automatic Integration
Zilu Li (Cornell University), Gordon Wetzstein (Stanford University)
OptimizationComputational EfficiencyScore-based ModelFlow-based ModelAuto EncoderTabularTime SeriesSequentialPhysics Related
🎯 What it does: Proposes an automatic integration method that constructs control variables using arbitrary neural network architectures, achieving low-variance estimation in Monte Carlo sampling by approximating the antiderivative of the integrand and obtaining an analytical integral through automatic differentiation.
Neural Gaussian Scale-Space Fields
Felix Mujkanovic (Max-Planck-Institut für Informatik), Thomas Leimkühler (Max-Planck-Institut für Informatik)
RestorationConvolutional Neural NetworkTransformerDiffusion modelScore-based ModelNeural Radiance FieldGaussian SplattingImageMultimodalityPoint CloudMesh
🎯 What it does: This study proposes a continuous anisotropic Gaussian Scale-Space learning method based on neural fields, which can generate smooth results that can be queried at any spatial position and any covariance matrix without using any artificial filters, under self-supervised training.
Neural Geometry Fields For Meshes
Venkataram Edavamadathil Sivaram, Ravi Ramamoorthi
CompressionNeural Radiance FieldAuto EncoderMesh
🎯 What it does: Propose a neural representation method called Neural Geometry Fields, which compensates for details in triangle meshes using rough quadrilateral patches and coordinate neural networks, and achieves mesh compression by sampling displacement to extract traditional triangle meshes.
Neural Monte Carlo Fluid Simulation
Pranav Jain, Oded Stein
Neural Radiance FieldAuto EncoderPhysics Related
🎯 What it does: Proposed a neural network-based fluid simulation method that uses explicit boundary conditions and a Monte Carlo pressure solver, completely mesh-free.
Neural Slicer for Multi-Axis 3D Printing
Tao Liu (University of Manchester), Charlie C. L. Wang (University of Manchester)
OptimizationReinforcement LearningDiffusion modelScore-based ModelNeural Radiance FieldAuto EncoderContrastive LearningPoint CloudMesh
🎯 What it does: Proposed a Neural Slicer based on neural networks, which can automatically generate surface layers that meet the requirements of support freedom and strength enhancement for multi-axis 3D printing;
Neural-Assisted Homogenization of Yarn-Level Cloth
Xudong Feng, Weiwei Xu
OptimizationComputational EfficiencyDiffusion modelScore-based ModelFlow-based ModelRectified FlowNeural Radiance Field
🎯 What it does: A neural-assisted homogenized constitutive model for simulating yarn-layer fabric was developed, and a sector-based hot-start strategy was proposed.
NeuralTO: Neural Reconstruction and View Synthesis of Translucent Objects
Yuxiang Cai, Bo-Ning Ren
RestorationGenerationDiffusion modelScore-based ModelFlow-based ModelNeural Radiance FieldAuto EncoderImage
🎯 What it does: Proposed a two-stage framework called NeuralTO for learning from multi-view images and achieving neural reconstruction and view synthesis of semi-transparent objects.
NeurCADRecon: Neural Representation for Reconstructing CAD Surfaces by Enforcing Zero Gaussian Curvature
Qiujie Dong (Shandong University), Changhe Tu (Shandong University)
GenerationData SynthesisRepresentation LearningDiffusion modelNeural Radiance FieldAuto EncoderContrastive LearningGaussian SplattingPoint CloudMesh
🎯 What it does: Propose a self-supervised neural SDF network, NeurCADRecon, for reconstructing CAD models from unordered low-quality point clouds.
NICER: A New and Improved Consumed Endurance and Recovery Metric to Quantify Muscle Fatigue of Mid-Air Interactions
Yi Li (Monash University), Barrett Ens (University of British Columbia)
Biomedical Data
🎯 What it does: This paper proposes and verifies the NICER model, aimed at accurately quantifying muscle fatigue in aerial gesture interactions.
Object-level Scene Deocclusion
Zhengzhe Liu (Chinese University of Hong Kong), Chi-Wing Fu (Chinese University of Hong Kong)
RestorationSegmentationGenerationDepth EstimationTransformerPrompt EngineeringVision Language ModelDiffusion modelAuto EncoderContrastive LearningImageTextMultimodality
🎯 What it does: Proposes a parallel visible-to-complete diffusion framework called PACO based on self-supervised learning, used for complete reconstruction and recombination of occluded objects in real scenes;
One Noise to Rule Them All: Learning a Unified Model of Spatially-Varying Noise Patterns
Arman Maesumi (Brown University), Daniel Ritchie (Brown University)
GenerationData SynthesisConvolutional Neural NetworkTransformerDiffusion modelScore-based ModelImage
🎯 What it does: This paper trains a unified diffusion model capable of generating multiple types of procedural noise and supporting spatially variable noise mixing and interpolation.
Part123: Part-aware 3D Reconstruction from a Single-view Image
Anran Liu (University of Hong Kong), Wenping Wang (Texas A&M University)
GenerationData SynthesisConvolutional Neural NetworkTransformerDiffusion modelNeural Radiance FieldAuto EncoderContrastive LearningImagePoint CloudMesh
🎯 What it does: This study proposes the Part123 framework, which generates consistent multi-view images using a multi-view diffusion model, performs instance segmentation on each image with SAM, and then achieves part-aware 3D reconstruction of single-view images through contrastive learning-based NeuS neural rendering.
Path-Space Differentiable Rendering of Implicit Surfaces
Siwei Zhou, Shuang Zhao
Diffusion modelNeural Radiance FieldMesh
🎯 What it does: Generalize the theory of differential path integration to implicit geometry (such as isosurfaces and signed distance functions), and propose a new Monte Carlo estimator to efficiently sample implicit discontinuity boundaries.
PEA-PODs: Perceptual Evaluation of Algorithms for Power Optimization in XR Displays
Kenneth Chen, Qi Sun
Optimization
🎯 What it does: Conducted a large-scale perceptual evaluation to assess the visual quality of six XR display power consumption optimization algorithms, and constructed a unified conversion function between JOD and milliwatt power consumption.
Perceptual Evaluation of Steered Retinal Projection
Seungjae Lee, Barry Silverstein
Vision-Language-Action ModelOptical Flow
🎯 What it does: Established the first perceptual experimental platform specifically for Steered Retinal Projection (SRP), and conducted two experiments using this platform, respectively exploring the impact of pupil-directed dynamics on visual suppression and the trade-off between eye tracking and pupil-directed performance; meanwhile, proposed a numerical model for predicting the probability of SRP artifact detection.
Physical Non-inertial Poser (PNP): Modeling Non-inertial Effects in Sparse-inertial Human Motion Capture
Xinyu Yi (Tsinghua University), Feng Xu (Tsinghua University)
Pose EstimationOptimizationRecurrent Neural NetworkSupervised Fine-TuningTime SeriesSequentialPhysics Related
🎯 What it does: Proposed a real-time sparse IMU motion capture method, which for the first time models and compensates virtual forces in a non-inertial root coordinate system, thereby more accurately utilizing acceleration information;
Physics-based Scene Layout Generation from Human Motion
Jianan Li (Chinese University of Hong Kong), Tien-Tsin Wong (Chinese University of Hong Kong)
OptimizationReinforcement LearningDiffusion modelGenerative Adversarial NetworkContrastive LearningVideoPoint CloudMeshPhysics Related
🎯 What it does: Propose a physics-based dual optimization framework (INFERACT), which simultaneously learns a motion imitation controller and a scene layout generator, utilizing physics simulation to achieve natural interaction between given human motions and scene objects.
Physics-Informed Learning of Characteristic Trajectories for Smoke Reconstruction
Yiming Wang, Mengyu Chu
RestorationGenerationOptimizationTransformerDiffusion modelScore-based ModelRectified FlowNeural Radiance FieldAuto EncoderGenerative Adversarial NetworkContrastive LearningOptical FlowImageVideoPhysics RelatedStochastic Differential Equation
🎯 What it does: Recover physically learnable flow fields and NeRF scenes of smoke and obstacles from sparse RGB video perspectives.
Portrait3D: Text-Guided High-Quality 3D Portrait Generation Using Pyramid Representation and GANs Prior
Yiqian Wu (Zhejiang University), Xiaogang Jin (Zhejiang University)
GenerationData SynthesisTransformerPrompt EngineeringDiffusion modelScore-based ModelNeural Radiance FieldAuto EncoderGenerative Adversarial NetworkImageTextMesh
🎯 What it does: Proposes Portrait3D, a text-driven high-quality 3D portrait generation framework based on neural rendering, Pyramid tri-grid 3D representation, and 3DGAN prior.
Position-Based Nonlinear Gauss-Seidel for Quasistatic Hyperelasticity
Yizhou Chen, Joseph Teran
OptimizationGaussian SplattingPhysics Related
🎯 What it does: A position-based nonlinear Gauss-Seidel solver is proposed to address quasi-static hyperelastic problems, improving the convergence and stability of traditional position-based dynamics (PBD) in such problems.
Practical Error Estimation for Denoised Monte Carlo Image Synthesis
A. Firmino, Henrik Wann Jensen
OptimizationComputational Efficiency
🎯 What it does: A practical global error estimation technique is proposed to combine Monte Carlo ray tracing with deep learning-based denoising. Based on the aggregated estimation of bias and variance, the squared error distribution of pixels is obtained, leading to the design of a stopping criterion that terminates rendering when the user-specified error threshold is reached.
Preconditioned Nonlinear Conjugate Gradient Method for Real-time Interior-point Hyperelasticity
Xing Shen (Fuxi AI Lab, NetEase Inc), Tangjie Lv (Fuxi AI Lab, NetEase Inc)
OptimizationDiffusion modelScore-based ModelOptical FlowMeshPhysics RelatedStochastic Differential EquationOrdinary Differential Equation
🎯 What it does: Proposed a Jacobi-preconditioned nonlinear conjugate gradient (PNCG) method for real-time simulation of internal point hyperelastic models.
Primal-Dual Non-Smooth Friction for Rigid Body Animation
Yi-Lu Chen, C. Wojtan
Optimization
🎯 What it does: A new method is proposed to bridge smooth and nonsmooth friction solvers, converting highly constrained nonsmooth friction problems into unconstrained smooth problems via logarithmic barriers, and solving them using an interior-point primal-dual Newton iteration.
Progressive Dynamics for Cloth and Shell Animation
J. Zhang, Danny M. Kaufman
MeshPhysics Related
🎯 What it does: Proposes a Progressive Dynamics method for coarse-to-fine, multi-resolution simulation of physics-driven thin shells and fabric animation.
Proxy Asset Generation for Cloth Simulation in Games
Zhongtian Zheng, Kui Wu
GenerationOptimizationDiffusion modelAuto EncoderGenerative Adversarial NetworkContrastive LearningMesh
🎯 What it does: A set of automated workflow is proposed to convert high-resolution visual meshes into single-layer low-polygon proxy meshes, and through proxy mesh simulation and differential skinning weight optimization, the final high-resolution visual mesh presents a reasonable appearance in simulation.
Proxy Tracing: Unbiased Reciprocal Estimation for Optimized Sampling in BDPT
Fujia Su (Peking University), Sheng Li (Peking University)
OptimizationComputational EfficiencyDiffusion modelNeural Radiance FieldImagePhysics Related
🎯 What it does: A proxy tracing method is proposed, which efficiently samples difficult-to-connect light paths involving high gloss/mirror materials through the 'abandon and replenish' strategy and unbiased reciprocal estimation, thereby improving the sampling efficiency of bidirectional path tracing (BDPT).
QT-Font: High-efficiency Font Synthesis via Quadtree-based Diffusion Models
Yitian Liu, Zhouhui Lian
GenerationData SynthesisGraph Neural NetworkTransformerDiffusion modelAuto EncoderGenerative Adversarial NetworkImage
🎯 What it does: Propose an efficient diffusion model called QT-Font based on a quadtree for few-shot font generation, which includes sparse quadtree glyph representation, a U-net based on a dual quadtree graph network, and a content-aware pooling module.
Quad-Optimized Low-Discrepancy Sequences
V. Ostromoukhov, J. Iehl
Optimization
🎯 What it does: Improved the Sobol sequence, enhancing sample uniformity in continuous two-dimensional and four-dimensional projections, while maintaining practical advantages such as sequentiality, high dimensionality, speed, and memory efficiency.
Radar Fields: Frequency-Space Neural Scene Representations for FMCW Radar
D. Borts, Felix Heide
Autonomous DrivingRepresentation LearningSupervised Fine-TuningDiffusion modelScore-based ModelFlow-based ModelRectified FlowNeural Radiance FieldAuto EncoderMultimodalityPoint CloudTime Series
🎯 What it does: Proposes Radar Fields, a neural scene reconstruction method based on frequency-domain radar measurements, which directly supervises raw FMCW radar FFT data to recover dense 3D occupancy information and can synthesize radar echoes from new viewpoints;
Ray Tracing Harmonic Functions
M. Gillespie, Keenan Crane
Neural Radiance FieldPoint CloudMeshPhysics Related
🎯 What it does: Proposed a Harnack tracing algorithm based on Harnack estimation for visualizing isosurfaces of harmonic functions, including angular value functions and cases with singularities.
Reach for the Arcs: Reconstructing Surfaces from SDFs via Tangent Points
Silvia Sellán, Oded Stein
Diffusion modelScore-based ModelAuto EncoderGaussian SplattingPoint CloudMesh
🎯 What it does: Proposes an algorithm for reconstructing meshes from the signed distance function (SDF) of discrete samples.
Real-Time Hair Rendering with Hair Meshes
Gaurav Bhokare, Cem Yuksel
Mesh
🎯 What it does: Utilize the hair mesh structure to achieve real-time geometry generation on the GPU, combining mesh shaders, texture layouts, and procedural modeling to realize efficient strand-based hair rendering; and provide LOD technology to improve distant rendering performance.
Real-time Neural Woven Fabric Rendering
Xiang Chen (Shandong University), Beibei Wang (Nanjing University)
GenerationData SynthesisComputational EfficiencyConvolutional Neural NetworkDiffusion modelNeural Radiance FieldAuto EncoderContrastive LearningImageMesh
🎯 What it does: Designed a lightweight Encoder-Decoder neural network to achieve multi-scale rendering of various fabric materials and support real-time editing.
Real-Time Path Guiding Using Bounding Voxel Sampling
Haolin Lu, Tzu-Mao Li
OptimizationComputational EfficiencyDiffusion modelScore-based ModelFlow-based ModelRectified FlowNeural Radiance FieldGaussian SplattingPoint CloudMeshPhysics Related
🎯 What it does: Proposes a real-time path guiding method called Voxel Path Guiding (VXPG), which utilizes a spatial radiance voxel data structure to guide the positions of path vertices across all shaded points, improving fitting efficiency under a limited sampling budget.
Real-time Physically Guided Hair Interpolation
J. Hsu, Kui Wu
Physics Related
🎯 What it does: A physics-based hair interpolation method is proposed, which interpolates using internal forces from simulated guide hairs and reconstructs the hair based on a material model.
Real-time Wing Deformation Simulations for Flying Insects
Qiang Chen, Yifan Zuo
🎯 What it does: A real-time wing deformation simulation model based on a skeleton-driven approach was constructed for simulating wing deformation in various insects.
RealFill: Reference-Driven Generation for Authentic Image Completion
Luming Tang (Cornell University), Michael Rubinstein (Google Research)
RestorationGenerationTransformerSupervised Fine-TuningPrompt EngineeringDiffusion modelImageBenchmark
🎯 What it does: Proposed a reference image-based image completion method called RealFill, which personalizes a pre-trained diffusion model using a small number of reference images to fill in missing regions of the target image with real-world scene content.
Recompose Grammars for Procedural Architecture
Niklaus Houska, Matthias Specht
GenerationData SynthesisMesh
🎯 What it does: A new syntax language called Recomp is proposed for architectural procedural modeling, which enhances geometric expression capabilities and shape granularity control by reorganizing shape trees through rule inlining and geometric tagging.
ReFiNe: Recursive Field Networks for Cross-Modal Multi-Scene Representation
Sergey Zakharov (Toyota Research Institute), Rares Ambrus (Toyota Research Institute)
Data SynthesisComputational EfficiencyRepresentation LearningRecurrent Neural NetworkDiffusion modelScore-based ModelFlow-based ModelRectified FlowNeural Radiance FieldAuto EncoderMultimodalityPoint CloudMesh
🎯 What it does: Propose a recursive field network called ReFiNe, which can encode and reconstruct various 3D assets (SDF, SDF+RGB, NeRF) as continuous fields within a single lightweight neural network, and support continuous queries and ray tracing.
Repulsive Shells
Josua Sassen, Keenan Crane
Diffusion modelScore-based ModelOptical FlowMeshStochastic Differential EquationOrdinary Differential Equation
🎯 What it does: A collision-aware shape space framework was developed to automatically avoid interpenetration in geometric operations, supporting non-crossing shape interpolation and motion extrapolation.
RGB↔X: Image decomposition and synthesis using material- and lighting-aware diffusion models
Zheng Zeng (Adobe Research), M. Hašan
Image TranslationRestorationGenerationData SynthesisTransformerSupervised Fine-TuningPrompt EngineeringDiffusion modelContrastive LearningImageTextMultimodalityRetrieval-Augmented Generation
🎯 What it does: Proposes a unified diffusion model framework that can estimate intrinsic channels such as material, normal, roughness, metallic, and illumination from RGB images (RGB→X), as well as synthesize realistic indoor images from these channels (X→RGB), achieving a closed loop between inverse rendering and realistic rendering.
Rip-NeRF: Anti-aliasing Radiance Fields with Ripmap-Encoded Platonic Solids
Junchen Liu (Beihang University), Hao Zhao (Tsinghua University)
GenerationData SynthesisConvolutional Neural NetworkTransformerDiffusion modelNeural Radiance FieldAuto EncoderGaussian SplattingImageMesh
🎯 What it does: Propose a supersampling neural radiance field (Rip-NeRF) based on Ripmap encoding and Platonic Solid Projection, achieving high-fidelity view synthesis.
Robust Containment Queries over Collections of Rational Parametric Curves via Generalized Winding Numbers
Jacob Spainhour, Kenneth Weiss
🎯 What it does: Extended the point-inclusion query method based on the general winding number to support unstructured rational parametric curve sets, while maintaining geometric integrity through adaptive polyline equivalence.
RTG-SLAM: Real-time 3D Reconstruction at Scale using Gaussian Splatting
Zhexi Peng (Zhejiang University), Kun Zhou (Zhejiang University)
Pose EstimationDepth EstimationAutonomous DrivingOptimizationComputational EfficiencyNeural Radiance FieldGaussian SplattingSimultaneous Localization and MappingImageVideoPoint Cloud
🎯 What it does: Proposed a real-time 3D reconstruction system called RTG-SLAM, which uses an RGB-D camera to achieve instant reconstruction in large-scale environments through efficient Gaussian splatting;
S3: Speech, Script and Scene driven Head and Eye Animation
Yifang Pan, Karan Singh
GenerationPose EstimationVision Language ModelDiffusion modelGenerative Adversarial NetworkTextMultimodalityAudio
🎯 What it does: Based on speech, director's script, and 3D scene inputs, automatically generate head and eye animations in character dialogue.
Saccade-Contingent Rendering
Yuna Kwak (Reality Labs Research, Meta), Phillip Guan (Reality Labs Research, Meta)
Computational EfficiencyOptical FlowImageVideo
🎯 What it does: This paper measures the change in visual acuity after saccades through psychophysical experiments, and based on this, proposes and verifies a 'saccadic intermittent rendering' algorithm that can reduce rendering resolution hundreds of milliseconds after a saccade by only detecting saccades. Subsequently, the algorithm was tested on a real 90 ppd virtual reality headset.
Scale-Invariant Monocular Depth Estimation via SSI Depth
S. Mahdi H. Miangoleh (Simon Fraser University), Yağız Aksoy (Simon Fraser University)
Depth EstimationConvolutional Neural NetworkTransformerAuto EncoderGenerative Adversarial NetworkContrastive LearningImage
🎯 What it does: Propose a two-stage framework that first captures scene structure and details with low-resolution and high-resolution SSI depth, and then uses this as prior input to the SI network for high-resolution scale-invariant depth regression.
Scintilla: Simulating Combustible Vegetation for Wildfires
Andrzej Kokosza, Wojciech Pałubicki
Diffusion modelOptical FlowMeshAgriculture RelatedPhysics Related
🎯 What it does: Propose a new wildfire simulation method that simulates dynamic interactions such as convection, combustion, and heat transfer between flames and vegetation, soil, and the atmosphere.
Seamless Parametrization in Penner Coordinates
Ryan Capouellez (New York University), Denis Zorin (New York University)
OptimizationMesh
🎯 What it does: Proposes a conceptually simple and efficient seamless parameterization algorithm, focusing on constructing quadrilateral layouts and texture charts on surfaces, particularly considering parameterization with specified global features.
Self-Supervised Video Defocus Deblurring with Atlas Learning
Lingyan Ruan, Bin Chen
RestorationConvolutional Neural NetworkTransformerDiffusion modelContrastive LearningOptical FlowVideo
🎯 What it does: Propose an end-to-end late defocus correction scheme, which achieves video deblurring and focus restoration by generating sharp hierarchical neural atlases and propagating consistent focus tracking back to video frames.
Semantic Gesticulator: Semantics-Aware Co-Speech Gesture Synthesis
Zeyi Zhang (Peking University), Libin Liu (Peking University)
GenerationData SynthesisPose EstimationRetrievalTransformerLarge Language ModelSupervised Fine-TuningDiffusion modelAuto EncoderVideoTextMultimodalityRetrieval-Augmented GenerationAudio
🎯 What it does: Propose the Semantic Gesticulator framework, achieving semantic co-speech gesture synthesis, capable of generating gestures that are synchronized with the rhythm of speech and semantically aligned.
Semantic Shape Editing with Parametric Implicit Templates
Uday Kusupati, Adrien Kaiser
Diffusion modelScore-based ModelMesh
🎯 What it does: Proposes a semantic shape editing method based on parameterized implicit templates, which can edit 3D triangular meshes and provides template fitting and mesh deformation techniques.
Separate-and-Enhance: Compositional Finetuning for Text-to-Image Diffusion Models
Zhipeng Bao (Carnegie Mellon University), Martial Hebert (Carnegie Mellon University)
GenerationData SynthesisTransformerSupervised Fine-TuningDiffusion modelImageTextMultimodality
🎯 What it does: This paper analyzes the issues of low attention activation and mask overlap in text-image diffusion models, and proposes separation loss and enhancement loss, combining them for fine-tuning to improve the text-image alignment capability for multi-object generation.
Simplicits: Mesh-Free, Geometry-Agnostic Elastic Simulation
Vismay Modi (University of Toronto), David I. W. Levin (University of Toronto)
OptimizationComputational EfficiencyGraph Neural NetworkTransformerDiffusion modelNeural Radiance FieldGaussian SplattingImageTextPoint CloudMeshBiomedical DataComputed TomographyReview/Survey PaperPhysics Related
🎯 What it does: Propose a mesh-free, data-free elastic simulation framework called Simplicits, which uses neural fields to learn skin weights, enabling elastic dynamics simulation on any three-dimensional representation (mesh, point cloud, SDF, NeRF, Gaussian Splats, medical CT, etc.).
Singular Foliations for Knit Graph Design
Rahul Mitra, Edward Chien
GenerationData SynthesisDiffusion modelScore-based ModelGenerative Adversarial NetworkContrastive LearningMeshGraph
🎯 What it does: Building upon a stripe-based knitting planning framework, this paper treats stripe patterns as integral curves within a singular foliation, defines a vector field using a spinning form, and strictly controls its topological structure through linear level set constraints, eliminating any spiral issues in integral curves. Meanwhile, a new 'effective' interpolation method is introduced to enhance the robustness of knitting diagram generation, and the model is extended to surfaces with higher base numbers, achieving automatic singularity pairing through Morse-based cylindrical decomposition.
SketchDream: Sketch-based Text-To-3D Generation and Editing
Feng-Lin Liu, Lin Gao
GenerationData SynthesisDepth EstimationTransformerDiffusion modelScore-based ModelNeural Radiance FieldOptical FlowImageTextMesh
🎯 What it does: By combining hand-drawn sketches with text prompts, SketchDream is proposed to generate and perform local editing of high-quality NeRF 3D models from sketches.
SMEAR: Stylized Motion Exaggeration with ARt-direction
Jean Basset, Pascal Barla
GenerationDiffusion modelOptical FlowMesh
🎯 What it does: Propose an automatic and artist-guided method for generating 3D shadow frames, focusing on elongated intermediate frames along the trajectory.
SMERF: Streamable Memory Efficient Radiance Fields for Real-Time Large-Scene Exploration
Daniel Duckworth (Google DeepMind), Jonathan T. Barron (Google Research)
Computational EfficiencyKnowledge DistillationConvolutional Neural NetworkTransformerNeural Radiance FieldGaussian SplattingOptical FlowImagePoint Cloud
🎯 What it does: Propose SMERF, a hierarchical partitioning real-time radiance field method, which achieves real-time high-quality rendering of large scenes covering hundreds of square meters by utilizing sub-model switching and teacher distillation.
Smooth Bijective Projection in a High-order Shell
Shibo Liu, Xiao-Ming Fu
Mesh
🎯 What it does: Propose a high-order shell structure and achieve smooth bijective attribute transfer, automatically constructing the shell using an internal point strategy.
Soft Pneumatic Actuator Design using Differentiable Simulation
Arvi Gjoka, D. Panozzo
OptimizationRobotic IntelligenceDiffusion modelScore-based Model
🎯 What it does: A pipeline for the computational design of pneumatically actuated soft robots is proposed, which controls the shape of internal cavities through shape optimization and high-order finite element simulations, enabling the robot to achieve user-specified postures or apply user-controlled forces.
Solid Knitting
Yuichi Hirose, James Mccann
🎯 What it does: Introduces a novel solid knitting technology that combines layer-by-layer stacking from 3D printing with the interlacing of traditional knitting. A controllable solid knitting machine and corresponding volumetric design tool were developed, and its feasibility in fabricating rectangular models was verified.
Spatial and Surface Correspondence Field for Interaction Transfer
Zeyu Huang (Shenzhen University), Ruizhen Hu (Shenzhen University)
OptimizationDiffusion modelNeural Radiance FieldContrastive LearningPoint CloudMesh
🎯 What it does: Propose an interactive transfer method that simultaneously utilizes spatial and surface correspondence to automatically transfer existing interactive actions to target objects with similar shapes but different forms.