SIGGRAPH 2024 Papers with AI Summaries
ACM SIGGRAPH (Transactions on Graphics) · 252 papers
→ SIGGRAPH 2024 papers with code (37)
Each paper below shows an AI-generated one-line summary. Get the full 6-part summary (innovation, method, data, results, limitations) and search all 252 SIGGRAPH 2024 papers by keyword, author or institution —
free trial on arXivSub.
2D Gaussian Splatting for Geometrically Accurate Radiance Fields
Binbin Huang (ShanghaiTech University), Shenghua Gao (ShanghaiTech University)
OptimizationComputational EfficiencyRepresentation LearningDiffusion modelNeural Radiance FieldGaussian SplattingImagePoint CloudMesh
🎯 What it does: Propose a scene representation and rendering method based on the 2D Gaussian primitive, which can simultaneously optimize geometry (surface) and appearance in multi-view RGB images, ultimately achieving high-quality novel view synthesis and three-dimensional mesh reconstruction with extremely low noise.
3D Gaussian Blendshapes for Head Avatar Animation
Shengjie Ma (Zhejiang University), Kun Zhou (Zhejiang University)
GenerationData SynthesisDiffusion modelNeural Radiance FieldContrastive LearningGaussian SplattingVideoPoint CloudMesh
🎯 What it does: Proposed a head avatar animation model based on 3D Gaussian Blendshapes, which trains a high-frequency detail rendering system that can be linearly mixed using monocular video
3D Gaussian Splatting with Deferred Reflection
Keyang Ye (Zhejiang University), Kun Zhou (Zhejiang University)
GenerationOptimizationComputational EfficiencyDiffusion modelScore-based ModelNeural Radiance FieldAuto EncoderGaussian SplattingPoint CloudMesh
🎯 What it does: A deferred shading pipeline based on 3D Gaussian scattering is proposed, achieving high-quality specular reflection rendering.
3D-Layers: Bringing Layer-Based Color Editing to VR Painting
Emilie Yu, A. Bousseau
🎯 What it does: Proposes a 3D color editing method based on layers for VR painting, defining 3D-Layers as a layered structure consisting of sub-layers and appearance layers, and implementing it in VR painting applications.
3Doodle: Compact Abstraction of Objects with 3D Strokes
Changwoon Choi (Seoul National University), Young Min Kim (Seoul National University)
GenerationCompressionDiffusion modelNeural Radiance FieldContrastive LearningImagePoint CloudMesh
🎯 What it does: Directly optimize a set of 3D geometric primitives (view-independent cubic Bézier curves and view-dependent super tetrahedron silhouettes) using multi-view images, and generate consistent hand-drawn style sketches across multiple views through fully differentiable rendering;
4D-Rotor Gaussian Splatting: Towards Efficient Novel View Synthesis for Dynamic Scenes
Yuanxing Duan (Peking University), B. Chen
GenerationData SynthesisComputational EfficiencyNeural Radiance FieldGaussian SplattingOptical FlowImageVideo
🎯 What it does: This paper proposes an efficient dynamic scene novel view synthesis method based on 4D rotation operators (Rotor) — 4D-Rotor Gaussian Splatting.
A Construct-Optimize Approach to Sparse View Synthesis without Camera Pose
Kaiwen Jiang (University of California, San Diego), Ravi Ramamoorthi (University of California, San Diego)
GenerationData SynthesisPose EstimationDepth EstimationOptimizationDiffusion modelNeural Radiance FieldContrastive LearningGaussian SplattingOptical FlowImagePoint Cloud
🎯 What it does: Propose a construct-optimization method for sparse view synthesis without camera poses, first constructing a rough scene using monocular depth backprojection to generate a 3D Gaussian representation, and then jointly optimizing the poses and depth under correspondence supervision.
A Differential Monte Carlo Solver For the Poisson Equation
Zihan Yu, Shuang Zhao
Physics RelatedStochastic Differential EquationOrdinary Differential Equation
🎯 What it does: A general differential Monte Carlo method is proposed for solving the Poisson equation with Dirichlet boundary conditions, and a new boundary integral formula is designed to compute derivatives with respect to arbitrary parameters (including domain shape). Based on this formula, an efficient walk-on-spheres (WoS) technique and a new method for estimating the normal derivative of the solution field are developed.
A Dynamic Duo of Finite Elements and Material Points
Xuan Li, Chenfanfu Jiang
Optical FlowPoint CloudMeshPhysics RelatedStochastic Differential EquationOrdinary Differential Equation
🎯 What it does: Proposed a novel method that couples the finite element method (FEM) with the material point method (MPM), utilizing an IMEX time separation strategy to achieve bidirectional coupling between the two methods;
A Free-Space Diffraction BSDF
Shlomi Steinberg, Matt Pharr
Diffusion modelMeshPhysics Related
🎯 What it does: Proposed an efficient free-space diffraction simulation method based on the edge-based Fraunhofer diffraction formula and dynamically constructed BSDF
A Fully-correlated Anisotropic Micrograin BSDF Model
Simon Lucas, Pascal Barla
Diffusion modelNeural Radiance FieldPhysics Related
🎯 What it does: An improved particle BSDF model is proposed for rendering anisotropic porous layers, and a fully correlated geometric attenuation factor (GAF) is derived, which is used to determine the mixing weights between the porous layer and the base layer BSDF.
A Heat Method for Generalized Signed Distance
Nicole Feng, Keenan Crane
OptimizationDiffusion modelScore-based ModelMesh
🎯 What it does: Propose a SDF approximation method based on heat diffusion, capable of handling geometries with holes, noise, or self-intersections.
A Hierarchical 3D Gaussian Representation for Real-Time Rendering of Very Large Datasets
Bernhard Kerbl (Inria), G. Drettakis
Autonomous DrivingComputational EfficiencyRepresentation LearningNeural Radiance FieldGaussian SplattingImagePoint Cloud
🎯 What it does: This paper proposes a hierarchical 3D Gaussian representation, combining block-wise training and level-of-detail control, which enables real-time rendering of thousands of images and scenes spanning several kilometers on a GPU.
A Linear Method to Consistently Orient Normals of a 3D Point Cloud
C. Gotsman, Kai Hormann
OptimizationComputational EfficiencyContrastive LearningPoint CloudStochastic Differential EquationOrdinary Differential Equation
🎯 What it does: Designed a linear method to consistently orient the normals of point clouds, applicable to 3D point clouds with noise and non-smooth features.
A Neural Network Model for Efficient Musculoskeletal-Driven Skin Deformation
Yushan Han, Joseph Teran
Computational EfficiencyRobotic IntelligenceDiffusion modelAuto EncoderGenerative Adversarial NetworkContrastive LearningMeshBiomedical Data
🎯 What it does: A neural network model was developed for efficiently simulating skin and soft tissue deformation driven by the human musculoskeletal system, providing kinematic and active correction of linear blend skinning.
A Realistic Multi-scale Surface-based Cloth Appearance Model
Junqiu Zhu, Matt Jen-Yuan Chiang
GenerationData SynthesisComputational EfficiencyDiffusion modelAuto EncoderContrastive LearningGaussian SplattingMesh
🎯 What it does: A multi-scale, surface-based realistic fabric appearance model is proposed.
A Unified Differentiable Boolean Operator with Fuzzy Logic
H. Liu (Roblox), Victor Zordan (Roblox)
GenerationData SynthesisOptimizationDiffusion modelScore-based ModelAuto EncoderContrastive LearningPoint CloudMesh
🎯 What it does: Proposed a unified differentiable Boolean operator based on fuzzy logic, which can simultaneously optimize the type of Boolean operations and geometric primitives with continuous gradients in Constructive Solid Geometry (CSG);
A Vortex Particle-on-Mesh Method for Soap Film Simulation
Ningxiao Tao, Bao Chen
Optical FlowMeshPhysics Related
🎯 What it does: A novel vortex particle-grid method for soap film simulation is proposed.
Accelerating Saccadic Response through Spatial and Temporal Cross-Modal Misalignments
Daniel Jiménez Navarro, Ana Serrano
Autonomous DrivingOptimizationRobotic IntelligenceReinforcement Learning from Human FeedbackVision-Language-Action ModelDiffusion modelContrastive LearningOptical FlowImageVideoMultimodalityAgriculture RelatedAudio
🎯 What it does: Investigate the effect of auditory cues presented before visual stimuli on saccadic latency in a virtual reality environment, and verify its application in basketball training tasks and interactive farm games.
Adaptive grid generation for discretizing implicit complexes
Yiwen Ju, Tao Ju
Mesh
🎯 What it does: Generate adaptive triangular or tetrahedral meshes to discretize implicit composite shapes defined by vector functions (including implicit surface intersections, CSG, material interfaces, and curve networks)
Alignment conditions for NURBS-based design of mixed tension-compression grid shells
Masaaki Miki, Toby Mitchell
OptimizationDiffusion modelScore-based ModelAuto EncoderGenerative Adversarial NetworkContrastive LearningMeshPhysics Related
🎯 What it does: This study extends the variable projection method by incorporating alignment conditions into NURBS basis design, enabling the generation of planar panels with no bending for hybrid tension-compression grid shells, and provides several practical design cases.
An Induce-on-Boundary Magnetostatic Solver for Grid-Based Ferrofluids
Xingyu Ni, Bao Chen
Point CloudPhysics Related
🎯 What it does: Proposed and implemented an Induce-on-Boundary (IoB) solver based on a single-layer potential for solving the magnetostatic equations of magnetic fluids.
Analogist: Out-of-the-box Visual In-Context Learning with Image Diffusion Model
Zheng Gu (City University of Hong Kong), Yang Gao (Nanjing University)
Image TranslationImage HarmonizationRestorationGenerationTransformerPrompt EngineeringVision Language ModelDiffusion modelAuto EncoderContrastive LearningImageText
🎯 What it does: Propose Analogist, a zero-shot visual context learning method based on the pre-trained Stable Diffusion inpainting model, which achieves image analogy generation through 2×2 grid tiling, visual self-attention cloning (SAC), and text prompt generation (GPT‑4V + CAM).
AONeuS: A Neural Rendering Framework for Acoustic-Optical Sensor Fusion
Mohamad Qadri (Carnegie Mellon University), Christopher A. Metzler (University of Maryland)
GenerationDepth EstimationOptimizationComputational EfficiencyDiffusion modelScore-based ModelNeural Radiance FieldAuto EncoderGenerative Adversarial NetworkContrastive LearningSimultaneous Localization and MappingOptical FlowImageMultimodalityPoint CloudMeshUltrasoundAudio
🎯 What it does: Propose an AONeuS framework based on inverse differentiable rendering, which integrates underwater camera and imaging sonar data to achieve high-resolution 3D surface reconstruction under constrained baseline conditions.
Aperture-Aware Lens Design
Arjun Teh, Matthew O'Toole
OptimizationDiffusion modelOptical FlowPhysics Related
🎯 What it does: A gradient calculation method for optical systems was developed, which can simultaneously optimize focusing performance and light throughput;
Area ReSTIR: Resampling for Real-Time Defocus and Antialiasing
Song Zhang, Chris Wyman
Reinforcement LearningOptical Flow
🎯 What it does: Propose Area ReSTIR, which achieves more efficient path reuse, especially in high-frequency detail regions, by expanding the ReSTIR repository to integrate the 4D ray space of pixels (including 2D regions on the film and lens).
Audio Matters Too! Enhancing Markerless Motion Capture with Audio Signals for String Performance Capture
Yitong Jin (Central Conservatory of Music), Qionghai Dai (Tsinghua University)
Pose EstimationOptimizationConvolutional Neural NetworkTransformerDiffusion modelScore-based ModelContrastive LearningOptical FlowVideoMultimodalityPoint CloudMeshAudio
🎯 What it does: Proposed the first large-scale multi-modal string performance dataset (String Performance Dataset, SPD), and designed an audio-guided unlabeled motion capture framework based on this dataset, which can accurately recover finger positions on the strings through audio (pitch) information, achieving fine-grained hand motion capture;
Automatic Digital Garment Initialization from Sewing Patterns
Chen Liu, Huamin Wang
ClassificationOptimizationImage
🎯 What it does: Developed an automated system that converts digital sewing patterns into well-fitted garments on a human body model, significantly reducing user intervention.
Biharmonic Coordinates and their Derivatives for Triangular 3D Cages
Jean-Marc Thiery, Jiong Chen
Mesh
🎯 What it does: This paper derives closed-form expressions for the second-order harmonic coordinates and their derivatives of a three-dimensional triangular cage.
Bilateral Guided Radiance Field Processing
Yuehao Wang (Chinese University of Hong Kong), Tianfan Xue (Chinese University of Hong Kong and Shanghai AI Laboratory)
RestorationDiffusion modelNeural Radiance FieldAuto EncoderImageMesh
🎯 What it does: A bilateral grid-based radiance field processing method is proposed. During the NeRF training phase, a differentiable 3D bilateral grid is used to separate the camera post-processing differences across views. Subsequently, a low-rank 4D bilateral grid is utilized to migrate 2D edits from a single view to a 3D scene, achieving a complete process without floating points and with enhanced visualization.
Binary Opacity Grids: Capturing Fine Geometric Detail for Mesh-Based View Synthesis
C. Reiser, Andreas Geiger
GenerationComputational EfficiencyDiffusion modelNeural Radiance FieldAuto EncoderGaussian SplattingOptical FlowImageMesh
🎯 What it does: Propose a surface reconstruction method based on binary opacity grids, which can capture fine geometric structures such as leaves, branches, and grass in multi-view images, and generate triangle meshes that can be rendered in real-time on mobile phones.
BlockFusion: Expandable 3D Scene Generation using Latent Tri-plane Extrapolation
Zhennan Wu (University of Tokyo), Pan Ji (Tencent XR Vision Labs)
GenerationData SynthesisConvolutional Neural NetworkTransformerDiffusion modelScore-based ModelNeural Radiance FieldAuto EncoderGenerative Adversarial NetworkImageTextPoint CloudMesh
🎯 What it does: Generate infinitely expandable 3D scenes using a block-based latent diffusion model (BlockFusion), where the model generates scenes in blocks and achieves seamless expansion through latent tri-plane extrapolation.
Blue noise for diffusion models
Xingchang Huang (MPI Informatics), Gurprit Singh (MPI Informatics)
GenerationConvolutional Neural NetworkTransformerDiffusion modelScore-based ModelRectified FlowImage
🎯 What it does: Propose using time-varying correlated noise (especially blue noise) in deterministic diffusion models to improve image generation quality.
BoostMVSNeRFs: Boosting MVS-based NeRFs to Generalizable View Synthesis in Large-scale Scenes
Chih-Hai Su (National Yang Ming Chiao Tung University), Yu-Lun Liu (National Yang Ming Chiao Tung University)
GenerationData SynthesisNeural Radiance FieldContrastive LearningGaussian SplattingOptical FlowImagePoint Cloud
🎯 What it does: Propose BoostMVSNeRFs, which significantly improves the quality of view synthesis for large-scale scenes by combining multiple cost volumes during the rendering process of MVS-based NeRF and introducing visual scores.
BrepGen: A B-rep Generative Diffusion Model with Structured Latent Geometry
Xiang Xu (Simon Fraser University), Yasutaka Furukawa (Simon Fraser University)
GenerationData SynthesisTransformerDiffusion modelAuto EncoderMeshGraph
🎯 What it does: By unifying the geometry and topology of B-rep into a structured latent tree and using a Transformer-based diffusion model for layer-by-layer denoising, high-quality B-rep CAD models are directly generated.
Capacitive Touch Sensing on General 3D Surfaces
Gianpaolo Palma, Paolo Cignoni
Mesh
🎯 What it does: Propose an automated method to generate uniformly distributed transmitter and receiver electrode grids on any three-dimensional surface to achieve high-resolution, multi-touch; optimize electrode layout through quadrilateral meshing, bilateral graph algorithm, simplification and clustering algorithms; further plan surface grooves and internal pipelines to minimize the number of touch controllers and IO pins, and fabricate a prototype through 3D printing;
Categorical Codebook Matching for Embodied Character Controllers
Sebastian Starke, Yuting Ye
GenerationRobotic IntelligenceTransformerDiffusion modelAuto EncoderGenerative Adversarial NetworkSequential
🎯 What it does: Proposed a generative framework that realizes real-time mapping from sparse sensor signals to the motion of a complete virtual body avatar, preserving the user's motion context.
CharacterGen: Efficient 3D Character Generation from Single Images with Multi-View Pose Canonicalization
Haoyue Peng, Shi-Min Hu (Tsinghua University)
GenerationData SynthesisPose EstimationConvolutional Neural NetworkTransformerPrompt EngineeringDiffusion modelImageMesh
🎯 What it does: Generate high-quality, A-pose 3D character meshes with texture maps from a single 2D image with arbitrary pose.
Ciallo: GPU-Accelerated Rendering of Vector Brush Strokes
Shen Ciao, Zeyu Wang
Image
🎯 What it does: Developed a GPU-based rendering technology, constructing a system that can simultaneously accommodate raster and vector brush performances in digital painting and animation, and implemented an open-source drawing prototype with vector fill functionality.
CLAY: A Controllable Large-scale Generative Model for Creating High-quality 3D Assets
Longwen Zhang (ShanghaiTech University), Jingyi Yu (ShanghaiTech University)
GenerationData SynthesisTransformerLarge Language ModelPrompt EngineeringVision Language ModelDiffusion modelAuto EncoderContrastive LearningImageTextMultimodalityPoint CloudMeshRetrieval-Augmented Generation
🎯 What it does: CLAY proposes a large-scale, controllable 3D asset generation model that can generate high-quality geometry and PBR materials based on multimodal inputs such as text, images, point clouds, and voxels, and supports various control methods;
CNS-Edit: 3D Shape Editing via Coupled Neural Shape Optimization
Jingyu Hu (Chinese University of Hong Kong), Chi-Wing Fu (Chinese University of Hong Kong)
GenerationData SynthesisOptimizationDiffusion modelScore-based ModelAuto EncoderPoint CloudMesh
🎯 What it does: Propose a coupled neural shape representation and optimization framework that supports editing 3D shapes through copy, scale, delete, and drag operations.
Coin3D: Controllable and Interactive 3D Assets Generation with Proxy-Guided Conditioning
Wenqi Dong (Zhejiang University), Zhaopeng Cui (Zhejiang University)
GenerationData SynthesisConvolutional Neural NetworkTransformerPrompt EngineeringDiffusion modelScore-based ModelNeural Radiance FieldAuto EncoderImageTextMultimodalityPoint CloudMesh
🎯 What it does: Proposes a controllable and interactive 3D asset generation framework called Coin3D, which utilizes user input through simple geometric proxies and text prompts to control the multi-view diffusion process, enabling local part editing and second-level preview.
ColorVideoVDP: A visual difference predictor for image, video and display distortions
Rafał K. Mantiuk (University of Cambridge), Alexandre Chapiro (Reality Labs)
OptimizationComputational EfficiencyConvolutional Neural NetworkDiffusion modelScore-based ModelFlow-based ModelRectified FlowAuto EncoderContrastive LearningOptical FlowImageVideoBenchmark
🎯 What it does: Proposed and implemented ColorVideoVDP, a full-reference video and image quality assessment metric based on a visual perception model, which can simultaneously consider the visible differences in chrominance, temporal, and spatial frequency domains.
Compressed Skinning for Facial Blendshapes
L. Kavan, Doug Roble (Meta)
CompressionOptimizationComputational EfficiencyAuto EncoderMesh
🎯 What it does: This paper proposes an algorithm for compressing linear blended skinning, used to convert facial blendshapes into efficient skinning representations.
Computational Homogenization for Inverse Design of Surface-based Inflatables
Yingying Ren, Mark Pauly
OptimizationMesh
🎯 What it does: A computational framework for analyzing and designing surface-based inflatable structures was developed, supporting arbitrary seam curve patterns and generating a behavioral database of inflatable patches through numerical averaging; this database was used to construct an inverse design algorithm, achieving approximation of target surfaces and optimization of structural performance.
Computational Illusion Knitting
Amy Zhu, Adriana Schulz
GenerationData SynthesisDiffusion modelAuto EncoderImageMesh
🎯 What it does: The study investigates and proposes a fundamental method for computer-aided design and manufacturing of illusionary weaving, clarifies design space constraints, proposes a strategy for weavable illusion patterns, develops a partially automated interactive design tool, and introduces a new algorithm to mitigate manufacturing failures in multi-color mixing and texture.
Conditional Mixture Path Guiding for Differentiable Rendering
Zhimin Fan, Jie Guo
Data SynthesisOptimizationComputational EfficiencyMixture of ExpertsDiffusion modelScore-based ModelNeural Radiance FieldImage
🎯 What it does: Proposed and implemented conditional hybrid path guiding, targeting ray path sampling in differentiable rendering, significantly reducing the variance of Monte Carlo estimation through real-time computation of optimal weights.
Consistent Point Orientation for Manifold Surfaces via Boundary Integration
Weizhou Liu (Beijing Normal University), Ying He (Nanyang Technological University)
OptimizationComputational EfficiencyPoint Cloud
🎯 What it does: This paper proposes an algorithm based on boundary energy maximization, which gradually restores the global consistent normals of point clouds under a random method.
Contact detection between curved fibres: high order makes a difference
Octave Crespel, F. Bertails-Descoubes
Diffusion modelScore-based ModelOptical FlowFibre Orientation DistributionPhysics RelatedStochastic Differential EquationOrdinary Differential Equation
🎯 What it does: Analyze the impact of low-order geometry on contact forces in fiber systems, identify the pseudo-jump phenomenon, and propose a high-order curve-to-curve contact detection scheme to eliminate these artifacts.
ContourCraft: Learning to Resolve Intersections in Neural Multi-Garment Simulations
Artur Grigorev (ETH Zurich), Bernhard Thomaszewski (ETH Zurich)
OptimizationRobotic IntelligenceGraph Neural NetworkDiffusion modelContrastive LearningMeshGraph
🎯 What it does: Developed a ClothCraft method based on graph neural networks to learn solving fabric intersection problems in neural multi-cloth simulations.
Controllable Neural Style Transfer for Dynamic Meshes
Guilherme Gomes Haetinger, Vinicius C. Azevedo
GenerationGraph Neural NetworkDiffusion modelGenerative Adversarial NetworkContrastive LearningMesh
🎯 What it does: Propose a controllable neural style transfer technique for dynamic grids, achieving high-quality generation of large-scale grid deformation and style by improving the Gram-Matrix loss into a neural neighborhood loss, utilizing Laplace-Beltrami implicit reparameterization, and employing coarse-to-fine level style transfer.
Creating LEGO Figurines from Single Images
Jiahao Ge, Chi-Wing Fu
Image TranslationRestorationGenerationConvolutional Neural NetworkTransformerPrompt EngineeringVision Language ModelDiffusion modelImageTextRetrieval-Augmented Generation
🎯 What it does: Based on user-provided portrait photographs, a three-stage pipeline is constructed to automatically generate assemblable physical LEGO detailed character models.
Cricket: A Self-Powered Chirping Pixel
Shree K. Nayar (Columbia University), M. Fridberg
Autonomous DrivingOptimizationFederated LearningRobotic IntelligenceDiffusion modelScore-based ModelNeural Radiance FieldContrastive LearningGaussian SplattingOptical FlowImageVideoPoint CloudTime SeriesPhysics Related
🎯 What it does: Developed a battery-free, self-powered photodetector (Cricket) that collects light energy through photovoltaic cells and wirelessly encodes light intensity by emitting short and strong RF chirps during charging intervals;
Cross-Image Attention for Zero-Shot Appearance Transfer
Yuval Alaluf (Tel Aviv University), D. Cohen-Or
Image TranslationGenerationData SynthesisTransformerDiffusion modelAuto EncoderContrastive LearningImageText
🎯 What it does: Leveraging the self-attention mechanism of pre-trained text-to-image diffusion models, we propose Cross-Image Attention to implicitly establish correspondence between two semantically different images, and during the denoising process, integrate the keys and values from the appearance image to achieve structure-appearance zero-shot transfer; simultaneously, three enhancement techniques are introduced, including attention contrast, classifier-free guidance, and AdaIN alignment, to improve image quality.
Curvature-Driven Conformal Deformations
E. Corman
OptimizationMesh
🎯 What it does: A new method for computing conformal deformations in three-dimensional space was developed, achieving surface deformation by minimizing curvature-based energy.
CWF: Consolidating Weak Features in High-quality Mesh Simplification
Rui Xu (Shandong University), Changhe Tu (Shandong University)
OptimizationDiffusion modelScore-based ModelMesh
🎯 What it does: Propose a unified energy function that simultaneously maintains accuracy, triangle quality, and alignment of strong and weak features during mesh simplification.
Cybersickness Reduction via Gaze-Contingent Image Deformation
Colin Groth, P. Didyk
Optical FlowImageVideo
🎯 What it does: A technique is proposed to reduce vection and alleviate internet addiction by subtly adjusting screen space velocity in the user's peripheral vision and introducing minor geometric distortions, followed by the implementation of a real-time post-processing step that can be integrated into existing rendering frameworks.
Cyclogenesis: Simulating Hurricanes and Tornadoes
Jorge Alejandro Amador Herrera, D. Michels
Diffusion modelScore-based ModelOptical FlowPhysics RelatedStochastic Differential EquationOrdinary Differential Equation
🎯 What it does: Propose a physics-based three-dimensional method to simulate cyclone formation, capable of simulating various phenomena such as hurricanes and tornadoes.
DAE-Net: Deforming Auto-Encoder for fine-grained shape co-segmentation
Zhiqin Chen (Adobe Research), Hao Zhang (Simon Fraser University)
SegmentationConvolutional Neural NetworkDiffusion modelScore-based ModelFlow-based ModelAuto EncoderContrastive LearningPoint CloudMesh
🎯 What it does: Propose an unsupervised 3D shape co-segmentation method called DAE-Net, which reconstructs each shape by learning deformable part templates and assembling them in needed subsets, achieving fine-grained and semantically consistent segmentation;
Deep Fourier-based Arbitrary-scale Super-resolution for Real-time Rendering
Haonan Zhang, Yanwen Guo
Super ResolutionDiffusion modelNeural Radiance FieldAuto EncoderImage
🎯 What it does: Supports arbitrary-scale super-resolution reconstruction after training.
Deep Hybrid Camera Deblurring for Smartphone Cameras
Jaesung Rim (POSTECH), Sunghyun Cho (POSTECH)
RestorationConvolutional Neural NetworkTransformerAuto EncoderContrastive LearningOptical FlowImage
🎯 What it does: Proposes an end-to-end learning framework for image deblurring using the dual cameras of a smartphone (wide-angle long-exposure image and ultra-wide short-exposure burst image).
Deep Sketch Vectorization via Implicit Surface Extraction
Chuan Yan, Y. Gingold
Image TranslationRestorationConvolutional Neural NetworkTransformerDiffusion modelAuto EncoderGenerative Adversarial NetworkContrastive LearningImageMesh
🎯 What it does: Propose a sketch vectorization algorithm based on implicit surface extraction, achieved through a two-stage neural network and dual contour domain post-processing;
DiffCAD: Weakly-Supervised Probabilistic CAD Model Retrieval and Alignment from an RGB Image
Daoyi Gao (Technical University of Munich), Angela Dai (Technical University of Munich)
Data SynthesisPose EstimationDepth EstimationRetrievalConvolutional Neural NetworkTransformerDiffusion modelAuto EncoderGenerative Adversarial NetworkContrastive LearningImagePoint CloudMesh
🎯 What it does: Proposed a weakly supervised probabilistic CAD model retrieval and alignment method called DiffCAD, which generates multiple hypotheses of scene scale, object pose, and shape from a single RGB image using a diffusion model, achieving CAD model retrieval and alignment.
Differentiable Geodesic Distance for Intrinsic Minimization on Triangle Meshes
Yue Li (ETH Zürich), Stelian Coros (ETH Zürich)
OptimizationDiffusion modelScore-based ModelOptical FlowMeshStochastic Differential EquationOrdinary Differential Equation
🎯 What it does: This paper proposes a differentiable geodesic distance framework on triangular meshes, utilizing geodesic length as the objective function to achieve intrinsic optimization, including problems such as elastic curve networks, elastic triangular membranes, bidirectional coupling, and geodesic Voronoi diagrams.
Differentiable Voronoi Diagrams for Simulation of Cell-Based Mechanical Systems
Logan Numerow (ETH Zürich), Bernhard Thomaszewski (ETH Zürich)
OptimizationComputational EfficiencyDiffusion modelScore-based ModelOptical FlowImagePoint CloudMeshBenchmarkPhysics RelatedStochastic Differential EquationOrdinary Differential Equation
🎯 What it does: Implicitly model cell-level mechanical systems using differentiable Voronoi diagrams (power diagrams), where a single Voronoi site represents each cell, enabling continuous topological changes and cell deformations;
DiffPoseTalk: Speech-Driven Stylistic 3D Facial Animation and Head Pose Generation via Diffusion Models
Zhiyao Sun (Tsinghua University), Yongjin Liu
GenerationData SynthesisPose EstimationTransformerDiffusion modelScore-based ModelAuto EncoderContrastive LearningVideoMeshAudio
🎯 What it does: DiffPoseTalk proposes a system based on diffusion models to generate 3D facial animation and head pose driven by speech, and achieves controllable style control through speaking style codes extracted from short videos.
DiffSound: Differentiable Modal Sound Rendering and Inverse Rendering for Diverse Inference Tasks
Xutong Jin (Peking University), Sheng Li (Peking University)
OptimizationConvolutional Neural NetworkRecurrent Neural NetworkTransformerDiffusion modelScore-based ModelNeural Radiance FieldAuto EncoderGenerative Adversarial NetworkContrastive LearningPoint CloudMeshTime SeriesPhysics RelatedAudio
🎯 What it does: A differentiable modal sound rendering framework called DiffSound was constructed, which can inversely infer material parameters, geometric shapes (thickness, details), and impact positions and amplitudes from real or synthesized audio signals.
Diffusion Illusions: Hiding Images in Plain Sight
R. Burgert, M. Ryoo
GenerationData SynthesisTransformerPrompt EngineeringDiffusion modelScore-based ModelImageTextStochastic Differential Equation
🎯 What it does: This paper proposes the Diffusion Illusions pipeline, which automatically generates original images that can produce illusion effects under different arrangements using a frozen text-to-image diffusion model and differentiable compositional operations, and realizes physical printing.
Diffusion Texture Painting
Anita Hu, Maria Shugrina
Image TranslationGenerationConvolutional Neural NetworkTransformerPrompt EngineeringDiffusion modelScore-based ModelImageMesh
🎯 What it does: Using 2D generative diffusion models for interactive texture painting, supporting artists to apply arbitrarily complex image textures in real-time seamless brush strokes and realistic texture transitions on 3D mesh surfaces.
DiLightNet: Fine-grained Lighting Control for Diffusion-based Image Generation
Chong Zeng (Zhejiang University), Xin Tong (Microsoft Research Asia)
Image TranslationImage HarmonizationGenerationConvolutional Neural NetworkTransformerPrompt EngineeringDiffusion modelScore-based ModelNeural Radiance FieldAuto EncoderImageTextMultimodality
🎯 What it does: Propose a three-stage diffusion-based image generation process that achieves fine-grained control over the lighting of foreground objects, given text prompts and target illumination, and completes consistency reconstruction with the background.
Direct-a-Video: Customized Video Generation with User-Directed Camera Movement and Object Motion
Shiyuan Yang (City University of Hong Kong), Jing Liao (City University of Hong Kong)
GenerationData SynthesisTransformerPrompt EngineeringDiffusion modelContrastive LearningOptical FlowVideoText
🎯 What it does: Developed Direct‑a‑Video, a text-to-video generation framework based on diffusion models, which allows users to separately specify camera motion and object motion, enabling customized video generation.
Dragon's Path: Synthesizing User-Centered Flying Creature Animation Paths for Outdoor Augmented Reality Experiences
Minyoung Kim, Lap-Fai Yu
GenerationData SynthesisPrompt EngineeringDiffusion modelGenerative Adversarial NetworkImageVideoMultimodality
🎯 What it does: This paper proposes a method for automatically generating user-centered flight biological animation paths for outdoor augmented reality experiences;
DreamFont3D: Personalized Text-to-3D Artistic Font Generation
Xiang Li, Xiangxu Meng
GenerationTransformerVision Language ModelDiffusion modelAuto EncoderGenerative Adversarial NetworkTextMesh
🎯 What it does: Proposed a text-to-3D artistic font generation method called DreamFont3D
DreamMat: High-quality PBR Material Generation with Geometry- and Light-aware Diffusion Models
Yuqing Zhang (State Key Lab Of Cad Cg Zhejiang University), Xiaogang Jin (State Key Lab Of Cad Cg Zhejiang University)
GenerationData SynthesisTransformerPrompt EngineeringDiffusion modelScore-based ModelImageTextMesh
🎯 What it does: Use a geometry and illumination-aware diffusion model to generate high-quality PBR materials on textureless meshes with text prompts;
DressCode: Autoregressively Sewing and Generating Garments from Text Guidance
Kai He (ShanghaiTech University), Lan Xu (ShanghaiTech University)
GenerationData SynthesisTransformerLarge Language ModelPrompt EngineeringVision Language ModelDiffusion modelImageTextMesh
🎯 What it does: Propose the DressCode framework, achieving 3D garment generation based on text interaction, capable of automatically generating sewing patterns and physical rendering textures usable for computer graphics (CG).
EASI-Tex: Edge-Aware Mesh Texturing from Single Image
Sai Raj Kishore Perla (Simon Fraser University), Hao Zhang (Simon Fraser University)
GenerationData SynthesisTransformerPrompt EngineeringDiffusion modelImageMeshRetrieval-Augmented Generation
🎯 What it does: Propose a single-image edge-aware 3D mesh texturing method called EASI-Tex, based on Stable Diffusion, ControlNet, and IP-Adapter, which can transfer textures from any single image to any 3D mesh without optimization.
Efficient Debris-flow Simulation for Steep Terrain Erosion
Aryamaan Jain, G. Cordonnier
Physics Related
🎯 What it does: A new method is proposed that utilizes a mathematical formulation of debris flow erosion based on geomorphology and a unified GPU algorithm to simulate the interaction between debris flows and river erosion, and an approximate flow direction algorithm is designed to estimate water flow paths in newly formed depressions.
Efficient Position-Based Deformable Colon Modeling for Endoscopic Procedures Simulation
Marcelo Gomes Martins, Anderson Maciel
Computational EfficiencyRobotic IntelligenceBiomedical Data
🎯 What it does: Propose a colon model based on XPBD, combining Cosserat rod constraints and tetrahedral meshes, which can maintain the overall shape of the colon, and design a real-time tubular structure contact detection algorithm for endoscopic operation simulation.
Eulerian-Lagrangian Fluid Simulation on Particle Flow Maps
Junwei Zhou (University of Michigan), Bo Zhu (Georgia Institute of Technology)
Optical FlowPoint CloudBenchmarkPhysics RelatedOrdinary Differential Equation
🎯 What it does: This paper proposes an Eulerian-Lagrangian fluid simulation method based on Particle Flow Map (PFM), which utilizes the precise flow graph naturally formed by forward particle trajectories to achieve long-range non-dissipative turbulent transport, and solves incompressible fluids through an impulse flow model;
EyeIR: Single Eye Image Inverse Rendering In the Wild
S. Liang, Feng Lu
Image TranslationRestorationGenerationData SynthesisDiffusion modelNeural Radiance FieldAuto EncoderGenerative Adversarial NetworkContrastive LearningImage
🎯 What it does: Propose a method to inverse render monocular images into chromaticity, shading, specular, normal vectors, and illumination, and achieve decomposition by synthesizing into a real data adaptation framework and region-aware self-supervised loss.
Fabric Tessellation: Realizing Freeform Surfaces by Smocking
Aviv Segall, O. Sorkine-Hornung
Diffusion modelOptical FlowMesh
🎯 What it does: Proposes a method to achieve free-form surfaces by using flat fabric through sewing points.
Fabricable 3D Wire Art
Kenji Tojo, Nobuyuki Umetani
GenerationOptimizationTransformerVision Language ModelDiffusion modelAuto EncoderGenerative Adversarial NetworkContrastive LearningImageTextMesh
🎯 What it does: This work proposes a computational method for automatically generating manufacturable 3D wire sculpture models, and designs a printable fixture structure to realize the generated wire paths in the manufacturing process;
Factorized Motion Fields for Fast Sparse Input Dynamic View Synthesis
Nagabhushan Somraj (Indian Institute of Science Bengaluru), R. Soundararajan
Data SynthesisExplainability and InterpretabilityComputational EfficiencyNeural Radiance FieldGaussian SplattingOptical FlowImageVideo
🎯 What it does: This paper proposes a fast sparse input dynamic view synthesis method, utilizing explicit motion fields and factorized voxel models.
Filter-Guided Diffusion for Controllable Image Generation
Zeqi Gu (Cornell Tech), Abe Davis (Cornell University)
Image TranslationGenerationPrompt EngineeringDiffusion modelImageTextStochastic Differential Equation
🎯 What it does: Propose Filter-Guided Diffusion (FGD), a training-free black-box method, which inserts a fast filtering step into the diffusion iteration to achieve controllable guidance on image structure, thus completing text-guided image generation and image-to-image translation.
Flexible Motion In-betweening with Diffusion Models
S. Cohan, M. V. D. Panne
GenerationPose EstimationGraph Neural NetworkTransformerPrompt EngineeringDiffusion modelScore-based ModelVideoTextSequential
🎯 What it does: Proposes CondMDI, a unified diffusion model for generating high-quality, diverse human motion sequences under the condition of given sparse spatiotemporal keyframes and optional text prompts.
FlexScale: Modeling and Characterization of Flexible Scaled Sheets
Juan Sebastian Montes Maestre, Bernhard Thomaszewski
SegmentationDiffusion modelScore-based ModelFlow-based ModelRectified FlowNeural Radiance FieldAuto EncoderGenerative Adversarial NetworkContrastive LearningGaussian SplattingSimultaneous Localization and MappingWorld ModelOptical FlowPhysics RelatedStochastic Differential EquationOrdinary Differential Equation
🎯 What it does: A computational method for flexible scale materials is proposed, which refines original-level simulation data into a macroscopic mechanical model through contact-aware homogenization technology, and this model is used to explore the morphological scale pattern space, revealing its anisotropy and nonlinear material behavior;
Fluid Control with Laplacian Eigenfunctions
Yixin Chen, Timothy Langlois
OptimizationDiffusion modelOptical FlowPhysics Related
🎯 What it does: Propose a physics-driven fluid control pipeline based on Laplacian eigenfunctions, which can achieve real-time fluid simulation, editing, control, and optimal animation generation.
From microfacets to participating media: A unified theory of light transport with stochastic geometry
D. Seyb, Wojciech Jarosz
Optical FlowPhysics RelatedStochastic Differential Equation
🎯 What it does: Proposed and derived the theory of light transport based on random implicit surfaces, unifying various random geometric models such as deterministic geometry, microfacets, and participating media.
fVDB : A Deep-Learning Framework for Sparse, Large Scale, and High Performance Spatial Intelligence
Francis Williams (NVIDIA Research), Ken Museth (NVIDIA Research)
Autonomous DrivingOptimizationComputational EfficiencyConvolutional Neural NetworkTransformerDiffusion modelNeural Radiance FieldGaussian SplattingPoint CloudMesh
🎯 What it does: Propose fVDB, a GPU-accelerated sparse 3D deep learning framework based on IndexGrid, capable of handling large-scale data such as point clouds, meshes, and voxels.
GaussianPrediction: Dynamic 3D Gaussian Prediction for Motion Extrapolation and Free View Synthesis
Boming Zhao (Zhejiang University), Zhaopeng Cui (Zhejiang University)
GenerationPose EstimationDepth EstimationSuper ResolutionGraph Neural NetworkNeural Radiance FieldGaussian SplattingOptical FlowImageVideo
🎯 What it does: Propose the GaussianPrediction framework, which uses 3D Gaussian representations to reconstruct dynamic scenes from monocular videos and predict future perspective images.
GEM3D: GEnerative Medial Abstractions for 3D Shape Synthesis
Dmitry Petrov (University of Massachusetts Amherst), E. Kalogerakis
GenerationData SynthesisTransformerDiffusion modelNeural Radiance FieldAuto EncoderPoint CloudMesh
🎯 What it does: Propose GEM3D, which first generates a skeletonized medial abstraction through a diffusion model, and then uses a neural implicit function to generate the corresponding 3D surface.
Generative Escher Meshes
Noam Aigerman (University of Montreal), Thibault Groueix (Adobe Research)
GenerationData SynthesisTransformerPrompt EngineeringDiffusion modelScore-based ModelTextMesh
🎯 What it does: This study proposes a fully automatic, text-prompt-based generation method that can produce non-square 2D texture grids that perfectly tile the plane and contain only foreground objects;
Going with the Flow
Yousuf Soliman, Peter Schröder
Optical FlowPhysics RelatedStochastic Differential EquationOrdinary Differential Equation
🎯 What it does: Derived and solved second-order differential equations describing the motion of a body in (possibly moving) incompressible media, resulting in a dynamic model that requires only a 6-dimensional state.
Hand-Object Interaction Controller (HOIC): Deep Reinforcement Learning for Reconstructing Interactions with Physics
Haoyu Hu (Tsinghua University), Feng Xu (Tsinghua University)
Robotic IntelligenceConvolutional Neural NetworkRecurrent Neural NetworkTransformerReinforcement LearningImageVideoPoint Cloud
🎯 What it does: Propose a hand-object interaction controller (HOIC) based on deep reinforcement learning, which reconstructs hand-object interaction movements in real-time using a single RGBD camera.
HeadArtist: Text-conditioned 3D Head Generation with Self Score Distillation
Hongyu Liu (HKUST), Qifeng Chen (HKUST)
GenerationData SynthesisKnowledge DistillationConvolutional Neural NetworkTransformerPrompt EngineeringDiffusion modelScore-based ModelImageTextMesh
🎯 What it does: Proposes HeadArtist, which generates high-quality 3D head models using text prompts and supports geometry and texture editing based on language.
High-quality Surface Reconstruction using Gaussian Surfels
Pinxuan Dai (Zhejiang University), Weiwei Xu (Zhejiang University)
RestorationGenerationOptimizationDiffusion modelNeural Radiance FieldAuto EncoderGaussian SplattingPoint CloudMesh
🎯 What it does: Proposes a high-quality surface reconstruction method called Gaussian Surfels, which flattens 3D Gaussians along the local z-axis into 2D ellipses to achieve more accurate surface alignment and optimization stability.
Holographic Parallax Improves 3D Perceptual Realism
Dongyeon Kim (Seoul National University), Yoonchan Jeong (Seoul National University)
OptimizationComputational EfficiencyRepresentation LearningData-Centric LearningVision-Language-Action ModelDiffusion modelScore-based ModelNeural Radiance FieldContrastive LearningGaussian SplattingOptical FlowImageVideoPoint CloudMeshTabularReview/Survey PaperBenchmarkRetrieval-Augmented GenerationChain-of-Thought
🎯 What it does: Evaluate the 3D perceptual realism of different CGH target content types (2.5D RGB-Depth, 3D focal stack, 4D light field) in near-eye holographic displays, and validate their differences through laboratory hardware and user studies.
I2V-Adapter: A General Image-to-Video Adapter for Diffusion Models
Xun Guo (University of Science and Technology of China), Chongyang Ma (Kuaishou Technology)
GenerationData SynthesisTransformerPrompt EngineeringDiffusion modelImageVideoTextStochastic Differential Equation
🎯 What it does: Proposed a lightweight I2V-Adapter, which achieves image-to-video generation on a pre-trained T2V model through cross-frame attention;
Implicit Swept Volume SDF: Enabling Continuous Collision-Free Trajectory Generation for Arbitrary Shapes
Jingping Wang (Zhejiang University), Fei Gao (Zhejiang University)
Autonomous DrivingOptimizationRobotic IntelligencePoint CloudMesh
🎯 What it does: This paper proposes a continuous collision-free trajectory generation framework based on the SDF of an implicit swept volume (Swept Volume), which can generate continuous trajectories satisfying dynamic constraints for objects of arbitrary shapes.
Interactive Character Control with Auto-Regressive Motion Diffusion Models
Yi Shi (Simon Fraser University), Xue Bin Peng (Simon Fraser University)
GenerationPose EstimationRecurrent Neural NetworkTransformerReinforcement LearningDiffusion modelScore-based ModelAuto EncoderVideoSequential
🎯 What it does: Proposed an autoregressive motion diffusion model, A-MDM, which can generate high-fidelity, controllable character motions in real-time environments and supports multiple interactive control strategies (such as sampling, inpainting, and hierarchical reinforcement learning)