SIGGRAPH 2023 Papers with AI Summaries
ACM SIGGRAPH (Transactions on Graphics) · 212 papers
→ SIGGRAPH 2023 papers with code (27)
Each paper below shows an AI-generated one-line summary. Get the full 6-part summary (innovation, method, data, results, limitations) and search all 212 SIGGRAPH 2023 papers by keyword, author or institution —
free trial on arXivSub.
∇-Prox: Differentiable Proximal Algorithm Modeling for Large-Scale Optimization
Zeqiang Lai, Felix Heide
OptimizationLarge Language ModelReinforcement LearningPrompt EngineeringMixture of ExpertsDiffusion modelScore-based ModelAuto EncoderContrastive LearningImageTabularMagnetic Resonance ImagingComputed TomographyReview/Survey PaperChain-of-Thought
🎯 What it does: Proposed ∇-Prox, a differentiable proximal algorithm domain-specific modeling language and compiler for large-scale optimization problems; it allows users to define objective functions with concise high-level code, automatically generates computationally and memory-efficient solvers, and supports hybrid models and learning-based solutions integrated with neural network pipelines; its feasibility was demonstrated on tasks such as image optimization (computational optics, raindrop removal, compressed MRI) and energy system planning.
3D Gaussian Splatting for Real-Time Radiance Field Rendering
Bernhard Kerbl, G. Drettakis
OptimizationComputational EfficiencyRepresentation LearningNeural Radiance FieldGaussian SplattingImagePoint Cloud
🎯 What it does: This paper proposes a scene representation based on a 3D Gaussian distribution, and achieves high-quality real-time view synthesis through a differentiable real-time renderer;
3DShape2VecSet: A 3D Shape Representation for Neural Fields and Generative Diffusion Models
Biao Zhang (KAUST), Peter Wonka (KAUST)
GenerationData SynthesisRepresentation LearningTransformerDiffusion modelNeural Radiance FieldAuto EncoderContrastive LearningImageTextPoint CloudMesh
🎯 What it does: Propose a shape representation called 3DShape2VecSet, which encodes 3D shapes in neural fields using a set of fixed-length latent vectors, and combines diffusion models to achieve 3D shape generation and reconstruction from various conditions such as point clouds, images, and text.
A Contact Proxy Splitting Method for Lagrangian Solid-Fluid Coupling
Tianyi Xie (University of California Los Angeles), Chenfanfu Jiang
OptimizationComputational EfficiencyDiffusion modelScore-based ModelContrastive LearningOptical FlowPoint CloudMeshTabularPhysics RelatedStochastic Differential EquationOrdinary Differential Equation
🎯 What it does: This paper proposes a unified Lagrangian method to achieve two-way strong coupling between weakly compressible SPH fluids and nonlinear elastic FEM solids; through optimization-based time integration, barrier-type contact, approximate viscous potential energy, and proxy-based time splitting, the coupling efficiency and stability are significantly improved.
A Convex Optimization Framework for Regularized Geodesic Distances
M. Edelstein, M. Ben-Chen
OptimizationPoint CloudMesh
🎯 What it does: Propose a unified convex optimization framework for computing geodesic distances with regularization, providing theoretical guarantees, optional regularization terms, symmetric full-pair distance implementations, and an efficient ADMM solver.
A Fast Geometric Multigrid Method for Curved Surfaces
R. Wiersma (Delft University of Technology), K. Hildebrandt (Delft University of Technology)
OptimizationComputational EfficiencyDiffusion modelScore-based ModelFlow-based ModelRectified FlowNeural Radiance FieldAuto EncoderGenerative Adversarial NetworkContrastive LearningGaussian SplattingOptical FlowPoint CloudMeshGraphStochastic Differential EquationOrdinary Differential Equation
🎯 What it does: Propose a geometry multigrid based on graph Voronoi (Gravo-MG), which utilizes hierarchical construction of point clouds to achieve fast hierarchical structures and solve sparse Laplacian systems efficiently.
A Full-Wave Reference Simulator for Computing Surface Reflectance
Yunchen Yu, Steve Marschner
Physics Related
🎯 What it does: Developed an scalable three-dimensional full-wave boundary element method (BEM) simulator for computing reference solutions for rough surface scattering, and proposed a new system for efficiently calculating BRDF values over a large number of incident/exit directions.
A Hybrid Generator Architecture for Controllable Face Synthesis
D. Mensah, J. Lehtinen
GenerationData SynthesisDiffusion modelNeural Radiance FieldAuto EncoderGenerative Adversarial NetworkImageMesh
🎯 What it does: Proposes a hybrid generator architecture that integrates neural networks with fixed-function blocks (such as 3D deformable head models and texture-mapping rasterizers) for controllable facial synthesis.
A Practical Walk-on-Boundary Method for Boundary Value Problems
Ryusuke Sugimoto (University of Waterloo), T. Hachisuka
MeshBenchmarkPhysics Related
🎯 What it does: Proposed a Walk-on-Boundary (WoB) Monte Carlo solver based on boundary integral equations for solving Laplace and Poisson equations with Dirichlet, Neumann, Robin, and mixed boundary conditions, supporting both internal and external domains;
A Practical Wave Optics Reflection Model for Hair and Fur
Mengqi Xia, Steve Marschner
Diffusion modelOptical FlowMeshPhysics Related
🎯 What it does: Developed a 3D wave optics simulator based on the physical optics approximation, and proposed a practical model based on wavelet noise that can be combined with existing scattering models to capture the wave scattering characteristics of hair and fur;
A Realistic Surface-based Cloth Rendering Model
Junqiu Zhu, M. Chiang
Diffusion modelGaussian SplattingMesh
🎯 What it does: Proposed a surface-based fabric coloring model that can generate realistic fabric appearances with hierarchical details, and demonstrated its effectiveness through implementation
A Temporal Coherent Topology Optimization Approach for Assembly Planning of Bespoke Frame Structures
Ziqi Wang, Stelian Coros
Optimization
🎯 What it does: Propose a time-consistency constrained method based on topology optimization for planning the assembly sequence of customized frame structures
Acting as Inverse Inverse Planning
Kartik Chandra (MIT), Jonathan Ragan-Kelley (MIT)
GenerationOptimizationReinforcement Learning from Human FeedbackReinforcement LearningSequential
🎯 What it does: Proposes an inverse inverse planning framework, optimizing animation scripts to make the audience's reasoning (based on inverse planning) align with the story elements the creator intends to convey (such as helping, satire, flashbacks, etc.).
Adaptive Local Basis Functions for Shape Completion
Hui Ying (Zhejiang University), Kun Zhou (Zhejiang University)
GenerationComputational EfficiencyRepresentation LearningTransformerDiffusion modelAuto EncoderContrastive LearningPoint CloudMesh
🎯 What it does: Propose a 3D shape completion method based on adaptive local basis functions;
Algebraic Smooth Occluding Contours
Ryan Capouellez (New York University), D. Zorin
OptimizationComputational EfficiencyMesh
🎯 What it does: This paper proposes a method to approximate any triangular mesh using C1-continuous piecewise quadratic surfaces, and computes the visible occlusion silhouette on this surface in closed-form.
An Elastic Basis for Spectral Shape Correspondence
Florine Hartwig, M. Ben-Chen
Mesh
🎯 What it does: A spectral method based approach is developed to find correspondences between non-isometric shapes and align external features.
An Extensible, Data-Oriented Architecture for High-Performance, Many-World Simulation
Brennan Shacklett, Kayvon Fatahalian
OptimizationComputational EfficiencyReinforcement Learning from Human FeedbackWorld Model
🎯 What it does: Proposed an scalable data-oriented architecture for achieving high-performance multi-environment GPU-accelerated batch simulation;
Anatomically Detailed Simulation of Human Torso
Seunghwan Lee (Stanford University), C.Karen Liu
OptimizationDiffusion modelOptical FlowBiomedical DataPhysics RelatedStochastic Differential EquationOrdinary Differential Equation
🎯 What it does: A detailed human torso model including vertebrae, ligaments, joint surfaces, and intervertebral discs was constructed, and physically and biologically realistic motion was achieved through the coupling of finite element and rigid body methods;
AniFaceDrawing: Anime Portrait Exploration during Your Sketching
Zhengyu Huang (Japan Advanced Institute of Science and Technology), K. Miyata
GenerationData SynthesisTransformerDiffusion modelGenerative Adversarial NetworkImage
🎯 What it does: This paper proposes an unsupervised stroke-level disentanglement technique based on StyleGAN, achieving an AI-assisted drawing system called AniFaceDrawing that generates high-quality anime portraits incrementally during the hand-drawing process;
ArrangementNet: Learning Scene Arrangements for Vectorized Indoor Scene Modeling
Jingwei Huang, Li Yi
GenerationData SynthesisGraph Neural NetworkAuto EncoderContrastive LearningPoint CloudGraph
🎯 What it does: Proposes ArrangementNet, a graph neural network, for estimating indoor scene layouts from incomplete point clouds and generating BIM models.
Attend-and-Excite: Attention-Based Semantic Guidance for Text-to-Image Diffusion Models
Hila Chefer (Tel Aviv University), D. Cohen-Or
GenerationTransformerPrompt EngineeringDiffusion modelImageText
🎯 What it does: This paper proposes a method called 'Attend-and-Excite' for dynamically regulating cross-attention during the inference phase of diffusion models, ensuring that all subject words in the text prompt are generated.
AvatarMAV: Fast 3D Head Avatar Reconstruction Using Motion-Aware Neural Voxels
Yuelang Xu (Tsinghua University), Yebin Liu (Tsinghua University)
GenerationData SynthesisConvolutional Neural NetworkTransformerDiffusion modelNeural Radiance FieldAuto EncoderOptical FlowImageVideoMesh
🎯 What it does: Proposes AvatarMAV, a fast 3D head avatar reconstruction method that utilizes motion-aware neural voxels, capable of training a photorealistic 3D head model from monocular video in just 5 minutes.
AvatarReX: Real-time Expressive Full-body Avatars
Zerong Zheng (Tsinghua University), Yebin Liu (Tsinghua University)
GenerationData SynthesisPose EstimationConvolutional Neural NetworkTransformerDiffusion modelNeural Radiance FieldAuto EncoderGaussian SplattingVideoPoint CloudMesh
🎯 What it does: This paper proposes AvatarReX, which can learn full-body expressible and real-time renderable neural avatars from multi-view videos.
B-rep Matching for Collaborating Across CAD Systems
Benjamin T. Jones (University of Washington), Adriana Schulz (University of Washington)
Computational EfficiencyRepresentation LearningData-Centric LearningGraph Neural NetworkAuto EncoderContrastive LearningMeshGraph
🎯 What it does: A learning-based matching algorithm based on B-rep was developed, which can automatically match and transfer geometric references between different CAD systems, supporting seamless collaboration after model version updates.
BakedSDF: Meshing Neural SDFs for Real-Time View Synthesis
Lior Yariv (Weizmann Institute of Science), B. Mildenhall
GenerationComputational EfficiencyDiffusion modelScore-based ModelNeural Radiance FieldAuto EncoderGaussian SplattingImageVideoMesh
🎯 What it does: Use neural SDF to generate high-quality meshes, combined with spherical Gaussian viewpoint models to achieve real-time viewpoint synthesis.
Beyond Chainmail: Computational Modeling of Discrete Interlocking Materials
P. Tang, Stelian Coros
Diffusion modelScore-based ModelOptical FlowMeshPhysics RelatedStochastic Differential EquationOrdinary Differential Equation
🎯 What it does: A method for computational modeling, mechanical characterization, and macro-scale simulation of discrete interlocking materials (DIM) is proposed.
Bidirectional GaitNet: A Bidirectional Prediction Model of Human Gait and Anatomical Conditions
Jungnam Park (Seoul National University), Jungdam Won (Seoul National University)
GenerationData SynthesisPose EstimationConvolutional Neural NetworkRecurrent Neural NetworkReinforcement LearningMixture of ExpertsDiffusion modelAuto EncoderMeshBiomedical DataMagnetic Resonance ImagingComputed TomographyElectronic Health Records
🎯 What it does: Constructed Bidirectional GaitNet, a bidirectional generative model capable of predicting gait based on human anatomical parameters, and inversely estimating the corresponding anatomical and muscular conditions given a gait.
Blended Latent Diffusion
Omri Avrahami (Hebrew University of Jerusalem), Dani Lischinski (Hebrew University of Jerusalem)
Image TranslationImage HarmonizationGenerationTransformerSupervised Fine-TuningPrompt EngineeringDiffusion modelAuto EncoderContrastive LearningImageTextMultimodality
🎯 What it does: Propose a zero-shot local text-driven image editing method, which utilizes latent diffusion models to perform text-guided local editing in masked regions, and improves editing quality through background reconstruction optimization and evolutionary mask shrinking.
Bodyformer: Semantics-guided 3D Body Gesture Synthesis with Transformer
Kunkun Pang (Guangdong Academy of Sciences), T. Komura
GenerationData SynthesisPose EstimationTransformerDiffusion modelAuto EncoderContrastive LearningMultimodalityAudio
🎯 What it does: Propose a variational model called BodyFormer based on Transformer, used to generate semantically coherent and diverse 3D full-body poses from speech.
Boundary Value Caching for Walk on Spheres
Bailey Miller (Carnegie Mellon University), Ioannis Gkioulekas (Carnegie Mellon University)
OptimizationComputational EfficiencyPoint CloudMeshPhysics Related
🎯 What it does: Propose a boundary value caching (BVC) strategy that combines the walk-on-spheres (WoS) method with boundary integral equations (BIE), performing random walks only on the boundary and caching results to rapidly estimate solutions and gradients of partial differential equations at internal points.
Building a Virtual Weakly-Compressible Wind Tunnel Testing Facility
Chaoyang Lyu, Xiaopei Liu
Auto EncoderPhysics Related
🎯 What it does: Developed an efficient and accurate virtual weakly compressible wind tunnel testing system based on a high-performance dynamic solver
CALM: Conditional Adversarial Latent Models for Directable Virtual Characters
Chen Tessler (NVIDIA), Xue Bin Peng (NVIDIA)
GenerationPose EstimationTransformerReinforcement LearningAuto EncoderGenerative Adversarial NetworkContrastive LearningTime SeriesSequential
🎯 What it does: Propose Conditional Adversarial Latent Models (CALM), a physically plausible character animation framework that can be directly controlled and trained using unlabeled motion capture data;
CLIP-PAE: Projection-Augmentation Embedding to Extract Relevant Features for a Disentangled, Interpretable and Controllable Text-Guided Face Manipulation
Chenliang Zhou (University of Cambridge), Cengiz Öztireli (University of Cambridge)
GenerationExplainability and InterpretabilityTransformerPrompt EngineeringVision Language ModelDiffusion modelGenerative Adversarial NetworkContrastive LearningImageText
🎯 What it does: Propose the Projection-Augmentation Embedding (PAE) technique for CLIP text-guided image editing, improving the target vector to achieve more disentangled, interpretable, and controllable editing.
ClipFace: Text-guided Editing of Textured 3D Morphable Models
Shivangi Aneja (Technical University of Munich), M. Nießner
GenerationData SynthesisTransformerPrompt EngineeringVision Language ModelDiffusion modelNeural Radiance FieldAuto EncoderGenerative Adversarial NetworkContrastive LearningImageTextMeshStochastic Differential Equation
🎯 What it does: Propose ClipFace, enabling the editing of both texture and expression in 3D facial models through text prompts.
COFS: COntrollable Furniture layout Synthesis
W. Para, Peter Wonka (KAUST)
GenerationData SynthesisAnomaly DetectionTransformerMixture of ExpertsDiffusion modelGenerative Adversarial NetworkContrastive LearningPoint CloudMeshSequential
🎯 What it does: Propose COFS, a controllable furniture layout synthesis model based on Transformer, which supports fine-grained conditional control over any subset of object attributes and achieves non-autoregressive sampling.
ColorfulCurves: Palette-Aware Lightness Control and Color Editing via Sparse Optimization
Cheng-Kang Ted Chao, Y. Gingold
OptimizationImage
🎯 What it does: Propose a unified framework for editing color palettes and tone curves, supporting direct editing of palette colors as well as the hue, saturation, and brightness of image pixels, and achieving optimal satisfaction of user constraints through real-time L2,1 sparse optimization.
Complex Wrinkle Field Evolution
Zhen Chen, E. Vouga
Diffusion modelOptical FlowMesh
🎯 What it does: A new representation method for complex wrinkle fields is proposed, along with corresponding interpolation, visualization, and local editing algorithms, which can capture fine wrinkles on coarse triangular meshes and achieve smooth spatial and temporal evolution;
Composite Motion Learning with Task Control
Pei Xu (Clemson University), Ioannis Karamouzas (Clemson University)
Robotic IntelligenceRecurrent Neural NetworkReinforcement LearningGenerative Adversarial NetworkVideoSequential
🎯 What it does: This paper proposes a physics simulation character composite action learning and task control method based on deep reinforcement learning. It can achieve parallel imitation of different body parts through a multi-discriminator GAN structure without manually generating hybrid reference actions, and combine a multi-objective learning framework to achieve collaborative training of composite actions and goal-oriented control.
Computational Exploration of Multistable Elastic Knots
Michele Vidulis, M. Pauly
MeshGraphPhysics Related
🎯 What it does: Proposed and implemented a computational workflow that combines random spatial sampling with physical simulation to systematically discover, study, and design multistable elastic knots.
Computational Long Exposure Mobile Photography
Eric Tabellion (Google Inc.), Y. Pritch
Image TranslationRestorationSuper ResolutionComputational EfficiencyConvolutional Neural NetworkTransformerDiffusion modelAuto EncoderContrastive LearningOptical FlowImageVideo
🎯 What it does: Developed a computer vision system that can achieve long exposure photography on mobile phones, supporting two effects: foreground blur and background blur, and automatically generating high-resolution HDR images after a single shutter click.
Constructing Printable Surfaces with View-Dependent Appearance
Maxine Perroni-Scharf (Princeton University), S. Rusinkiewicz
OptimizationDiffusion modelNeural Radiance FieldAuto EncoderGenerative Adversarial NetworkContrastive LearningOptical FlowImageMesh
🎯 What it does: This paper proposes a method for designing color heightfields that utilize self-occlusion, enabling the presentation of desired images from different viewpoints, followed by direct fabrication of multi-image display surfaces using a 3D printer.
Contact Edit: Artist Tools for Intuitive Modeling of Hand-Object Interactions
A. Lakshmipathy (Carnegie Mellon University), N. Pollard (Carnegie Mellon University)
OptimizationRobotic IntelligencePoint CloudMesh
🎯 What it does: Developed an end-to-end artist tool that uses contact area as a first-primitive to achieve intuitive modeling, editing, transfer, and pose optimization for hand-object interaction.
Coupling Conduction, Convection and Radiative Transfer in a Single Path-Space: Application to Infrared Rendering
Mégane Bati, B. Piaud
Neural Radiance FieldPhysics Related
🎯 What it does: Propose a Monte Carlo algorithm that couples heat conduction, convection, and radiative transfer within a single path space, and apply it to infrared rendering, enabling the interactive computation of multiple thermal simulation scenarios based on different boundary conditions and observation times within the same scene.
CT2Hair: High-Fidelity 3D Hair Modeling using Computed Tomography
Yuefan Shen, Giljoo Nam
GenerationData SynthesisDiffusion modelNeural Radiance FieldAuto EncoderGenerative Adversarial NetworkPoint CloudMeshBiomedical DataComputed Tomography
🎯 What it does: Generate high-fidelity 3D hair models using computed tomography (CT) data.
DARAM: Dynamic Avatar-Human Motion Remapping Technique for Realistic Virtual Stair Ascending Motions
Soobin Lim, Hyeongyeop Kang
GenerationPose EstimationDiffusion modelOptical FlowVideo
🎯 What it does: Propose the DARAM dynamic avatar-human motion remapping technology, enabling VR users to achieve a realistic experience of climbing stairs in a virtual staircase.
Data-Free Learning of Reduced-Order Kinematics
Nicholas Sharp, J. Solomon
OptimizationRepresentation LearningAuto EncoderContrastive LearningPhysics Related
🎯 What it does: Propose a data-free, energy-function-based subspace learning method for neural networks, used to automatically extract low-dimensional, low-energy subspaces of physical systems.
DE-NeRF: DEcoupled Neural Radiance Fields for View-Consistent Appearance Editing and High-Frequency Environmental Relighting
Tong Wu, Lin Gao
GenerationData SynthesisNeural Radiance FieldAuto EncoderMesh
🎯 What it does: Propose DE-NeRF for achieving view-consistent appearance editing and high-frequency environmental relighting.
Deep Real-time Volumetric Rendering Using Multi-feature Fusion
Jinkai Hu, Xiaogang Jin
GenerationData SynthesisComputational EfficiencyDiffusion modelNeural Radiance FieldAuto EncoderContrastive LearningGaussian SplattingPhysics Related
🎯 What it does: Propose a multi-feature radiance prediction neural network (MRPNN) to achieve real-time rendering of high-order scattering participating media.
Deep SVBRDF Estimation from Single Image under Learned Planar Lighting
L. Zhang, Jiawan Zhang
RestorationConvolutional Neural NetworkDiffusion modelAuto EncoderImage
🎯 What it does: Proposes a deep learning-based method that estimates spatially varying BRDF (SVBRDF) from a single image and achieves high-quality reconstruction by learning the illumination patterns of planar light sources and the global correlation prior of the input image.
Denoising-Aware Adaptive Sampling for Monte Carlo Ray Tracing
A. Firmino, H. Jensen
OptimizationComputational Efficiency
🎯 What it does: Propose a new adaptive sampling technique to further improve the efficiency of Monte Carlo rendering combined with deep learning denoising.
Dense, Interlocking-Free and Scalable Spectral Packing of Generic 3D Objects
Qiaodong Cui, W. Matusik
OptimizationMesh
🎯 What it does: A novel 3D object stacking algorithm that utilizes discrete voxel representation and FFT to compute collision and cost functions is proposed.
Deployable strip structures
Daoming Liu, H. Pottmann
MeshPhysics Related
🎯 What it does: Introduce the concept of C-mesh, study strip structures that can be unfolded from compressed states, and explore their geometric properties and applications; meanwhile, provide tools for exploring the shape space and design of C-mesh.
Dictionary Fields: Learning a Neural Basis Decomposition
Anpei Chen, Andreas Geiger
Computational EfficiencyRepresentation LearningNeural Radiance FieldAuto EncoderImagePoint Cloud
🎯 What it does: Propose Dictionary Fields, which achieve more efficient and compact signal approximation by decomposing signals into coefficient fields and basis fields, and by sharing basis functions across different positions and scales using periodic coordinate transformations.
Differentiable Heightfield Path Tracing with Accelerated Discontinuities
Xiao-ran Tong, Alec Jacobson
OptimizationComputational EfficiencyDiffusion modelScore-based ModelNeural Radiance FieldPoint CloudMesh
🎯 What it does: Developed a differentiable height field path tracing renderer capable of achieving real-time inverse rendering for global illumination;
Differentiable Stripe Patterns for Inverse Design of Structured Surfaces
Juan Montes Maestre (ETH Zürich), Bernhard Thomaszewski (ETH Zürich)
OptimizationDiffusion modelScore-based ModelAuto EncoderGenerative Adversarial NetworkContrastive LearningPoint CloudMeshTabularPhysics Related
🎯 What it does: A differentiable process for generating and inverse designing of stripe patterns is proposed to achieve target mechanical behaviors on bi-material thin films.
Differential Operators on Sketches via Alpha Contours
M. Myronova, Mikhail Bessmeltsev
MeshStochastic Differential EquationOrdinary Differential Equation
🎯 What it does: Proposes the Alpha Contours algorithm, which constructs a conservative positive space estimate for vector sketches, and based on this, defines the Laplace and Steklov operators, enabling the application of classical geometric analysis methods on sketches.
Diffusion Image Analogies
Adéla Šubrtová, Daniel Sýkora
Image TranslationGenerationTransformerPrompt EngineeringDiffusion modelImage
🎯 What it does: Proposed a example-based image editing method called Diffusion Image Analogies, which edits the target image according to the transformation intention specified by a pair of example images, making it follow this analogy relationship.
DOC: Differentiable Optimal Control for Retargeting Motions onto Legged Robots
R. Grandia, Moritz Bächer
OptimizationRobotic Intelligence
🎯 What it does: Propose a differentiable optimal control (DOC) framework to achieve the transfer of motion from animals or animations to quadruped robots with different scales and mass distributions;
Drag Your GAN: Interactive Point-based Manipulation on the Generative Image Manifold
Xingang Pan (Max Planck Institute for Informatics), C. Theobalt
GenerationData SynthesisOptimizationDiffusion modelScore-based ModelGenerative Adversarial NetworkOptical FlowImage
🎯 What it does: Achieve fine-grained spatial attribute editing of GAN-generated images through an interactive point dragging method.
DreamFace: Progressive Generation of Animatable 3D Faces under Text Guidance
Longwen Zhang (ShanghaiTech University and Deemos Technology Co., Ltd.), Jingyi Yu (ShanghaiTech University and Shanghai Engineering Research Center of Intelligent Vision and Imaging)
GenerationData SynthesisTransformerLarge Language ModelPrompt EngineeringVision Language ModelDiffusion modelScore-based ModelImageTextMesh
🎯 What it does: Propose DreamFace, a text-prompt-based step-by-step generation system for creating 3D animatable, physically rendered facial assets.
Effect-based Multi-viewer Caching for Cloud-native Rendering
A. Weinrauch, M. Steinberger
OptimizationComputational EfficiencyNeural Radiance FieldGaussian SplattingOptical FlowImage
🎯 What it does: The feasibility of cloud-native rendering architecture was studied, and a multi-view caching mechanism based on effects (world-space and on-surface) was proposed to enable multiple clients to share rendering computations.
Efficient Embeddings in Exact Arithmetic
U. Finnendahl, M. Alexa
OptimizationComputational Efficiency
🎯 What it does: Provides a set of tools that utilize Schnyder labelings and realizers to generate planar embeddings of triangulated topological spheres, and through a new realizer representation based on dual trees, achieves linear-time mapping from triangle weights to barycentric coordinates and the reverse mapping, enabling the repair of flipped triangles in planar realizations.
Efficient Video Portrait Reenactment via Grid-based Codebook
Kaisiyuan Wang, Jingdong Wang
GenerationData SynthesisTransformerDiffusion modelAuto EncoderVideo
🎯 What it does: Proposes a video portrait reenactment framework called VPGC based on a grid codebook, which can efficiently generate high-quality video portraits.
EgoLocate: Real-time Motion Capture, Localization, and Mapping with Sparse Body-mounted Sensors
Xinyu Yi (Tsinghua University), Feng Xu (Tsinghua University)
Pose EstimationAutonomous DrivingOptimizationSimultaneous Localization and MappingOptical FlowImageVideoPoint Cloud
🎯 What it does: This paper proposes the EgoLocate system, which can achieve three functions: human motion capture, localization, and sparse environment mapping, using only six body-mounted IMUs and one head-mounted monocular camera, all in real-time at 60 FPS.
Encoder-based Domain Tuning for Fast Personalization of Text-to-Image Models
Rinon Gal, D. Cohen-Or
GenerationData SynthesisDomain AdaptationComputational EfficiencyTransformerSupervised Fine-TuningPrompt EngineeringDiffusion modelContrastive LearningImageText
🎯 What it does: Propose a domain adaptation method based on an encoder, enabling text-to-image models to rapidly personalize with only one image and a few training steps, achieving second-level speed.
End-to-end Procedural Material Capture with Proxy-Free Mixed-Integer Optimization
Beichen Li, W. Matusik
OptimizationHyperparameter SearchDiffusion modelImage
🎯 What it does: Build the Diffmat v2 library and provide a fully automated end-to-end procedural material capture framework, combining gradient optimization and gradient-free parameter search, to achieve matching between user-captured flash photographs and existing production-level procedural materials.
Étendue Expansion in Holographic Near Eye Displays through Sparse Eye-box Generation Using Lens Array Eyepiece
Minseok Chae, Yoonchan Jeong
OptimizationDiffusion modelOptical FlowPhysics Related
🎯 What it does: Propose a method to expand the Etendue of near-eye holographic display through sparse eyebox generation, and verify the method experimentally.
ETER: Elastic Tessellation for Real-Time Pixel-Accurate Rendering of Large-Scale NURBS Models
Ruicheng Xiong, Ligang Liu
Mesh
🎯 What it does: Proposed the ETER framework, achieving real-time pixel-accurate crack-free rendering of large-scale NURBS models.
Eventfulness for Interactive Video Alignment
Jiatian Sun, Abe Davis
Convolutional Neural NetworkTransformerContrastive LearningOptical FlowVideo
🎯 What it does: Learning visual event descriptors to simplify the video alignment process and integrate them into interactive tools
Evolutionary Piecewise Developable Approximations
Zheng-Yu Zhao, Xiaoming Fu
OptimizationMesh
🎯 What it does: Designed an evolutionary genetic algorithm to perform a developable block approximation of triangular meshes in a high-quality manner.
Example-based Motion Synthesis via Generative Motion Matching
Weiyu Li (Shandong University), Baoquan Chen (Peking University)
GenerationData SynthesisPose EstimationDiffusion modelScore-based ModelGenerative Adversarial NetworkVideoSequential
🎯 What it does: Propose an untrained generative motion matching framework called GenMM, which can quickly generate diverse, high-quality motion sequences based on a single or a few example sequences, and can be extended to applications such as motion completion, keyframe guidance, infinite looping, and recombination.
Example-Based Procedural Modeling Using Graph Grammars
Paul C. Merrell
GenerationData SynthesisGraph Neural NetworkDiffusion modelAuto EncoderGenerative Adversarial NetworkMeshGraph
🎯 What it does: Propose an example-driven polygon shape auto-generation method based on a grammar of images and text.
Expansion Cones: A Progressive Volumetric Mapping Framework
V. Z. Nigolian, D. Bommes
Mesh
🎯 What it does: Proposed a volume mapping framework based on progressive expansion, and implemented a prototype algorithm for converting spherical topology tetrahedral meshes into convex or star-shaped domains.
FactorMatte: Redefining Video Matting for Re-Composition Tasks
Zeqi Gu (Cornell Tech), Abe Davis (Cornell University)
SegmentationConvolutional Neural NetworkDiffusion modelAuto EncoderGenerative Adversarial NetworkContrastive LearningOptical FlowVideo
🎯 What it does: This paper proposes a new video matting method called FactorMatte, which redefines the matting problem as 'counterfactual video synthesis,' enabling the decomposition of videos into multiple mutually independent layers, supporting complex interactions between foreground and background (such as splashes, shadows, reflections), and facilitating post-production re-synthesis.
FashionTex: Controllable Virtual Try-on with Text and Texture
Anran Lin (SSE, CUHKSZ), Xiaoguang Han (SSE, CUHKSZ)
Image TranslationImage HarmonizationRestorationGenerationPose EstimationTransformerSupervised Fine-TuningPrompt EngineeringVision Language ModelDiffusion modelAuto EncoderGenerative Adversarial NetworkContrastive LearningImageTextMultimodality
🎯 What it does: Proposed the FashionTex framework, achieving full-body virtual try-on of clothing based on text and texture patches.
Fast Complementary Dynamics via Skinning Eigenmodes
Otman Benchekroun (University of Toronto), Alec Jacobson (University of Toronto)
OptimizationComputational EfficiencyRobotic IntelligenceDiffusion modelAuto EncoderContrastive LearningOptical FlowPoint CloudMeshGraph
🎯 What it does: Proposed a low-dimensional elastic dynamics subspace (Skinning Eigenmodes) based on linear mixed skinning features, along with a local-global solver, to achieve fast secondary motion supplementation for assembly animations;
Film Grain Rendering and Parameter Estimation
Kaixuan Zhang, T. Pappas
GenerationComputational EfficiencyDiffusion modelGaussian SplattingImagePhysics Related
🎯 What it does: Proposes a realistic movie grain rendering algorithm based on statistics derived from the physics-based Boolean model, and provides formulas for estimating model parameters from scanned movie grain images.
Flexible Isosurface Extraction for Gradient-Based Mesh Optimization
Tianchang Shen (NVIDIA), Jun Gao (NVIDIA)
OptimizationDiffusion modelScore-based ModelAuto EncoderGenerative Adversarial NetworkContrastive LearningPoint CloudMesh
🎯 What it does: Propose a new differentiable isosurface extraction framework called FlexiCubes, which can generate high-quality, differentiable 3D meshes during gradient optimization.
Fluid Cohomology
Hang Yin, Albert Chern
Supervised Fine-TuningReinforcement LearningPrompt EngineeringMixture of ExpertsVision Language ModelVision-Language-Action ModelDiffusion modelScore-based ModelFlow-based ModelRectified FlowNeural Radiance FieldAuto EncoderGenerative Adversarial NetworkContrastive LearningGaussian SplattingSimultaneous Localization and MappingWorld ModelOptical FlowPhysics RelatedRetrieval-Augmented GenerationChain-of-ThoughtStochastic Differential EquationOrdinary Differential Equation
🎯 What it does: Proposed and derived the commonly missed dynamic equation in the vorticity-potential function form for volume inviscid incompressible fluids under non-simply connected basins;
Fluid-Solid Coupling in Kinetic Two-Phase Flow Simulation
Wei Li, M. Desbrun
Diffusion modelScore-based ModelFlow-based ModelRectified FlowPhysics Related
🎯 What it does: An improved lattice Boltzmann method (LBM) solver based on phase field and velocity distribution was developed for fluid-structure interaction in multiphase flows, significantly reducing spurious oscillations in pressure and interfacial forces.
Focal Path Guiding for Light Transport Simulation
Alexander Rath, Philipp Slusallek
OptimizationComputational EfficiencyPhysics Related
🎯 What it does: Propose a path guidance method specifically targeting the focus effect, which can identify and sample focal points based on their contribution to the image, unifying all focus effects.
Forming Terrains by Glacial Erosion
G. Cordonnier, É. Guérin
Diffusion modelGenerative Adversarial NetworkPhysics Related
🎯 What it does: A new approach is studied for simulating the formation and evolution of glaciers and their erosive effects, covering glacial and interglacial cycles.
Galaxy Maps: Localized Foliations for Bijective Volumetric Mapping
Steffen Hinderink, M. Campen
Mesh
🎯 What it does: Proposes a method capable of computing object volume mappings and parameterizations on 3D domains, supporting arbitrary tetrahedral mesh objects with spherical topology, as well as arbitrary convex and star-shaped domains, and provides full control over boundary mappings.
Generalizing Shallow Water Simulations with Dispersive Surface Waves
S. Jeschke, C. Wojtan
Physics Related
🎯 What it does: A simulation method is proposed that represents large water bodies as height fields and, at each time step, divides the waves into body flows that approximately satisfy the shallow water equations and surface waves that approximately satisfy the Airy wave theory. These are solved separately and then recombined to recalculate the wave speed.
Generating Activity Snippets by Learning Human-Scene Interactions
Changyang Li, L. Yu
GenerationOptimizationGraph Neural NetworkTransformerDiffusion modelScore-based ModelContrastive LearningGraphSequential
🎯 What it does: Based on learning from human-computer interaction recordings, a method is proposed to generate virtual activity segments (including keyframe sequences of multi-character, multi-object interaction scenes). First, an abstract interaction graph description is generated through a serialized depth map generation model, and then an optimization framework is used to instantiate the description into a three-dimensional environment;
Generating Procedural Materials from Text or Image Prompts
Yiwei Hu (Yale University), V. Deschaintre
GenerationData SynthesisGraph Neural NetworkTransformerPrompt EngineeringVision Language ModelDiffusion modelContrastive LearningImageTextMultimodalityGraph
🎯 What it does: The study proposes a multimodal conditional generation framework capable of generating procedural material node graphs based on text or image prompts.
Generative Design of Sheet Metal Structures
Amir Barda, Amit H. Bermano
Optimization
🎯 What it does: Proposes a framework for automatically designing thin plate metal (Sheet Metal) structures that can minimize manufacturing costs and generate high-performance manufacturable load-bearing parts while satisfying structural, spatial, and manufacturing constraints.
GestureDiffuCLIP: Gesture Diffusion Model with CLIP Latents
Tenglong Ao (Peking University), Libin Liu (Peking University)
GenerationData SynthesisPose EstimationKnowledge DistillationRepresentation LearningTransformerPrompt EngineeringDiffusion modelScore-based ModelAuto EncoderGenerative Adversarial NetworkContrastive LearningImageVideoTextMultimodalityAudio
🎯 What it does: Designed and implemented a GestureDiffuCLIP system based on the CLIP latent space, which can generate realistic and semantically coherent co-speech gestures from multimodal style prompts such as speech, text, video, or motion clips, and supports fine-grained style control over specific body parts.
Globally Consistent Normal Orientation for Point Clouds by Regularizing the Winding-Number Field
Rui Xu, Changhe Tu
OptimizationDiffusion modelScore-based ModelContrastive LearningPoint Cloud
🎯 What it does: Obtain globally consistent normals for unoriented point clouds through regularized rotation number fields, and use these normals for surface reconstruction.
Gloss-Aware Color Correction for 3D Printing
J. Condor, P. Didyk
OptimizationMesh
🎯 What it does: Designed and verified a gloss-aware color correction method based on differentiable rendering, which can maintain the same perceptual color under different surface gloss levels.
GREIL-Crowds: Crowd Simulation with Deep Reinforcement Learning and Examples
Panayiotis Charalambous, N. Pelechano
GenerationData SynthesisReinforcement Learning
🎯 What it does: Guided by deep reinforcement learning, it learns a crowd behavior model that can capture complex crowd behaviors such as goal pursuit, collision avoidance, and group consistency, and generate realistic crowd simulations;
Guided Linear Upsampling
Shuangbing Song (Shandong University), Changhe Tu (Shandong University)
Super ResolutionComputational EfficiencyAuto EncoderOptical FlowImageVideo
🎯 What it does: A Guided Linear Upsampling (GLU) method is proposed to accelerate high-resolution image processing. The method jointly optimizes the downsampling and upsampling parameters of low-resolution images, allowing each high-resolution pixel to be represented by the linear interpolation of only two low-resolution pixels, thereby accelerating various image operations.
HACK: Learning a Parametric Head and Neck Model for High-fidelity Animation
Longwen Zhang (ShanghaiTech University), Jingyi Yu (ShanghaiTech University)
GenerationData SynthesisTransformerDiffusion modelAuto EncoderGenerative Adversarial NetworkMultimodalityMeshBiomedical DataUltrasoundReview/Survey Paper
🎯 What it does: Propose a parameterized head and neck model named HACK, which can simultaneously control facial expressions, neck posture, laryngeal motion, and appearance texture, achieving high-fidelity digital human animation.
Helix-Free Stripes for Knit Graph Design
Rahul Mitra, Edward Chien
OptimizationDiffusion modelGraph
🎯 What it does: This paper proposes a method to generate non-spiral stripe patterns and track them as weavable knitted diagrams, directly optimizing the discrete differential (first-order form) of the stripe texture function.
High-Order Incremental Potential Contact for Elastodynamic Simulation on Curved Meshes
Z. Ferguson, Daniele Panozzo (New York University)
Diffusion modelOptical FlowPoint CloudMeshPhysics RelatedStochastic Differential EquationOrdinary Differential Equation
🎯 What it does: The paper proposes a high-order incremental potential contact (IPC) method for elastic dynamics simulation on curved mesh surfaces, achieving the collaborative use of high-order geometry and high-order basis functions by mapping high-order volumetric displacements to facial collision proxies through linear mapping.
HumanRF: High-Fidelity Neural Radiance Fields for Humans in Motion
Mustafa Işık (Synthesia), M. Nießner
GenerationData SynthesisVision-Language-Action ModelDiffusion modelNeural Radiance FieldAuto EncoderGaussian SplattingImageVideoMesh
🎯 What it does: Construct and train a 4D dynamic neural radiance field (HumanRF) that can reconstruct and synthesize high-fidelity novel view images of moving humans from multi-view video input.
Image vectorization and editing via linear gradient layer decomposition
Zheng-Jun Du, Kun Xu
Image TranslationRestorationAuto EncoderGenerative Adversarial NetworkImage
🎯 What it does: Automatically converting grating images into multi-layer linear gradient regions
Improved Water Sound Synthesis using Coupled Bubbles
Kangrui Xue, Doug L. James
Data SynthesisPhysics RelatedAudio
🎯 What it does: Proposed a coupled bubble-based acoustic synthesis framework to capture low-frequency acoustic emissions from bubble clouds.
In the Blink of an Eye: Event-based Emotion Recognition
Haiwei Zhang (Dalian University of Technology), Xin Yang (Dalian University of Technology)
RecognitionConvolutional Neural NetworkSpiking Neural NetworkContrastive LearningOptical FlowImageVideo
🎯 What it does: Developed a wearable monocular emotion recognition system based on an event camera
In-Timestep Remeshing for Contacting Elastodynamics
Z. Ferguson, Daniele Panozzo
Diffusion modelOptical FlowMeshPhysics RelatedStochastic Differential EquationOrdinary Differential Equation
🎯 What it does: Proposes an algorithm for adaptive mesh reconstruction within time steps for contact elastodynamic problems; automatically refines and coarsens the mesh during the solution process of each time step, ensuring convergence stability of the solution results after mesh updates and temporal consistency with the previous time step.