SIGGRAPH Asia 2023 Papers with AI Summaries
ACM SIGGRAPH Asia (Transactions on Graphics) · 224 papers
→ SIGGRAPH Asia 2023 papers with code (31)
Each paper below shows an AI-generated one-line summary. Get the full 6-part summary (innovation, method, data, results, limitations) and search all 224 SIGGRAPH Asia 2023 papers by keyword, author or institution —
free trial on arXivSub.
360° Reconstruction From a Single Image Using Space Carved Outpainting
Nuri Ryu (POSTECH), Sunghyun Cho (POSTECH)
GenerationDepth EstimationTransformerPrompt EngineeringDiffusion modelScore-based ModelNeural Radiance FieldAuto EncoderGenerative Adversarial NetworkImagePoint CloudMesh
🎯 What it does: Propose the POP3D framework, which uses spatial peeling + incremental extrapolation rendering to recover a complete 360° 3D model from a single RGB image.
3D Bézier Guarding: Boundary-Conforming Curved Tetrahedral Meshing
Payam Khanteimouri, M. Campen
Mesh
🎯 What it does: A method for generating high-order surface-compatible tetrahedral meshes is proposed, which ensures that the tetrahedral elements on the surface are non-degenerate and non-inverted, and can accurately conform to piecewise polynomial surfaces;
A Hessian-Based Field Deformer for Real-Time Topology-Aware Shape Editing
Yunxiao Zhang (Shandong University), Changhe Tu (Shandong University)
OptimizationComputational EfficiencyDiffusion modelScore-based ModelNeural Radiance FieldAuto EncoderContrastive LearningGaussian SplattingOptical FlowPoint CloudMeshStochastic Differential EquationOrdinary Differential Equation
🎯 What it does: Proposed a Hessian field deformer based on SDF concave points, achieving a real-time interactive system for topology and geometry editing.
A Locality-based Neural Solver for Optical Motion Capture
Xiaoyu Pan (Zhejiang University), Xiaogang Jin (Zhejiang University)
Pose EstimationOptimizationRecurrent Neural NetworkGraph Neural NetworkAuto EncoderContrastive LearningOptical FlowPoint CloudGraphTime Series
🎯 What it does: Propose a method for cleaning and motion solving of optical motion capture data based on locality and heterogeneous graph neural networks.
A Micrograin BSDF Model for the Rendering of Porous Layers
Simon Lucas, Pascal Barla
Diffusion modelScore-based ModelNeural Radiance FieldMeshPhysics Related
🎯 What it does: A new BSDF model is proposed for rendering porous layers covered with dust, rust, dirt, or spray paint. The model is based on elliptical opaque particles and extends the Trowbridge-Reitz (GGX) distribution to handle pores.
A Neural Space-Time Representation for Text-to-Image Personalization
Yuval Alaluf (Tel Aviv University), D. Cohen-Or
GenerationData SynthesisRepresentation LearningTransformerPrompt EngineeringDiffusion modelAuto EncoderContrastive LearningImageText
🎯 What it does: Propose a neural spatiotemporal representation NeTI, which implicitly encodes concepts using a small mapper and can achieve high-fidelity personalized generation without fine-tuning the model.
A Parametric Kinetic Solver for Simulating Boundary-Dominated Turbulent Flow Phenomena
Mengyun Liu, Xiaopei Liu
Physics RelatedStochastic Differential EquationOrdinary Differential Equation
🎯 What it does: Propose a parameterized dynamic solver for turbulent flow phenomena governed by boundary dominance.
A Physically-inspired Approach to the Simulation of Plant Wilting
Filippo Maggioli, D. Michels
Agriculture RelatedPhysics Related
🎯 What it does: Propose a plant wilting simulation method based on physical principles, by simulating internal water transport and mapping changes in water content to the stiffness of branches, and then using position-based dynamics to generate wilting morphologies.
ACE: Adversarial Correspondence Embedding for Cross Morphology Motion Retargeting from Human to Nonhuman Characters
Tianyu Li (Georgia Tech), Sehoon Ha (Georgia Tech)
GenerationData SynthesisPose EstimationRobotic IntelligenceTransformerDiffusion modelAuto EncoderGenerative Adversarial NetworkContrastive LearningVideoTime SeriesSequential
🎯 What it does: Developed the ACE framework, which can map human motion to non-human characters with significant morphological differences, generating natural and feasible actions.
ActRay: Online Active Ray Sampling for Radiance Fields
Jiangkai Wu, Xinggong Zhang
Computational EfficiencyNeural Radiance Field
🎯 What it does: Proposes an online active ray sampling strategy called ActRay to accelerate NeRF training
Adaptive Recurrent Frame Prediction with Learnable Motion Vectors
Zhizhen Wu, Hujun Bao
Super ResolutionRecurrent Neural NetworkDiffusion modelOptical FlowVideo
🎯 What it does: Proposes an adaptive recursive frame prediction framework that integrates learnable motion vectors to improve the frame rate and resolution of real-time rendering.
Adaptive Shells for Efficient Neural Radiance Field Rendering
Zian Wang (NVIDIA), Zan Gojcic (NVIDIA)
Computational EfficiencyDiffusion modelScore-based ModelNeural Radiance FieldGaussian SplattingOptical FlowImagePoint CloudMesh
🎯 What it does: Propose an adaptive thin-shell rendering framework that automatically extracts explicit thin-shells via SDF with spatial transformation kernel width, and achieves efficient NeRF synthesis by using volume rendering (single-sample or few-sample) within the thin-shell.
Adaptive Tracking of a Single-Rigid-Body Character in Various Environments
Tae-Joung Kwon (Hanyang University), Yoonsang Lee (Hanyang University)
OptimizationRobotic IntelligenceReinforcement LearningVideoSequential
🎯 What it does: Learning an adaptive tracking policy on a single rigid body model, which can perform various actions in unseen environments and support controller switching and interactive control.
AdaptNet: Policy Adaptation for Physics-Based Character Control
Pei Xu (Clemson University), V. Zordan
Robotic IntelligenceRecurrent Neural NetworkReinforcement LearningAuto EncoderGenerative Adversarial NetworkVideoPoint CloudPhysics Related
🎯 What it does: This paper proposes AdaptNet, a method that rapidly adapts physical role control strategies by modifying the latent space and internal layers based on existing reinforcement learning control strategies.
Amortizing Samples in Physics-Based Inverse Rendering Using ReSTIR
Yu-Chen Wang, Shuang Zhao
Physics Related
🎯 What it does: Implementing spatiotemporal sampling in inverse direct illumination of physically differentiable rendering using ReSTIR
An Adaptive Fast-Multipole-Accelerated Hybrid Boundary Integral Equation Method for Accurate Diffusion Curves
Seungbae Bang (University of Toronto), Alec Jacobson (University of Toronto)
Computational EfficiencyDiffusion modelGaussian SplattingImagePhysics Related
🎯 What it does: Proposes an adaptive hybrid boundary integral equation (Hybrid BIE) method combined with the fast multipole method (FMM) to render diffusion curves with high precision and infinite resolution.
An Architecture and Implementation of Real-Time Sound Propagation Hardware for Mobile Devices
Eunjae Kim, Woo-Chan Park
Computational EfficiencyAudio
🎯 What it does: Propose a high-performance, low-power real-time audio rendering hardware architecture, implemented and verified on an FPGA, and further evaluated for ASIC implementation using 8nm process technology.
An Implicit Neural Representation for the Image Stack: Depth, All in Focus, and High Dynamic Range
Chao Wang, Thomas Leimkühler
Depth EstimationFlow-based ModelNeural Radiance FieldAuto EncoderImage
🎯 What it does: Propose a unified implicit neural representation method that utilizes image stacks to achieve depth estimation, panoramic focus, and high dynamic range image reconstruction.
An Implicit Physical Face Model Driven by Expression and Style
Lingchen Yang (ETH Zurich), Derek Bradley (DisneyResearch|Studios)
GenerationData SynthesisTransformerDiffusion modelNeural Radiance FieldAuto EncoderContrastive LearningGaussian SplattingImageVideoMeshPhysics Related
🎯 What it does: Proposed a multi-identity facial animation framework based on an implicit neural physics model, which can independently control expressions and styles, and support physical effects such as collision and skeletal deformation.
An Implicitly Stable Mixture Model for Dynamic Multi-fluid Simulations
Yanrui Xu, Xiaojuan Ban
Physics Related
🎯 What it does: A high-level implicit hybrid model is proposed for smooth particle hydrodynamics (SPH), which generates a continuous velocity phase field by computing phase momentum sources, and implicitly obtains the hybrid field using a phase-hybrid momentum mapping mechanism, ensuring incompressibility, mass, and momentum conservation.
An Unified λ-subdivision Scheme for Quadrilateral Meshes with Optimal Curvature Performance in Extraordinary Regions
Weiyin Ma, Yue Ma
Mesh
🎯 What it does: A unified λ-subdivision scheme is proposed, which achieves optimal curvature performance for quadrilateral meshes in singular regions through continuous parameterized subdivision.
Analysis and Synthesis of Digital Dyadic Sequences
Abdalla G. M. Ahmed (KAUST), Peter Wonka (KAUST)
Data SynthesisOptimizationSequentialBenchmark
🎯 What it does: This paper explores the design space of digital binary sequences, provides a complete feature description, and demonstrates how each digital binary net can be reordered into a sequence, while proposing a new family of self-similar digital binary sequences.
Animating Street View
Mengyi Shan (University of Washington), Steven M. Seitz
SegmentationGenerationData SynthesisDepth EstimationAutonomous DrivingConvolutional Neural NetworkTransformerDiffusion modelAuto EncoderGenerative Adversarial NetworkContrastive LearningOptical FlowImageVideoPoint Cloud
🎯 What it does: Given a single street view image or panorama, the system automatically removes existing people and vehicles and inserts animatable pedestrians and vehicles, generating high-resolution videos of arbitrary length.
AniPortraitGAN: Animatable 3D Portrait Generation from 2D Image Collections
Yue Wu (HKUST), Xin Tong (Microsoft Research Asia)
GenerationData SynthesisPose EstimationDiffusion modelNeural Radiance FieldAuto EncoderGenerative Adversarial NetworkContrastive LearningGaussian SplattingImage
🎯 What it does: This paper proposes an animatable 3D portrait generation model that can generate controllable head-and-shoulder region portraits from unstructured 2D image collections, supporting control over facial expressions, head poses, and shoulder movements;
Anti-Aliased Neural Implicit Surfaces with Encoding Level of Detail
Yi Zhuang, Xun Cao (Najing University)
RestorationGenerationNeural Radiance FieldGaussian SplattingImagePoint Cloud
🎯 What it does: Designed and implemented LoD-NeuS, a hybrid explicit-implicit surface representation that combines multi-scale tri-plane encoding and conical sampling, used for anti-aliasing and restoring high-frequency geometric details.
Anything to Glyph: Artistic Font Synthesis via Text-to-Image Diffusion Model
Changshuo Wang, Xiangxu Meng
GenerationData SynthesisTransformerPrompt EngineeringDiffusion modelScore-based ModelImageText
🎯 What it does: Using a pre-trained text-to-image diffusion model, generate artistic character images composed of the object as input with shape images and prompt words describing the object.
ART-Owen Scrambling
Abdalla G. M. Ahmed (KAUST), Peter Wonka (KAUST)
Data SynthesisOptimizationComputational EfficiencyDiffusion modelScore-based ModelImagePoint CloudMeshBenchmark
🎯 What it does: Propose a context-free grammar based on Adaptive Regular Tiles (ART), integrating Owen Scrambling's random bit assignment with tree traversal into a compact process;
Authoring and Simulating Meandering Rivers
A. Paris, E. Galin
Diffusion modelOptical FlowImagePoint CloudMeshPhysics RelatedOrdinary Differential Equation
🎯 What it does: Propose an interactive method for creating and simulating meandering river networks, evolving river channel paths through physics-based advection equations with added control terms.
AvatarStudio: Text-Driven Editing of 3D Dynamic Human Head Avatars
Mohit Mendiratta (Max Planck Institute for Informatics), C. Theobalt
GenerationData SynthesisConvolutional Neural NetworkTransformerSupervised Fine-TuningPrompt EngineeringDiffusion modelScore-based ModelNeural Radiance FieldImageVideoText
🎯 What it does: Propose AvatarStudio, a method for visually editing dynamic 3D head avatars based on text prompts;
BakedAvatar: Baking Neural Fields for Real-Time Head Avatar Synthesis
Hao-Bin Duan (Beihang University), Yan-Pei Cao (Tencent PCG)
GenerationData SynthesisComputational EfficiencyConvolutional Neural NetworkDiffusion modelNeural Radiance FieldAuto EncoderContrastive LearningGaussian SplattingOptical FlowImageVideoMesh
🎯 What it does: BakedAvatar achieves real-time head avatar synthesis in standard rasterization pipelines by baking neural fields into deformable multi-layer meshes and corresponding textures.
Bézier Spline Simplification Using Locally Integrated Error Metrics
Siqi Wang, Alec Jacobson
OptimizationMesh
🎯 What it does: Propose a method to reduce the number of Bézier curves by removing and optimizing local segments while maintaining high fidelity.
Break-A-Scene: Extracting Multiple Concepts from a Single Image
Omri Avrahami (Hebrew University of Jerusalem), D. Lischinski
GenerationData SynthesisTransformerSupervised Fine-TuningPrompt EngineeringDiffusion modelImageText
🎯 What it does: Propose a method to extract multiple concepts from a single image and learn dedicated textual handles for each concept, achieving multi-concept scene reconstruction and combination generation based on text.
C-Shells: Deployable Gridshells with Curved Beams
Quentin Becker, Mark Pauly
Optimization
🎯 What it does: Proposes a computational design process for C-shells, including two methods: forward exploration and reverse design, and reduces elastic energy through design optimization.
C·ASE: Learning Conditional Adversarial Skill Embeddings for Physics-based Characters
Zhiyang Dou (University of Hong Kong), Wenping Wang (Texas A&M University)
Robotic IntelligenceTransformerReinforcement LearningPrompt EngineeringDiffusion modelAuto EncoderGenerative Adversarial NetworkContrastive LearningOptical FlowVideoSequentialPhysics Related
🎯 What it does: Propose the C ASE framework, which utilizes conditional adversarial imitation learning to enable physics-driven characters to master multiple skills and be directly controllable.
CamP: Camera Preconditioning for Neural Radiance Fields
Keunhong Park (Google Research), Ricardo Martin-Brualla (Google Research)
OptimizationComputational EfficiencyRepresentation LearningNeural Radiance FieldImagePoint Cloud
🎯 What it does: Propose and study the CamP camera preprocessing method, which aims to improve the joint optimization of camera parameters in NeRF, thereby enhancing the quality of 3D scene reconstruction.
Capturing Animation-Ready Isotropic Materials Using Systematic Poking
Huanyu Chen, J. Barbič
OptimizationDiffusion modelScore-based ModelContrastive LearningOptical FlowMeshPhysics Related
🎯 What it does: We conduct ordered impacts using 3D-printed rigid cylinders with known radii on elastic solids, record contact force and indentation depth, and optimize the nonlinear isotropic elastic energy by combining finite element methods with the Valanis-Landel material model, obtaining high-precision material parameters that can be directly used for animation.
CLIPXPlore: Coupled CLIP and Shape Spaces for 3D Shape Exploration
Jingyu Hu (Chinese University of Hong Kong), Chi-Wing Fu (Chinese University of Hong Kong)
GenerationData SynthesisRepresentation LearningTransformerPrompt EngineeringVision Language ModelDiffusion modelAuto EncoderContrastive LearningImageTextPoint CloudMesh
🎯 What it does: Propose the CLIPXPlore framework, which utilizes the CLIP vision-language model to guide the exploration and modification of the 3D shape space;
Close the Design-to-Manufacturing Gap in Computational Optics with a 'Real2Sim' Learned Two-Photon Neural Lithography Simulator
Cheng Zheng, Peter T. C. So
OptimizationComputational EfficiencyDiffusion modelAuto EncoderGenerative Adversarial NetworkContrastive LearningPhysics Related
🎯 What it does: Proposed a differentiable photolithography simulator integrated into the model-based optical design loop, and designed and manufactured holographic optical elements (HOE) and multi-level diffractive lenses (MDL) using a two-photon photolithography system
ClothCombo: Modeling Inter-Cloth Interaction for Draping Multi-Layered Clothes
Dohae Lee (Yonsei University), In-Kwon Lee (Yonsei University)
GenerationOptimizationGraph Neural NetworkDiffusion modelGenerative Adversarial NetworkContrastive LearningPoint CloudMeshGraph
🎯 What it does: Propose ClothCombo, which can real-time fit multi-layered clothing of arbitrary topology onto 3D human body models with different shapes and poses, achieving natural arrangement of multi-layered clothing through topology-agnostic embedding and GNN interaction disentanglement network.
Collapsing Embedded Cell Complexes for Safer Hexahedral Meshing
H. Brückler, M. Campen
OptimizationComputational EfficiencyRobotic IntelligenceDiffusion modelScore-based ModelMesh
🎯 What it does: A set of operators performing collapse and split operations on volumetric cell complexes within discrete embedded background grids is proposed, and applied to hexahedral mesh generation for T-grid/base complexes, avoiding expensive and unreliable global volumetric remapping steps while supporting feature alignment constraints.
Commonsense Knowledge-Driven Joint Reasoning Approach for Object Retrieval in Virtual Reality
Haiyan Jiang (Beijing Institute for General Artificial Intelligence), Zhenliang Zhang (Beijing Institute for General Artificial Intelligence)
RetrievalGraph Neural NetworkMultimodality
🎯 What it does: In VR environments, a joint reasoning method driven by common-sense knowledge is proposed, which models gestures and context simultaneously through And-Or graphs, constructing an object retrieval system.
Compact Neural Graphics Primitives with Learned Hash Probing
Towaki Takikawa (NVIDIA), Alexander Keller (NVIDIA)
CompressionNeural Radiance FieldAuto EncoderImageMesh
🎯 What it does: Achieve compact neural graphics primitives by learning hash probing feature grids, providing queryable compressed representations without quantization or compression steps.
Computational Design of Flexible Planar Microstructures
Zhan Zhang, J. Panetta
OptimizationPhysics Related
🎯 What it does: A computational framework for designing flexible planar microstructures was developed, enabling the simulation of target nonlinear elastic behavior under large deformations.
Computational Design of LEGO® Sketch Art
Mingjun Zhou, Chi-Wing Fu
GenerationData SynthesisDiffusion modelAuto EncoderGenerative Adversarial NetworkContrastive LearningOptical FlowImageMesh
🎯 What it does: Propose a computational method for generating LEGO puzzle models from simple input images, utilizing sloped and rounded bricks to achieve smoother curves and sharp features.
Computational Design of Wiring Layout on Tight Suits with Minimal Motion Resistance
Kai Wang (Xiamen University), Xiaohu Guo (University of Texas at Dallas)
OptimizationDiffusion modelOptical FlowVideoPoint CloudMesh
🎯 What it does: This paper proposes an automated method for cable layout design in tight-fitting clothing, aiming to minimize cable resistance during the wearer's movements.
Concept Decomposition for Visual Exploration and Inspiration
Yael Vinker (Tel Aviv University), Ariel Shamir (Reichman University)
GenerationExplainability and InterpretabilityRepresentation LearningTransformerPrompt EngineeringVision Language ModelDiffusion modelContrastive LearningImageTextMultimodality
🎯 What it does: Propose a concept decomposition method based on large-scale vision-language models, which uses a binary tree structure to decompose a single visual concept into multiple sub-concepts, and achieves infinite generation and combination of each sub-concept through text embedding vectors, thereby stimulating design inspiration.
Conditional Resampled Importance Sampling and ReSTIR
M. Kettunen, Chris Wyman
OptimizationComputational EfficiencyReinforcement LearningDiffusion modelScore-based ModelPhysics Related
🎯 What it does: Building upon Generalized Resampled Importance Sampling (GRIS), it is extended to the conditional probability space, and a final gather stage is implemented in ReSTIR Path Tracing, allowing the reuse of photon sub-paths with a delay of at least one bounce.
Constrained Delaunay Tetrahedrization: A Robust and Practical Approach
Lorenzo Diazzi (UniMoRe), M. Attene
OptimizationComputational EfficiencyRobotic IntelligenceDiffusion modelScore-based ModelAuto EncoderGenerative Adversarial NetworkContrastive LearningPoint CloudMeshBenchmark
🎯 What it does: Designed and implemented a numerically robust, parameter-independent 3D Constrained Delaunay Tetrahedralization (CDT) algorithm that can successfully construct CDT on all valid PLCs.
Constructive Solid Geometry on Neural Signed Distance Fields
Z. Marschner, Alec Jacobson
GenerationOptimizationDiffusion modelScore-based ModelNeural Radiance FieldAuto EncoderMesh
🎯 What it does: This paper studies the challenges of using Signed Distance Field (SDF) based on neural networks in shape editing. First, it performs spatial characterization of pseudo-SDF (i.e., outputs that satisfy the Eikonal equation but are not true distance functions). Subsequently, it proposes a closest point loss as a regularization term to encourage the edited output to become a true SDF. This regularization is applied to common geometric operations such as Boolean operations (CSG) and swept volumes, successfully generating neural SDF results that conform to the SDF definition.
Content-based Search for Deep Generative Models
Daohan Lu (Carnegie Mellon University), Jun-Yan Zhu (Carnegie Mellon University)
GenerationData SynthesisRetrievalRepresentation LearningTransformerVision Language ModelDiffusion modelScore-based ModelFlow-based ModelRectified FlowContrastive LearningImageTextMultimodalityBenchmarkAudio
🎯 What it does: This paper proposes a content-driven deep generative model retrieval task and constructs the Generative Model Zoo benchmark. It introduces a retrieval framework based on probabilistic and contrastive learning, which can retrieve text, images, sketches, and models themselves.
Controllable Group Choreography Using Contrastive Diffusion
Nhat Le (AIOZ), Anh Nguyen (University of Liverpool)
GenerationData SynthesisPose EstimationTransformerDiffusion modelContrastive LearningVideoMultimodalityAudio
🎯 What it does: Propose a contrastive diffusion model that generates controllable group dance animations driven by music, capable of generating dance sequences with different numbers of people, varying levels of consistency, and diversity under the same piece of music.
Curl Noise Jittering
J. A. Bærentzen, Jonàs Martínez
GenerationData SynthesisDiffusion modelScore-based ModelPoint Cloud
🎯 What it does: By using a curl noise vector field to jitter regular lattices, implicit blue noise point sets are generated.
Decaf: Monocular Deformation Capture for Face and Hand Interactions
Soshi Shimada (MPI for Informatics), C. Theobalt
Pose EstimationOptimizationConvolutional Neural NetworkDiffusion modelAuto EncoderGenerative Adversarial NetworkContrastive LearningOptical FlowImageVideoMesh
🎯 What it does: This work reconstructs the 3D interaction between hands and faces from monocular RGB videos and captures the non-rigid deformations of the face caused by the interaction.
DeepBasis: Hand-Held Single-Image SVBRDF Capture via Two-Level Basis Material Model
Li Wang, Jiawan Zhang
OptimizationConvolutional Neural NetworkDiffusion modelAuto EncoderContrastive LearningImage
🎯 What it does: Propose the DeepBasis method, which utilizes a two-layer basis model and deep learning to jointly predict basis and mixing weights, and estimates the illumination direction of handheld photography through an optimization algorithm, thus achieving the recovery of spatially varying bidirectional reflectance distribution function (SVBRDF) under a single image.
Depolarized Holography with Polarization-Multiplexing Metasurface
Seung-Woo Nam (Seoul National University), Yoonchan Jeong (Seoul National University)
OptimizationDiffusion modelNeural Radiance FieldContrastive LearningOptical FlowImagePhysics Related
🎯 What it does: In this work, the authors propose and implement a depolarization holographic display technique that utilizes polarization multiplexed metal gratings in conjunction with a spatial light modulator (SLM) to achieve interference-free image superposition. The aim is to significantly reduce speckle noise and improve image quality by utilizing mutually independent orthogonal polarization states.
Developable Quad Meshes and Contact Element Nets
Victor Ceballos Inza (KAUST), H. Pottmann
OptimizationDiffusion modelScore-based ModelMesh
🎯 What it does: This paper proposes a discrete foldable surface modeling and optimization framework based on quadrilateral grids and contact element networks.
Differentiable Dynamic Visible-Light Tomography
Kaizhang Kang, Hongzhi Wu
OptimizationDiffusion modelScore-based ModelFlow-based ModelRectified FlowNeural Radiance FieldAuto EncoderOptical FlowImageVideoComputed Tomography
🎯 What it does: Proposed the first visible-light tomography system capable of real-time acquisition and reconstruction of general time-varying 3D phenomena, using a single high-speed camera, a high-performance LED array, and 5 kilometers of fiber optic cable to build an acquisition device with no mechanical movement, 360° coverage, and 1920 interleaved sources/detectors.
Differentiable Rendering of Parametric Geometry
Markus Worchel, Marc Alexa
OptimizationDiffusion modelNeural Radiance FieldGaussian SplattingMesh
🎯 What it does: Propose an efficient differentiable rendering method for parameterizing surfaces and curves, converting continuous parameterized objects into triangular meshes through differentiable meshing operations, and then processing these geometries using existing differentiable mesh rendering techniques.
DiffFR: Differentiable SPH-Based Fluid-Rigid Coupling for Rigid Body Control
Zhehao Li, Ligang Liu
OptimizationRobotic IntelligenceDiffusion model
🎯 What it does: Proposed a differentiable SPH two-way fluid-solid coupling simulator for solving inverse design problems in rigid body control.
Diffusing Colors: Image Colorization with Text Guided Diffusion
Nir Zabari (Lightricks), Ohad Fried (Reichman University)
Image TranslationGenerationTransformerPrompt EngineeringVision Language ModelDiffusion modelAuto EncoderImageText
🎯 What it does: Proposed a text-guided image coloring method based on diffusion models, which can generate high-quality and controllable color results on grayscale images.
Diffusion Posterior Illumination for Ambiguity-Aware Inverse Rendering
Linjie Lyu (Max-Planck-Institut für Informatik), C. Theobalt
Image TranslationRestorationGenerationConvolutional Neural NetworkTransformerDiffusion modelScore-based ModelNeural Radiance FieldAuto EncoderGenerative Adversarial NetworkImagePoint CloudMesh
🎯 What it does: This paper proposes an inverse rendering framework that combines the prior of unconditional diffusion models with differentiable path tracing, enabling the simultaneous inference of spatially varying surface materials and natural environmental lighting from a single or multiple images, and generating diverse and natural environment maps.
Diffusion-based Holistic Texture Rectification and Synthesis
Guoqing Hao (University of Tsukuba), K. Fukui
RestorationGenerationData SynthesisTransformerDiffusion modelAuto EncoderImage
🎯 What it does: Studies a framework for completing and synthesizing texture samples that are occluded or deformed in natural images.
Discontinuity-Aware 2D Neural Fields
Yash Belhe, Tzu-Mao Li
Diffusion modelScore-based ModelNeural Radiance FieldMesh
🎯 What it does: Proposes a 2D neural field based on known discontinuous boundaries, using Bézier curves to describe discontinuities, and constructing a feature field for triangular mesh surfaces, with signals decoded through a shallow multi-layer perceptron.
Discovering Fatigued Movements for Virtual Character Animation
N. Cheema (DFKI), P. Slusallek (DFKI)
Reinforcement LearningGenerative Adversarial NetworkVideoSequential
🎯 What it does: Studied the spontaneous simulation of fatigue accumulation in virtual characters during long-term actions, and proposed a full-body control strategy based on deep reinforcement learning and a three-compartment muscle model.
Domain-Agnostic Tuning-Encoder for Fast Personalization of Text-To-Image Models
Moab Arar (Tel Aviv University), Amit H. Bermano (Tel Aviv University)
GenerationDomain AdaptationComputational EfficiencyTransformerDiffusion modelContrastive LearningImageTextMultimodality
🎯 What it does: Propose a domain-agnostic tuning encoder for rapidly personalizing text-to-image diffusion models using only a single image and 12 or fewer training steps.
Doppler Time-of-Flight Rendering
Juhyeon Kim (Dartmouth), A. Pediredla
OptimizationComputational EfficiencyNeural Radiance FieldOptical FlowImageVideoPhysics Related
🎯 What it does: Proposed a Doppler Time-of-Flight rendering framework for dynamic scenes, derived the dynamic ToF path integral and implemented unbiased Monte Carlo sampling.
DR-Occluder: Generating Occluders Using Differentiable Rendering
Jiaxian Wu, Dehui Lu
GenerationOptimizationConvolutional Neural NetworkDiffusion modelNeural Radiance FieldAuto EncoderImageMesh
🎯 What it does: Propose the DR-Occluder framework, which uses differentiable rendering to generate occluders. First, 3D models are projected into silhouette images, then convolutional networks output vertex offsets, converting distributed triangles into a preliminary occluder. Subsequently, differentiable rendering further optimizes it, and finally, triangles with areas below a threshold are removed to obtain the final occluder.
DreamEditor: Text-Driven 3D Scene Editing with Neural Fields
Jingyu Zhuang (Sun Yat-sen University), Guanbin Li (Sun Yat-sen University)
GenerationData SynthesisKnowledge DistillationTransformerPrompt EngineeringVision Language ModelDiffusion modelScore-based ModelNeural Radiance FieldImageTextMesh
🎯 What it does: Achieved 3D scene neural field editing based on text prompts
Drivable Avatar Clothing: Faithful Full-Body Telepresence with Dynamic Clothing Driven by Sparse RGB-D Input
Donglai Xiang (Carnegie Mellon University), Timur M. Bagautdinov
GenerationData SynthesisPose EstimationDepth EstimationConvolutional Neural NetworkGraph Neural NetworkTransformerDiffusion modelNeural Radiance FieldAuto EncoderContrastive LearningOptical FlowImageVideoPoint CloudMesh
🎯 What it does: Constructed a full-body holographic avatar driven by sparse RGB-D input, capable of realistically reproducing the dynamics and details of loose clothing.
DROP: Dynamics Responses from Human Motion Prior and Projective Dynamics
Yifeng Jiang (Stanford University), C. Liu
GenerationOptimizationTransformerReinforcement LearningDiffusion modelContrastive LearningOptical FlowVideoPoint CloudMesh
🎯 What it does: Propose the DROP framework, which combines pre-trained motion generation models with projective dynamics to achieve physical responses to human actions;
Editing Motion Graphics Video via Motion Vectorization and Transformation
Sharon Zhang (Stanford University), Maneesh Agrawala (Stanford University)
Object TrackingGenerationOptimizationDiffusion modelAuto EncoderOptical FlowVideo
🎯 What it does: This paper proposes a complete workflow to convert motion graphics videos into editable SVG animations, and provides a program transformation API based on the query-operation model, which enables programmatic editing of the temporal, motion, and appearance attributes of objects, thus quickly generating various animation variants.
Efficient Cone Singularity Construction for Conformal Parameterizations
Mo Li, Xiao-Ming Fu
OptimizationComputational EfficiencyMesh
🎯 What it does: An efficient method for constructing sparse cone singularities under deformation constraints is proposed.
Efficient Graphics Representation with Differentiable Indirection
Sayantani Datta (McGill University), Derek Nowrouzezahrai (McGill University)
CompressionComputational EfficiencyRepresentation LearningDiffusion modelNeural Radiance FieldAuto EncoderContrastive LearningImagePoint CloudMesh
🎯 What it does: Proposes a novel differentiable memory table primitive called Differentiable Indirection (DIn), which replaces MLP computations in traditional neural networks to achieve efficient representation and compression for various graphics tasks such as geometry, texture, shading, and radiance fields.
Efficient Human Motion Reconstruction from Monocular Videos with Physical Consistency Loss
Lin Cong, Jianwei Zhang
Pose EstimationOptimizationComputational EfficiencyOptical FlowVideo
🎯 What it does: An efficient method based on gradients is proposed, which reconstructs complex human motion (including high dynamic and aerial movements) from monocular videos using a physical consistency loss, while considering contact and camera alignment.
Efficient Hybrid Zoom Using Camera Fusion on Mobile Phones
Xiaotong Wu (Google), Chia-Kai Liang (Google)
Super ResolutionConvolutional Neural NetworkTransformerDiffusion modelAuto EncoderGenerative Adversarial NetworkContrastive LearningOptical FlowImage
🎯 What it does: A hybrid zoom super-resolution system that efficiently runs on mobile devices was developed, utilizing synchronized wide-angle (W) and telephoto (T) images for alignment, detail fusion, and generating clear multi-times zoomed images through adaptive hybrid mask generation.
Efficient Visualization of Light Pollution for the Night Sky
Y. Dobashi, Kei Iwasaki
Computational EfficiencyPhysics Related
🎯 What it does: Propose a new system for visualizing light pollution in the night sky, which precomputes the sky intensity distribution under urban lighting conditions at different locations and atmospheric conditions, and achieves efficient visualization using principal component analysis (PCA) and fast Fourier transform (FFT); it also supports interactive inverse problem solving to determine the required urban lighting intensity to reduce light pollution.
Ego3DPose: Capturing 3D Cues from Binocular Egocentric Views
Taeho Kang, Youngki Lee
Pose EstimationDepth EstimationConvolutional Neural NetworkTransformerAuto EncoderContrastive LearningOptical FlowImageVideoPoint Cloud
🎯 What it does: This paper proposes a binocular perspective self-centered 3D pose reconstruction system named Ego3DPose, which improves pose accuracy by leveraging binocular stereo matching and perspective effects.
Emotional Speech-Driven Animation with Content-Emotion Disentanglement
Radek Daněček (Max Planck Institute for Intelligent Systems), Timo Bolkart (Max Planck Institute for Intelligent Systems)
Image TranslationGenerationPose EstimationTransformerDiffusion modelNeural Radiance FieldAuto EncoderGenerative Adversarial NetworkContrastive LearningOptical FlowVideoMeshAudio
🎯 What it does: Develop the EMOTE system, which can generate 3D talking-head animations from speech that are consistent with specified emotions and synchronized with lip movements.
EMS: 3D Eyebrow Modeling from Single-View Images
Chenghong Li (Chinese University of Hong Kong Shenzhen), Xiaoguang Han (Chinese University of Hong Kong Shenzhen)
RestorationGenerationConvolutional Neural NetworkRecurrent Neural NetworkDiffusion modelAuto EncoderContrastive LearningImageMesh
🎯 What it does: Proposed the EMS framework to reconstruct fiber-level 3D eyebrow models using only single-view portrait images.
Enhancing Diffusion Models with 3D Perspective Geometry Constraints
Rishi Upadhyay (University of California, Los Angeles), Achuta Kadambi (University of California, Los Angeles)
GenerationDepth EstimationAutonomous DrivingConvolutional Neural NetworkTransformerSupervised Fine-TuningDiffusion modelOptical FlowImage
🎯 What it does: Improve the linearity perspective accuracy of generated images by introducing geometry constraints based on vanishing points during the training process of latent diffusion models.
Example-Based Sampling with Diffusion Models
Bastien Doignies (University of Lyon), V. Ostromoukhov
GenerationData SynthesisConvolutional Neural NetworkTransformerDiffusion modelPoint Cloud
🎯 What it does: Learn the distribution of a 2D sampler from example point sets using diffusion models, enabling the generation of point sets similar to those produced by various traditional samplers.
EXIM: A Hybrid Explicit-Implicit Representation for Text-Guided 3D Shape Generation
Zhengzhe Liu, Chi-Wing Fu
GenerationData SynthesisTransformerVision Language ModelDiffusion modelNeural Radiance FieldAuto EncoderImageTextMesh
🎯 What it does: This paper proposes a hybrid explicit-implicit representation method called EXIM, which generates high-fidelity 3D shapes guided by text and supports local editing.
Explorable Mesh Deformation Subspaces from Unstructured 3D Generative Models
Arman Maesumi (Brown University), Daniel Ritchie (Brown University)
GenerationData SynthesisDiffusion modelScore-based ModelFlow-based ModelGenerative Adversarial NetworkPoint CloudMesh
🎯 What it does: By constructing a two-dimensional navigable exploration space, the high-dimensional latent space of a pre-trained 3D shape generation model is projected onto a two-dimensional plane, enabling smooth and natural interpolation among a given set of landmark shapes, and transferring the generated variations to high-quality meshes.
Extended Path Space Manifolds for Physically Based Differentiable Rendering
J.-G. Xing, Kun Xu
Diffusion modelScore-based ModelNeural Radiance FieldOptical FlowImagePhysics Related
🎯 What it does: Propose an extended path space manifold and build a physics-based differential rendering method based on it, to better handle complex lighting effects.
ExtraSS: A Framework for Joint Spatial Super Sampling and Frame Extrapolation
Songyin Wu, Ling-Qi Yan
Super ResolutionDiffusion modelOptical FlowImageVideo
🎯 What it does: Proposes a framework called ExtraSS, which combines spatial supersampling with frame extrapolation to enhance real-time rendering performance.
Face0: Instantaneously Conditioning a Text-to-Image Model on a Face
Dani Valevski (Google Research), Yaniv Leviathan (Google Research)
Image TranslationGenerationRepresentation LearningConvolutional Neural NetworkTransformerSupervised Fine-TuningPrompt EngineeringDiffusion modelContrastive LearningImageTextMultimodality
🎯 What it does: The Face0 method generates images that are similar to a given face and consistent with text prompts by directly conditioning on facial features in a single step, while maintaining the speed of the original text-image diffusion model.
Fast-MSX: Fast Multiple Scattering Approximation
E. Rosales, Ligang Liu
Computational EfficiencyDiffusion modelOptical FlowPhysics Related
🎯 What it does: A new method is proposed to estimate the contribution of multiple scattering using secondary photorealistic bounces.
FLARE: Fast Learning of Animatable and Relightable Mesh Avatars
Shrisha Bharadwaj (Max Planck Institute for Intelligent Systems), Victoria Fernandez-Abrevaya
GenerationPose EstimationComputational EfficiencyRepresentation LearningConvolutional Neural NetworkGraph Neural NetworkTransformerDiffusion modelNeural Radiance FieldAuto EncoderGenerative Adversarial NetworkContrastive LearningGaussian SplattingOptical FlowImageVideoMesh
🎯 What it does: Propose FLARE, which can quickly learn animatable and relightable 3D head mesh avatars from monocular videos, directly outputting high-fidelity meshes and animatable expression parameters that can be used in existing rendering pipelines.
Fluid Simulation on Neural Flow Maps
Yitong Deng (Dartmouth College), Bo Zhu (Georgia Institute of Technology)
Diffusion modelScore-based ModelFlow-based ModelRectified FlowNeural Radiance FieldOptical FlowImageVideoPhysics Related
🎯 What it does: Proposed the Neural Flow Maps method in fluid simulation, combining implicit neural representations with flow graph theory to achieve high-precision inviscid fluid simulation.
From Skin to Skeleton: Towards Biomechanically Accurate 3D Digital Humans
Marilyn Keller, Michael J. Black
GenerationPose EstimationOptimizationGraph Neural NetworkDiffusion modelAuto EncoderContrastive LearningPoint CloudMeshBiomedical DataBenchmark
🎯 What it does: Developed the SKEL model, which integrates the SMPL human surface model with the Biomechanical Skeleton Model (BSM), achieving a 3D digital human driven by shape parameters and biomechanical pose parameters.
FuseSR: Super Resolution for Real-time Rendering through Efficient Multi-resolution Fusion
Zhihua Zhong (Zhejiang University), Rui Wang (Zhejiang University)
Super ResolutionConvolutional Neural NetworkDiffusion modelScore-based ModelAuto EncoderContrastive LearningOptical FlowImageVideo
🎯 What it does: Propose FuseSR, a real-time super-resolution method that fuses low-resolution rendering images with high-resolution G-buffer, capable of generating high-quality images at magnification factors of 4×4 or even 8×8.
Fusing Monocular Images and Sparse IMU Signals for Real-time Human Motion Capture
Shaohua Pan (Tsinghua University), Feng Xu
Pose EstimationOptimizationRecurrent Neural NetworkImageTime Series
🎯 What it does: Propose a real-time motion capture framework that fuses monocular images with sparse IMU data, maintaining high accuracy even when visual input fails or is severely occluded;
GANeRF: Leveraging Discriminators to Optimize Neural Radiance Fields
Barbara Roessle, M. Nießner
GenerationData SynthesisOptimizationDiffusion modelNeural Radiance FieldAuto EncoderGenerative Adversarial NetworkContrastive LearningImagePoint Cloud
🎯 What it does: Propose the GANeRF method, which combines the GAN discriminator to directly perform 3D consistency training on NeRF, and further improves the rendering quality using a multi-scale conditional generator;
GARM-LS: A Gradient-Augmented Reference-Map Method for Level-Set Fluid Simulation
Xingqiao Li, Bao Chen
Optical FlowPhysics Related
🎯 What it does: A new level set method combining gradient enhancement with reference mapping is proposed for high-fidelity interface tracking and surface tension flow simulation, and a complete pipeline is developed.
GarmentCode: Programming Parametric Sewing Patterns
Maria Korosteleva (ETH Zurich), O. Sorkine-Hornung
GenerationOptimizationAI Code AssistantPrompt EngineeringDiffusion modelAuto EncoderGenerative Adversarial NetworkImageTextMeshBenchmark
🎯 What it does: This paper proposes a domain-specific language called GarmentCode based on object-oriented programming, for generating programmable and interactive parameterized sewing patterns, and implements a garment configurator;
GeoLatent: A Geometric Approach to Latent Space Design for Deformable Shape Generators
Haitao Yang, Qixing Huang
GenerationOptimizationRepresentation LearningDiffusion modelAuto EncoderContrastive LearningMesh
🎯 What it does: This study explores how to optimize the latent space of neural shape generators from a differential geometry perspective. It defines a Riemannian metric based on 'as rigid as possible' and 'as conformal as possible' deformation energies, and investigates two ideal properties of the latent space under this metric: linear interpolation along coordinate axes follows geodesics, and latent codes can decouple pose and shape variations at different scales. It approximates the geodesic interpolation property by enforcing geodesic interpolation aligned with axis-aligned directions. An innovative method is proposed to decouple pose and shape using generalized eigen-decomposition, and an efficient regularization term is studied to promote smooth interpolation.
GroomGen: A High-Quality Generative Hair Model Using Hierarchical Latent Representations
Yuxiao Zhou (ETH Zurich), T. Beeler
GenerationData SynthesisConvolutional Neural NetworkDiffusion modelAuto EncoderGenerative Adversarial NetworkImageMesh
🎯 What it does: Propose GroomGen, a high-quality hair generation model based on a hierarchical latent space;
GroundLink: A Dataset Unifying Human Body Movement and Ground Reaction Dynamics
Xingjian Han (Boston University), Jun Saito (Adobe Research)
Data SynthesisPose EstimationConvolutional Neural NetworkContrastive LearningOptical FlowTime SeriesBiomedical DataBenchmark
🎯 What it does: Developed the GroundLink dataset, which unifies human motion capture with ground reaction force (GRF) and center of pressure (CoP) information, and proposed GroundLinkNet as a baseline model for prediction.
Hand Pose Estimation with Mems-Ultrasonic Sensors
Qiang Zhang (Princeton University), S. Rusinkiewicz
Pose EstimationRecurrent Neural NetworkTransformerContrastive LearningVideoPoint CloudTabularUltrasound
🎯 What it does: Developed a low-cost hand motion capture glove based on MEMS ultrasonic sensors, using sensors to measure the inter-distance matrix and a lightweight deep network to predict 3D gestures;
High Density Ratio Multi-Fluid Simulation with Peridynamics
Han Yan, Bo-Ning Ren
Physics Related
🎯 What it does: A hybrid model theory based on Peridynamics is proposed, which can stably handle particle simulations of high density ratio multi-fluid mixing and separation; meanwhile, a new scalar volume flux state and particle discretization scheme are proposed for calculating all terms of the multi-phase Navier-Stokes equations in integral form, and a novel mass update strategy is designed to improve inter-phase mass conservation and reduce particle volume changes under high density ratio conditions.