SIGGRAPH Asia 2024 Papers — Page 2
ACM SIGGRAPH Asia (Transactions on Graphics) · 265 papers
Follow-Your-Emoji: Fine-Controllable and Expressive Freestyle Portrait Animation
Yue Ma (HKUST), Qifeng Chen (HKUST)
Image TranslationGenerationPose EstimationConvolutional Neural NetworkTransformerPrompt EngineeringDiffusion modelContrastive LearningImageVideoText
🎯 What it does: Proposed a diffusion model-based framework for free-style human portrait animation called Follow-Your-Emoji, which can control the expression and posture of the portrait through a given sequence of facial key points.
FragmentDiff: A Diffusion Model for Fractured Object Assembly
Qun-Ce Xu, Tai-Jiang Mu
RestorationPose EstimationTransformerDiffusion modelScore-based ModelImage
🎯 What it does: Proposed a fragment pose prediction method based on diffusion models and Transformer, using diffusion denoising technology to predict the pose parameters of each fragment.
Frankenstein: Generating Semantic-Compositional 3D Scenes in One Tri-Plane
Han Yan (Shanghai Jiao Tong University), Pan Ji (Tencent XR Vision Labs)
GenerationData SynthesisTransformerDiffusion modelScore-based ModelNeural Radiance FieldAuto EncoderImageTextPoint CloudMesh
🎯 What it does: Propose Frankenstein, a framework based on tri-plane diffusion models, capable of generating semantically separable 3D scenes (such as indoor rooms, avatars, etc.) in one go and supporting layout control.
FreeAvatar: Robust 3D Facial Animation Transfer by Learning an Expression Foundation Model
Feng Qiu (NetEase Fuxi AI Lab), Xin Yu (University of Queensland)
Image TranslationGenerationRepresentation LearningTransformerDiffusion modelNeural Radiance FieldAuto EncoderGenerative Adversarial NetworkContrastive LearningImageMesh
🎯 What it does: Proposes FreeAvatar, a system that achieves 3D facial animation transfer by utilizing a learned expression basis model, enabling the generation of high-fidelity expression animations for multiple 3D avatars driven by arbitrary in-the-wild facial images.
From Sim-to-Real: Toward General Event-based Low-light Frame Interpolation with Per-scene Optimization
Ziran Zhang (Zhejiang University), Shi Guo (Shanghai AI Laboratory)
RestorationData SynthesisOptimizationConvolutional Neural NetworkTransformerSupervised Fine-TuningDiffusion modelContrastive LearningOptical FlowImageVideoTime SeriesBenchmark
🎯 What it does: This paper proposes an event-driven video frame interpolation method based on scene adaptation, and improves the interpolation quality in low-light environments through full-sequence fine-tuning, while constructing a low-light event-RGB benchmark dataset called EVFI-LL.
FürElise: Capturing and Physically Synthesizing Hand Motion of Piano Performance
Ruocheng Wang (Stanford University), C. K. Liu (Stanford University)
Data SynthesisPose EstimationTransformerReinforcement LearningDiffusion modelGenerative Adversarial NetworkOptical FlowVideoMultimodalityTime SeriesRetrieval-Augmented GenerationAudio
🎯 What it does: This work constructs the first large-scale 3D hand motion dataset, FürElise, and designs a physical simulation pipeline based on this dataset, which can generate physically plausible piano hand motions with precise key pressing under given musical scores;
GarVerseLOD: High-Fidelity 3D Garment Reconstruction from a Single In-the-Wild Image using a Dataset with Levels of Details
Zhongjin Luo (CUHKSZ), Xiaoguang Han (CUHKSZ)
RestorationGenerationData SynthesisPose EstimationTransformerPrompt EngineeringDiffusion modelNeural Radiance FieldAuto EncoderGenerative Adversarial NetworkContrastive LearningImagePoint CloudMesh
🎯 What it does: Achieve high-fidelity 3D garment reconstruction from a single image, utilizing a hierarchical framework to progressively generate the garment mesh from coarse to fine;
Gaussian Opacity Fields: Efficient Adaptive Surface Reconstruction in Unbounded Scenes
Zehao Yu (University of Tübingen), Andreas Geiger (University of Tübingen)
GenerationDepth EstimationOptimizationComputational EfficiencyDiffusion modelNeural Radiance FieldGaussian SplattingOptical FlowImagePoint CloudMesh
🎯 What it does: This paper proposes Gaussian Opacity Fields (GOF), which directly generate visually consistent opacity fields from 3D Gaussian distributions using the volumetric rendering formula with ray-Gaussian intersection, and designs an adaptive mesh extraction process based on tetrahedral meshes and Marching Tetrahedra, achieving high-quality surface reconstruction and novel view synthesis for unbounded scenes.
Gaussian Surfel Splatting for Live Human Performance Capture
Zheng Dong, Rynson W. H. Lau
GenerationPose EstimationDepth EstimationComputational EfficiencyConvolutional Neural NetworkTransformerDiffusion modelNeural Radiance FieldAuto EncoderGaussian SplattingImageVideoPoint Cloud
🎯 What it does: Proposes a general neural network method that can reconstruct and real-time render high-quality human performances from sparse RGB-D streams.
GaussianHeads: End-to-End Learning of Drivable Gaussian Head Avatars from Coarse-to-fine Representations
Kartik Teotia (Max Planck Institute for Informatics), Christian Theobalt (Max Planck Institute for Informatics)
GenerationPose EstimationRepresentation LearningConvolutional Neural NetworkTransformerDiffusion modelNeural Radiance FieldAuto EncoderContrastive LearningGaussian SplattingImagePoint CloudMesh
🎯 What it does: Studied an end-to-end method based on 3D Gaussian rendering, capable of generating controllable high-quality human face avatars in real-time from multi-view images, supporting instant driving of expressions, poses, and viewpoints.
GaussianObject: High-Quality 3D Object Reconstruction from Four Views with Gaussian Splatting
Chen Yang (Shanghai Jiao Tong University), Qi Tian (Huawei)
GenerationData SynthesisPose EstimationDepth EstimationComputational EfficiencyTransformerSupervised Fine-TuningPrompt EngineeringDiffusion modelNeural Radiance FieldGaussian SplattingSimultaneous Localization and MappingOptical FlowImagePoint CloudMesh
🎯 What it does: Propose the GaussianObject framework, which can achieve high-quality 3D object reconstruction and rendering using only four perspective images.
GauWN: Gaussian-smoothed Winding Number and its Derivatives
Haoran Sun, Jin Huang
Diffusion modelScore-based ModelGaussian SplattingOptical FlowMesh
🎯 What it does: Proposed the Gaussian-smoothed differentiable winding number (GauWN), for handling potentially self-intersecting 2D polygons, and achieving efficient computation of its value and derivatives;
gDist: Efficient Distance Computation between 3D Meshes on GPU
Peng Fan (Zhejiang University), Mingbo Tang
OptimizationComputational EfficiencyPoint CloudMesh
🎯 What it does: This paper proposes an efficient algorithm for 3D mesh distance computation based on GPU, called gDist, which can perform maximum/minimum distance queries in milliseconds on a single RTX 4090 GPU, supporting complex models with up to 1.5 M triangles.
Generative Portrait Shadow Removal
Jae Shin Yoon (Adobe Inc.), He Zhang (Adobe Inc.)
RestorationConvolutional Neural NetworkTransformerSupervised Fine-TuningDiffusion modelAuto EncoderImage
🎯 What it does: A global generative portrait shadow removal method based on diffusion models was developed, which can simultaneously remove self-occlusion and external occlusion shadows while preserving the original illumination and high-frequency details.
Geometry-Aware Retargeting for Two-Skinned Characters Interaction
Inseok Jang, Jun-yong Noh
GenerationData SynthesisTransformerDiffusion modelContrastive LearningMesh
🎯 What it does: A spatially cooperative transformer (SCT) is proposed to remap interaction actions between two roles connected on arbitrary grids, and an anchor loss is introduced to maintain the geometric distance between interacting roles, while a deformation-driven action enhancement method is proposed to generate source-target paired datasets.
GFFE: G-buffer Free Frame Extrapolation for Low-latency Real-time Rendering
Songyin Wu (University of California Santa Barbara), Ling-Qi Yan (University of California Santa Barbara)
Computational EfficiencyConvolutional Neural NetworkDiffusion modelNeural Radiance FieldAuto EncoderGaussian SplattingOptical FlowVideo
🎯 What it does: Proposes a GFFE framework, which is a G-buffer-free frame extrapolation method, to achieve low-latency real-time rendering without increasing the keyframe delay.
GGHead: Fast and Generalizable 3D Gaussian Heads
Tobias Kirschstein (Technical University of Munich), Matthias Nießner (Technical University of Munich)
GenerationData SynthesisConvolutional Neural NetworkDiffusion modelNeural Radiance FieldAuto EncoderGenerative Adversarial NetworkContrastive LearningGaussian SplattingImageMesh
🎯 What it does: Developed GGHead, a fast and general 3D head generator based on 3D Gaussian Splatting, which can learn high-quality, full-resolution 3D head models from a set of 2D images and render them in real-time.
GPU Coroutines for Flexible Splitting and Scheduling of Rendering Tasks
Shaokun Zheng, Kun Xu
OptimizationComputational Efficiency
🎯 What it does: Introducing coroutines into GPU core programming, providing an automatic splitting and scheduling scheme for rendering tasks, solving the tediousness and errors of traditional manual splitting of large kernels.
GroomCap: High-Fidelity Prior-Free Hair Capture
Yuxiao Zhou (ETH Zurich), T. Beeler
RestorationGenerationConvolutional Neural NetworkTransformerDiffusion modelScore-based ModelNeural Radiance FieldAuto EncoderContrastive LearningGaussian SplattingImageVideoMeshFibre Orientation Distribution
🎯 What it does: Proposes a multi-view hair capture method named GroomCap, which can reconstruct high-fidelity, detail-rich single-object hair geometry without relying on external databases.
GS3: Efficient Relighting with Triple Gaussian Splatting
Zoubin Bi (Zhejiang University), Hongzhi Wu (Zhejiang University)
GenerationComputational EfficiencyRepresentation LearningDiffusion modelNeural Radiance FieldAuto EncoderGaussian SplattingImagePoint Cloud
🎯 What it does: Propose a Triple Gaussian Splatting framework based on spatial Gaussian and angular Gaussian, enabling the learning of high-quality images that can be synthesized in real-time under arbitrary viewpoints and lighting conditions from multi-view point-light source photographs.
Hairmony: Fairness-aware hairstyle classification
Givi Meishvili (Microsoft), Marta Wilczkowiak (Microsoft)
ClassificationTransformerSupervised Fine-TuningContrastive LearningImage
🎯 What it does: A method for hairstyle classification based on a single image was studied, and a fair evaluation hairstyle classification system was constructed and a model was trained.
HFH-Font: Few-shot Chinese Font Synthesis with Higher Quality, Faster Speed, and Higher Resolution
Hua Li (Peking University), Zhouhui Lian (Peking University)
GenerationData SynthesisSuper ResolutionTransformerDiffusion modelScore-based ModelAuto EncoderGenerative Adversarial NetworkImageText
🎯 What it does: To address the problem of few-shot Chinese font generation, the HFH-Font system is proposed, which can generate high-resolution (1024×1024) font images in one go and further convert them into high-quality vector fonts.
Hierarchical Light Sampling with Accurate Spherical Gaussian Lighting
Yusuke Tokuyoshi, Takahiro Harada
Gaussian Splatting
🎯 What it does: A hierarchical light sampling method based on spherical Gaussian illumination is proposed for more accurate significance sampling in multi-light source rendering.
High-quality Animatable Eyelid Shapes from Lightweight Captures
Junfeng Lyu (Tsinghua University), Feng Xu (Tsinghua University)
RestorationGenerationPose EstimationDepth EstimationConvolutional Neural NetworkTransformerDiffusion modelNeural Radiance FieldAuto EncoderGenerative Adversarial NetworkContrastive LearningGaussian SplattingOptical FlowImageVideoMesh
🎯 What it does: This paper proposes a complete method that can achieve high-quality animatable eyelid shape reconstruction and animation using only RGB video from a smartphone.
High-Throughput Batch Rendering for Embodied AI
Luc Guy Rosenzweig, Kayvon Fatahalian
Computational EfficiencyRobotic IntelligenceReinforcement Learning from Human FeedbackGaussian SplattingWorld ModelOptical FlowPoint CloudMeshBenchmark
🎯 What it does: Studied the problem of efficiently rendering images in embodied AI training workloads, designed a flexible batch rendering interface, built and compared two high-performance GPU-based renderers, and proposed a corresponding rendering benchmark.
Hodge decomposition of vector fields in Cartesian grids
Zhe Su, Guowei Wei
Biomedical DataPhysics Related
🎯 What it does: Proposes a framework for implementing a 5-component L2 orthogonal Hodge decomposition on a Cartesian grid, unifying normal and tangential components, applicable to scalar and vector fields represented implicitly.
HyperGAN-CLIP: A Unified Framework for Domain Adaptation, Image Synthesis and Manipulation
Abdul Basit Anees (Koç University), Aykut Erdem (Koç University)
Image TranslationGenerationData SynthesisDomain AdaptationTransformerMixture of ExpertsDiffusion modelGenerative Adversarial NetworkContrastive LearningImageTextMultimodality
🎯 What it does: Proposes HyperGAN-CLIP, a unified framework that can accomplish multi-domain adaptation, reference image synthesis, and text-guided image editing under single-sample conditions.
I2VEdit: First-Frame-Guided Video Editing via Image-to-Video Diffusion Models
Wenqi Ouyang (Nanyang Technological University), Xingang Pan (Nanyang Technological University)
Image TranslationRestorationGenerationTransformerSupervised Fine-TuningDiffusion modelOptical FlowImageVideoText
🎯 What it does: This paper proposes I2VEdit, the first frame-guided video editing framework based on image-to-video diffusion models, which can automatically propagate any edits made to the first frame by users in any powerful image editing tool (such as EditAnything) throughout the entire video while maintaining consistency in appearance and motion with the original video.
Identifying Behavioral Correlates to Visual Discomfort
David A. Tovar, Phillip Guan
OptimizationExplainability and InterpretabilityRepresentation LearningContrastive LearningOptical FlowImageVideo
🎯 What it does: Introduced a custom VR headset that induces visual discomfort through near-eye optical distortion (pupil swimming) in typical VR experiences, and evaluated visual comfort while users played 'Job Simulator'; validated that dynamic distortion correction using eye-tracking improves visual comfort in within-subject experimental comparisons across multiple sessions; used representational similarity analysis to reveal that changes in head and gaze behavior may be more sensitive indicators of visual discomfort than questionnaires.
InfNeRF: Towards Infinite Scale NeRF Rendering with O(log n) Space Complexity
Jiabin Liang (Sea AI Lab), Xiangyu Xu (Xi'an Jiaotong University)
Data SynthesisComputational EfficiencyDiffusion modelScore-based ModelNeural Radiance FieldAuto EncoderGenerative Adversarial NetworkContrastive LearningGaussian SplattingOptical FlowImagePoint CloudMesh
🎯 What it does: Proposes InfNeRF, a NeRF structure based on a hierarchical quadtree, which can render large-scale scenes at any scale with O(log n) space complexity;
InstanceTex: Instance-level Controllable Texture Synthesis for 3D Scenes via Diffusion Priors
Mingxin Yang, Hui Huang
GenerationData SynthesisConvolutional Neural NetworkTransformerPrompt EngineeringDiffusion modelImageMesh
🎯 What it does: Propose a method called InstanceTex based on instance-level controllable texture synthesis, which can generate high-quality, style-consistent textures for large 3D scenes containing multiple objects.
InstantDrag: Improving Interactivity in Drag-based Image Editing
Joonghyuk Shin (Seoul National University), Jaesik Park (Seoul National University)
Image TranslationImage HarmonizationRestorationGenerationTransformerDiffusion modelAuto EncoderGenerative Adversarial NetworkContrastive LearningOptical FlowImageVideo
🎯 What it does: Proposed an instant image editing framework called InstantDrag, which requires no optimization and only needs images and drag instructions.
Inverse Painting: Reconstructing The Painting Process
Bowei Chen (University of Washington), Steven M. Seitz (University of Washington)
RestorationGenerationTransformerPrompt EngineeringVision Language ModelDiffusion modelImageVideoText
🎯 What it does: This study proposes an inverse painting method that uses diffusion models to reconstruct the painting process from a final artwork and generate time-decay videos.
Inverse Rendering for Tomographic Volumetric Additive Manufacturing
Baptiste Nicolet, Wenzel Jakob
OptimizationComputational EfficiencyDiffusion modelNeural Radiance FieldOptical FlowImageComputed TomographyPhysics Related
🎯 What it does: Compute and realize the light patterns of TVAM as an inverse illumination transport problem to generate high-fidelity printed objects
iSeg: Interactive 3D Segmentation via Interactive Attention
Itai Lang (University of Chicago), Rana Hanocka (University of Chicago)
SegmentationKnowledge DistillationTransformerContrastive LearningPoint CloudMesh
🎯 What it does: This paper proposes iSeg, an interactive 3D mesh segmentation method based on user clicks, which can generate fine-grained and user-intention-aligned segmentation results under one or multiple positive and negative clicks.
L3DG: Latent 3D Gaussian Diffusion
Barbara Roessle (Technical University of Munich), Matthias Niessner (Technical University of Munich)
GenerationData SynthesisConvolutional Neural NetworkDiffusion modelAuto EncoderGaussian SplattingPoint CloudMesh
🎯 What it does: Propose a 3D Gaussian latent diffusion model based on sparse convolutional VQ-VAE compressed space, which can generate high-quality 3D Gaussian scenes at the object level and room level in an unsupervised manner, and achieve real-time rendering.
Large Étendue 3D Holographic Display with Content-adaptive Dynamic Fourier Modulation
Brian Chao (Stanford University), G. Wetzstein (Stanford University)
OptimizationDiffusion modelNeural Radiance FieldOptical FlowImageVideoPoint CloudPhysics Related
🎯 What it does: This paper designs and implements a large étendue 3D holographic display system, utilizing multi-source laser illumination combined with a programmable amplitude modulation Fourier plane, and jointly optimizing phase SLM and amplitude SLM to achieve high-quality 3D holographic image display under wide viewing angles and large eye boxes.
Large Scale Farm Scene Modeling from Remote Sensing Imagery
Zhiqing Xiao, Zhaoqi Wang
Image TranslationGenerationData SynthesisTransformerDiffusion modelAuto EncoderContrastive LearningImageAgriculture Related
🎯 What it does: An extensible framework for large-scale farm scene modeling is proposed, which utilizes remote sensing satellite images to accurately extract and classify scene elements into four levels (fields, trees, roads, grasslands). A controllable Parametric Layout Model (PLM) is introduced for each level, capable of learning layout parameters and generating realistic multi-scale farm scenes. Additionally, the framework provides intuitive user control to simulate different crop growth stages and planting patterns.
Learn to Create Simple LEGO Micro Buildings
Jiahao Ge, Chi-Wing Fu
GenerationData SynthesisTransformerPrompt EngineeringDiffusion modelScore-based ModelAuto EncoderGenerative Adversarial NetworkPoint CloudMesh
🎯 What it does: Proposed a learning-based generative pipeline for efficiently creating 3D LEGO miniature building models.
Learned Multi-aperture Color-coded Optics for Snapshot Hyperspectral Imaging
Zheng Shi, Yifan Peng
OptimizationDiffusion modelScore-based ModelFlow-based ModelRectified FlowNeural Radiance FieldAuto EncoderContrastive LearningOptical FlowImageComputed TomographyPhysics RelatedStochastic Differential EquationOrdinary Differential Equation
🎯 What it does: Propose a snapshot hyperspectral imaging system that combines a multi-channel lens array with an aperture-level color filter, and collaborates with an image reconstruction network for optimization;
Learning Based Toolpath Planner on Diverse Graphs for 3D Printing
Yuming Huang (University of Manchester), Charlie C. L. Wang (University of Manchester)
Graph Neural NetworkReinforcement LearningDiffusion modelScore-based ModelContrastive LearningMeshGraph
🎯 What it does: Proposed a tool path planner based on reinforcement learning, capable of generating optimal printing sequences in real-time for different structures and scales of 3D printing models (such as wireframes, continuous carbon fiber, and metal laser printing).
LetsGo: Large-Scale Garage Modeling and Rendering via LiDAR-Assisted Gaussian Primitives
Jiadi Cui (ShanghaiTech University), Jingyi Yu (ShanghaiTech University)
Autonomous DrivingOptimizationComputational EfficiencyNeural Radiance FieldGaussian SplattingImagePoint CloudMesh
🎯 What it does: Designed and implemented a LiDAR-assisted 3D Gaussian splatting framework called LetsGo for high-precision modeling and rendering of large garage scenes, and introduced the Polar scanner equipped with IMU, LiDAR, and fisheye camera, along with the GarageWorld dataset.
LLM-enhanced Scene Graph Learning for Household Rearrangement
Wenhao Li (National University of Defense Technology), Kai Xu (National University of Defense Technology)
Autonomous DrivingOptimizationExplainability and InterpretabilityRobotic IntelligenceGraph Neural NetworkTransformerLarge Language ModelPrompt EngineeringVision Language ModelImageTextMultimodalityGraphRetrieval-Augmented GenerationChain-of-Thought
🎯 What it does: This paper proposes to enhance scene graph learning by leveraging large language models, constructing an Augmented Empowerment Graph (AEG), to achieve indoor misplacement detection and item reorganization tasks without human intervention.
Local Gaussian Density Mixtures for Unstructured Lumigraph Rendering
Xiuchao Wu, Weiwei Xu
GenerationData SynthesisConvolutional Neural NetworkTransformerDiffusion modelNeural Radiance FieldAuto EncoderGaussian SplattingImagePoint Cloud
🎯 What it does: Proposes a new method for novel view synthesis in unstructured Lumigraph rendering using a local Gaussian density mixture model along rays.
Look Ma, no markers: holistic performance capture without the hassle
Charlie Hewitt (Microsoft), T. Baltrušaitis
Pose EstimationOptimizationConvolutional Neural NetworkTransformerDiffusion modelFlow-based ModelContrastive LearningOptical FlowImagePoint CloudMesh
🎯 What it does: Developed a full-body, markerless performance capture system that can automatically capture the complete 3D pose and shape of the body, face, hands, eyes, and tongue under arbitrary camera configurations.
Lumiere: A Space-Time Diffusion Model for Video Generation
Omer Bar-Tal (Google Research), Inbar Mosseri (Google Research)
GenerationConvolutional Neural NetworkTransformerPrompt EngineeringDiffusion modelImageVideoTextStochastic Differential Equation
🎯 What it does: Developed the Lumiere framework based on a pre-trained text-to-image diffusion model, which employs a space-time U-Net to generate complete-length videos (5 seconds, 80 frames) in one go, and supports multiple tasks such as text-to-video generation, image-to-video generation, video inpainting, and stylization.
LVCD: Reference-based Lineart Video Colorization with Diffusion Models
Zhitong Huang (City University of Hong Kong), Jing Liao (City University of Hong Kong)
GenerationData SynthesisTransformerPrompt EngineeringDiffusion modelAuto EncoderOptical FlowImageVideo
🎯 What it does: Ink sketch video coloring based on reference frames, generating long temporal and color-consistent animations using a pre-trained video diffusion model
MagicClay: Sculpting Meshes With Generative Neural Fields
Amir Barda (Tel Aviv University), Thibault Groueix (Adobe Research)
GenerationTransformerPrompt EngineeringDiffusion modelScore-based ModelNeural Radiance FieldGenerative Adversarial NetworkTextMesh
🎯 What it does: A tool called MagicClay is proposed, which combines triangle mesh and Signed Distance Field (SDF) hybrid representation to achieve local mesh sculpting based on text prompts while keeping the unedited regions unchanged.
Manifold Sampling for Differentiable Uncertainty in Radiance Fields
Linjie Lyu (Max-Planck-Institut für Informatik), C. Theobalt
OptimizationComputational EfficiencyRepresentation LearningNeural Radiance FieldGaussian SplattingImagePoint Cloud
🎯 What it does: Propose a differentiable uncertainty quantification method based on low-dimensional manifold sampling for training and inference in three-dimensional Gaussian radiance fields;
Markov-Chain Monte Carlo Sampling of Visibility Boundaries for Differentiable Rendering
Peiyu Xu, Shuang Zhao
OptimizationComputational EfficiencyMeshStochastic Differential Equation
🎯 What it does: Studies the problem of estimating visibility boundary path integrals in differentiable rendering using Markov Chain Monte Carlo (MCMC) methods.
MARS: Multi-sample Allocation through Russian roulette and Splitting
Joshua Meyer (Saarland University), P. Slusallek
OptimizationComputational EfficiencyImageBenchmark
🎯 What it does: This study proposes a multi-sample multi-importance sampling (MIS) sample allocation framework called MARS, which utilizes fixed-point iteration combined with Russian roulette and splitting to adaptively allocate spatially varying numbers of samples to each sampling strategy through local estimation;
MaskedMimic: Unified Physics-Based Character Control Through Masked Motion Inpainting
Chen Tessler (NVIDIA), X. Peng
GenerationRobotic IntelligenceReinforcement Learning from Human FeedbackTransformerReinforcement LearningDiffusion modelScore-based ModelAuto EncoderContrastive LearningVideoTextMultimodalityTime SeriesPhysics Related
🎯 What it does: Propose MaskedMimic, a unified physics-driven character control framework, which generates full-body animations by inpainting partial motion descriptions (such as keyframes, text, objects, etc.) to restore complete motion;
MATTopo: Topology-preserving Medial Axis Transform with Restricted Power Diagram
Ningna Wang (University of Texas at Dallas), Xiaohu Guo (University of Texas at Dallas)
OptimizationComputational EfficiencyDiffusion modelScore-based ModelAuto EncoderGenerative Adversarial NetworkContrastive LearningPoint CloudMeshBenchmark
🎯 What it does: Propose a 3D medial axis computation framework based on volume-restricted power diagram (RPD), which can preserve both external and internal features while maintaining topological consistency and achieving geometric convergence;
Measuring Human Motion Under Clothing
Luis Bolanos, Dinesh K. Pai
Pose EstimationOptical FlowBiomedical DataMagnetic Resonance ImagingComputed TomographyUltrasound
🎯 What it does: Proposed and evaluated a system called EMob, which estimates body motion hidden under clothing by directly attaching extremely small electromagnetic sensors to the skin; the system can also measure sensor motion under clothing layers or outside the clothing layer, and can achieve electromagnetic tracking inside a full-body scanner, combined with high-resolution shape measurement and body motion tracking under clothing.
Medial Skeletal Diagram: A Generalized Medial Axis Approach for Compact 3D Shape Representation
Minghao Guo (MIT), Wojciech Matusik (MIT)
OptimizationRepresentation LearningDiffusion modelAuto EncoderPoint CloudMesh
🎯 What it does: Propose a sparse yet complete 3D shape skeleton representation — Medial Skeletal Diagram (MSD), and provide an automated construction and global optimization process from arbitrary triangle meshes to MSD.
Millimetric Human Surface Capture in Minutes
Briac Toussaint, Jean-Sébastien Franco
Data SynthesisOptimizationComputational EfficiencyDiffusion modelNeural Radiance FieldAuto EncoderGaussian SplattingImagePoint CloudMesh
🎯 What it does: Propose an scalable neural surface radiance field method and a new dataset called MVMannequin, for achieving millimeter-level precision in human surface capture.
MiNNIE: a Mixed Multigrid Method for Real-time Simulation of Nonlinear Near-Incompressible Elastics
Liangwang Ruan, Bao Chen
Diffusion modelScore-based ModelOptical FlowMeshPhysics RelatedStochastic Differential EquationOrdinary Differential Equation
🎯 What it does: Propose the MiNNIE framework to achieve real-time simulation of nonlinear nearly incompressible elastic bodies;
MoA: Mixture-of-Attention for Subject-Context Disentanglement in Personalized Image Generation
Kuan-Chieh Jackson Wang, Kfir Aberman (Snap Inc.)
GenerationData SynthesisTransformerPrompt EngineeringMixture of ExpertsDiffusion modelContrastive LearningImageTextMultimodality
🎯 What it does: Propose a new architecture called Mixture-of-Attention (MoA), which enables multi-subject personalized image generation while preserving the prior capabilities of the original text-to-image diffusion models;
Monkey See, Monkey Do: Harnessing Self-attention in Motion Diffusion for Zero-shot Motion Transfer
Sigal Raab (Tel Aviv University), Daniel Cohen-Or (Tel Aviv University)
GenerationData SynthesisPose EstimationTransformerPrompt EngineeringDiffusion modelContrastive LearningVideoSequentialBenchmark
🎯 What it does: Proposed a zero-shot, unsupervised motion transfer method called MoMo, which leverages self-attention features from pre-trained motion diffusion models to transfer the silhouette of leader motion to follower motion while preserving the fine-grained features of the follower;
Motion-Driven Neural Optimizer for Prophylactic Braces Made by Distributed Microstructures
Xingjian Han, Charlie C. L. Wang
OptimizationRobotic IntelligenceSpiking Neural NetworkDiffusion modelAuto EncoderContrastive LearningGaussian SplattingOptical FlowPoint CloudMeshTime SeriesBiomedical DataMagnetic Resonance ImagingComputed TomographyReview/Survey Paper
🎯 What it does: This paper proposes a motion-driven neural network optimizer that utilizes time-deforming topology optimization to achieve customizable distributed microstructure protective orthoses.
MotionFix: Text-Driven 3D Human Motion Editing
Nikos Athanasiou (Max Planck Institute for Intelligent Systems), Gul Varol
GenerationPose EstimationRetrievalTransformerPrompt EngineeringVision Language ModelDiffusion modelVideoTextSequentialRetrieval-Augmented Generation
🎯 What it does: A method for 3D human motion editing based on natural language was studied and implemented, and the first text-driven motion editing dataset, MotionFix, was constructed.
mpcMech: Multi-Point Conjugation Mechanisms
Ke Chen, Ligang Liu
Optimization
🎯 What it does: Design and construct a geometric model of a multi-point conjugate mechanism through an optimization method to generate user-specified 1-3 degree-of-freedom motion, and verify its motion performance through 3D-printed prototypes.
Multi-level Partition of Unity on Differentiable Moving Particles
Jinjin He, Bo Zhu
OptimizationRepresentation LearningDiffusion modelNeural Radiance FieldOptical FlowMesh
🎯 What it does: Proposed a differentiable movable particle representation based on multi-layer partitioned unification (MPU) for modeling dynamic implicit geometries.
Multi-Resolution Real-Time Deep Pose-Space Deformation
Mianlun Zheng, J. Barbič
Pose EstimationComputational EfficiencyDiffusion modelAuto EncoderOptical FlowMesh
🎯 What it does: Proposes a hard real-time, multi-resolution skeletal-driven soft-body character mesh deformation technique.
MV2MV: Multi-View Image Translation via View-Consistent Diffusion Models
Youcheng Cai, Ligang Liu
Image TranslationTransformerDiffusion modelContrastive LearningGaussian SplattingImage
🎯 What it does: Proposes a multi-view image translation framework called MV2MV based on diffusion models, achieving view consistency and detail enhancement through self-supervised training and 3D Gaussian Splatting.
MVImgNet2.0: A Larger-scale Dataset of Multi-view Images
Yushuang Wu (CUHKSZ), Xiaoguang Han (CUHKSZ)
Data SynthesisPrompt EngineeringDiffusion modelScore-based ModelNeural Radiance FieldAuto EncoderGenerative Adversarial NetworkContrastive LearningGaussian SplattingSimultaneous Localization and MappingImageVideoPoint CloudBenchmark
🎯 What it does: Built and released the MVImgNet2.0 large-scale multi-view image dataset, containing 520k real objects across 515 categories, providing 360° capture, fine foreground segmentation, accurate camera poses, and high-quality dense point cloud annotations, and validated its effectiveness in improving 3D reconstruction models.
NASM: Neural Anisotropic Surface Meshing
Hongbo Li (Wayne State University), Zichun Zhong (Wayne State University)
OptimizationComputational EfficiencyNeural Architecture SearchGraph Neural NetworkDiffusion modelScore-based ModelAuto EncoderGenerative Adversarial NetworkContrastive LearningPoint CloudMeshGraph
🎯 What it does: Propose a high-dimensional Euclidean embedding method based on graph neural networks, which can capture anisotropy caused by curvature without providing curvature metrics, and perform feature-sensitive reconstruction through CVT with high-dimensional normal metrics in high-dimensional space, achieving automatic anisotropic mesh generation that preserves sharp angles and weak features.
NeRF-Casting: Improved View-Dependent Appearance with Consistent Reflections
Dor Verbin (Google), Jonathan T. Barron (Google)
GenerationData SynthesisDiffusion modelScore-based ModelFlow-based ModelRectified FlowNeural Radiance FieldAuto EncoderContrastive LearningGaussian SplattingImagePoint CloudMesh
🎯 What it does: Propose the NeRF-Casting method, which utilizes the reflection light cone for ray tracing in NeRF to generate high-quality view-dependent specular reflections, addressing the problem that traditional NeRF struggles with highlights and near-field reflections.
Neural Differential Appearance Equations
Chen Liu (University College London), Tobias Ritschel (University College London)
RestorationGenerationConvolutional Neural NetworkRecurrent Neural NetworkTransformerDiffusion modelAuto EncoderGenerative Adversarial NetworkContrastive LearningImageVideoTime SeriesOrdinary Differential Equation
🎯 What it does: Leverage neural ODE models to learn time-varying material appearance, generating RGB dynamic textures and learning re-lightable svBRDF through differentiable rendering, supporting time-evolving material relighting effects.
Neural Garment Dynamic Super-Resolution
Meng Zhang (Nanjing University of Science and Technology), Jun Li (Nanjing University of Science and Technology)
Super ResolutionGraph Neural NetworkDiffusion modelAuto EncoderGenerative Adversarial NetworkMeshGraph
🎯 What it does: Propose a dynamic super-resolution method based on Mesh-Graph-Net and Hyper-Net, generating high-resolution detailed wrinkle details using low-resolution clothing simulation and body motion input.
Neural Global Illumination via Superposed Deformable Feature Fields
Chuankun Zheng, Hujun Bao
GenerationDiffusion modelNeural Radiance FieldAuto EncoderGenerative Adversarial NetworkImage
🎯 What it does: Propose a neural rendering method that utilizes deformable neural feature fields to generate high-quality global illumination effects in dynamic scenes (such as spotlights, soft shadows, and indirect highlights).
Neural Implicit Reduced Fluid Simulation
Yuanyuan Tao, P. Kry
Computational EfficiencyDiffusion modelScore-based ModelRectified FlowNeural Radiance FieldAuto EncoderPhysics Related
🎯 What it does: This paper proposes a neural implicit dimensionality reduction fluid simulation method (NIRFS), which combines the implicit neural representation of fluid shapes with neural ordinary differential equations to model fluid dynamics in a low-dimensional latent space, thereby significantly reducing computational costs while preserving details.
Neural Kernel Regression for Consistent Monte Carlo Denoising
Pengju Qiao, Tao Liu
RestorationOptimizationImage
🎯 What it does: A kernel denoiser based on distribution-free kernel regression consistency theory is proposed for consistently removing Monte Carlo rendering noise.
Neural Laplacian Operator for 3D Point Clouds
Bo Pang (Peking University), Peng-Shuai Wang (Peking University)
OptimizationRepresentation LearningGraph Neural NetworkSupervised Fine-TuningContrastive LearningPoint CloudMesh
🎯 What it does: Propose a method that utilizes graph neural networks to learn the point cloud Laplacian operator (NeLo), directly learning edge weights and mass matrices on the K-nearest neighbor graph, enabling geometric processing without the need to construct a triangular mesh.
Neural Light Spheres for Implicit Image Stitching and View Synthesis
Ilya Chugunov (Princeton University), Felix Heide (Princeton University)
GenerationData SynthesisDiffusion modelNeural Radiance FieldAuto EncoderContrastive LearningGaussian SplattingOptical FlowImageVideoPoint Cloud
🎯 What it does: Directly fit panoramic videos along arbitrary paths during testing using a single-layer neural light sphere model, achieving both image stitching and view synthesis, supporting disparity, view-dependent illumination, and local scene motion.
Neural Product Importance Sampling via Warp Composition
Joey Litalien (McGill University), Iliyan Georgiev (Adobe Research)
OptimizationComputational EfficiencyConvolutional Neural NetworkTransformerDiffusion modelScore-based ModelFlow-based ModelImagePoint CloudMesh
🎯 What it does: Propose a product importance sampling method based on neural normalized flows, which improves sampling efficiency in rendering by approximating the product distribution of environment illumination and BRDF through cascading a small neural normalized flow (head warp) with a precomputed emitter tail warp.
NeuSmoke: Efficient Smoke Reconstruction and View Synthesis with Neural Transportation Fields
Jiaxiong Qiu, Bo-Ning Ren
GenerationData SynthesisConvolutional Neural NetworkDiffusion modelNeural Radiance FieldOptical FlowVideo
🎯 What it does: Proposes NeuSmoke, an efficient dynamic smoke reconstruction and novel view synthesis framework based on neural transport fields;
NFPLight: Deep SVBRDF Estimation via the Combination of Near and Far Field Point Lighting
Li Wang, Jiawan Zhang
RestorationConvolutional Neural NetworkDiffusion modelAuto EncoderContrastive LearningImage
🎯 What it does: Proposed a co-location capture strategy that combines near-field and far-field point light sources, and achieved estimation of spatially varying bidirectional reflectance distribution functions (SVBRDF) through deep learning.
NPGA: Neural Parametric Gaussian Avatars
Simon Giebenhain (Technical University of Munich), Matthias Nießner (Technical University of Munich)
RestorationGenerationConvolutional Neural NetworkTransformerDiffusion modelScore-based ModelFlow-based ModelRectified FlowNeural Radiance FieldAuto EncoderContrastive LearningGaussian SplattingImageVideoMesh
🎯 What it does: Propose a high-fidelity, controllable facial avatar method based on Neural Parametric Gaussian Avatar (NPGA), combining 3D Gaussian Splatting with the MonoNPHM expression space.
NU-NeRF: Neural Reconstruction of Nested Transparent Objects with Uncontrolled Capture Environment
Jiali Sun, Lin Gao
RestorationGenerationDiffusion modelScore-based ModelFlow-based ModelRectified FlowNeural Radiance FieldAuto EncoderGenerative Adversarial NetworkContrastive LearningGaussian SplattingImageMesh
🎯 What it does: Propose NU-NeRF, which can reconstruct the geometric structure of nested transparent objects under uncontrolled acquisition environments, using a two-stage process that first separates reflection and refraction, and then performs explicit ray tracing to obtain internal surfaces;
Occupancy-Based Dual Contouring
Jisung Hwang (KAIST), Minhyuk Sung (KAIST)
OptimizationComputational EfficiencyPoint CloudMesh
🎯 What it does: Propose Occupancy-Based Dual Contouring (ODC), which improves 1D binary search and 2D point search, utilizing auxiliary points in 3D grids to construct QEF, specifically designed for occupancy functions;
OLAT Gaussians for Generic Relightable Appearance Acquisition
Zhiyi Kuang, Youyi Zheng
Neural Radiance FieldGaussian SplattingImageMesh
🎯 What it does: Construct a real-time relightable object representation from multi-view OLAT images and render it using 3D Gaussian Splatting.
Online Neural Denoising with Cross-Regression for Interactive Rendering
Hajin Choi, Bochang Moon
RestorationConvolutional Neural NetworkSupervised Fine-TuningAuto EncoderImage
🎯 What it does: Propose an online neural network denoising framework that updates the network parameters in real-time during image rendering using runtime image sequences.
Optimized shock-protecting microstructures
Zizhou Huang (New York University), Denis Zorin (New York University)
OptimizationDiffusion modelScore-based ModelAuto EncoderContrastive LearningTabularTime SeriesPhysics Related
🎯 What it does: Designed a series of two-dimensional microstructure families that can achieve near-constant force-deformation curves for periodic impact protection.
PALP: Prompt Aligned Personalization of Text-to-Image Models
Moab Arar (Tel-Aviv University), Ariel Shamir (Google Research)
GenerationData SynthesisTransformerSupervised Fine-TuningPrompt EngineeringDiffusion modelScore-based ModelImageTextMultimodality
🎯 What it does: This paper proposes a Prompt Aligned Personalization (PALP) method for text-to-image diffusion models, which can achieve subject personalization using only a small number of individual images, while maintaining high alignment with specified text prompts.
Pano2Room: Novel View Synthesis from a Single Indoor Panorama
Guo Pu (Peking University), Zhouhui Lian (Peking University)
GenerationData SynthesisDepth EstimationTransformerSupervised Fine-TuningDiffusion modelNeural Radiance FieldGaussian SplattingImagePoint CloudMesh
🎯 What it does: Pano2Room automatically generates complete 3D Gaussian Splatting scenes from a single indoor panoramic image and supports high-quality panoramic view synthesis.
ParSEL: Parameterized Shape Editing with Language
Aditya Ganeshan (Brown University), Daniel Ritchie (Brown University)
GenerationOptimizationComputational EfficiencyTransformerLarge Language ModelPrompt EngineeringDiffusion modelTextMeshRetrieval-Augmented GenerationChain-of-Thought
🎯 What it does: This paper proposes the ParSEL system, which can generate parameterizable 3D shape editing programs based on natural language editing requests, and control the editing magnitude in real time through adjustable parameters.
Particle-Laden Fluid on Flow Maps
Zhiqi Li (Georgia Institute of Technology), Bo Zhu (Georgia Institute of Technology)
Diffusion modelScore-based ModelFlow-based ModelRectified FlowAuto EncoderGenerative Adversarial NetworkContrastive LearningGaussian SplattingOptical FlowImageVideoPhysics RelatedStochastic Differential EquationOrdinary Differential Equation
🎯 What it does: This paper proposes a novel framework based on particle flow graphs to simulate ink flow containing settling particles (i.e., particle-laden flow), and successfully achieves precise numerical solutions for viscosity, drag force, and multiphase coupling.
PC-Planner: Physics-Constrained Self-Supervised Learning for Robust Neural Motion Planning with Shape-Aware Distance Function
Xujie Shen (Zhejiang University), Zhaopeng Cui (Zhejiang University)
OptimizationRobotic IntelligenceDiffusion modelScore-based ModelContrastive LearningSimultaneous Localization and MappingOptical FlowPoint CloudMeshTime SeriesPhysics RelatedStochastic Differential EquationOrdinary Differential Equation
🎯 What it does: Propose PC-Planner, a self-supervised learning framework based on physical constraints, for neural motion planning of robots with arbitrary shapes in complex environments.
PCO: Precision-Controllable Offset Surfaces with Sharp Features
Lei Wang, Changhe Tu
Diffusion modelScore-based ModelOptical FlowMeshStochastic Differential EquationOrdinary Differential Equation
🎯 What it does: Proposes a method for precise surface offset using triangular base distance fields and tetrahedral segmentation, capable of accurately preserving sharp features and achieving simplification.
PDP: Physics-Based Character Animation via Diffusion Policy
Takara E. Truong (Stanford University), Karen Liu (Stanford University)
Robotic IntelligenceTransformerReinforcement LearningDiffusion modelVideoTextMultimodalityPhysics Related
🎯 What it does: This paper proposes the PDP method, combining RL with diffusion models to achieve human motion control under physical simulation.
PersonaTalk: Bring Attention to Your Persona in Visual Dubbing
Longhao Zhang (Bytedance), Tianshu Hu (Bytedance)
Image TranslationRestorationGenerationTransformerDiffusion modelAuto EncoderGenerative Adversarial NetworkContrastive LearningOptical FlowImageVideoAudio
🎯 What it does: Proposes a two-stage attention framework called PersonaTalk for visual dubbing, which can generate high-fidelity videos with lip-sync that preserve the speaker's personalized characteristics.
Perspective-Aligned AR Mirror with Under-Display Camera
Jian Wang, Gurunandan Krishnan
Image TranslationRestorationData SynthesisDiffusion modelOptical FlowImage
🎯 What it does: Propose placing the camera behind a transparent display to achieve viewpoint alignment, and design an image restoration algorithm and a complete AR mirror system, conducting user studies.
Planar Reflection-Aware Neural Radiance Fields
Chen Gao (Meta), Johannes Kopf (Meta)
RestorationVision-Language-Action ModelDiffusion modelNeural Radiance FieldAuto EncoderContrastive LearningGaussian SplattingOptical FlowImagePoint Cloud
🎯 What it does: Propose a plane reflection-aware neural radiance field (RA-NeRF), which explicitly projects reflection rays when the main ray intersects with a plane, jointly modeling plane reflection and scene geometry, enabling accurate geometry in the main view and the synthesis of realistic high-frequency reflections independently.
Polar Interpolants for Thin-Shell Microstructure Homogenization
Antoine Chan-Lock (Universidad Rey Juan Carlos), M. Otaduy
MeshTabularPhysics Related
🎯 What it does: Aiming at the macroscopic behavior of thin-shell microstructures, this paper proposes a conservative energy-based homogenized material model, and employs high-order RBF interpolation in polar coordinates to achieve precise fitting of the principal strain domain.
Polarimetric BSSRDF Acquisition of Dynamic Faces
Hyunho Ha (KAIST), Min H. Kim (KAIST)
OptimizationDiffusion modelNeural Radiance FieldAuto EncoderContrastive LearningOptical FlowImageVideoMultimodalityPhysics Related
🎯 What it does: This paper proposes a polarization BSSRDF acquisition method for dynamic faces, capable of simultaneously capturing facial geometry, spatially varying polarization appearance parameters, and biophysically based skin parameters.
Polynomial Cauchy Coordinates for Curved Cages
Zhehui Lin, Renjie Chen
🎯 What it does: Extend the Cauchy barycentric coordinates to curvilinear domains represented by Bézier curves, derive a numerical integration formula for the inverse mapping, and provide expressions for higher-order derivatives of the coordinates, used for deformation of curvilinear domains.
Portrait Video Editing Empowered by Multimodal Generative Priors
Xuan Gao (University of Science and Technology of China), Juyong Zhang (University of Science and Technology of China)
GenerationData SynthesisTransformerPrompt EngineeringDiffusion modelNeural Radiance FieldGaussian SplattingOptical FlowImageVideoTextMultimodality
🎯 What it does: Propose PortraitGen, a method for consistent and high-quality portrait video editing that leverages multi-modal generative priors.
Procedural Material Generation with Reinforcement Learning
Beichen Li, Wojciech Matusik
GenerationData SynthesisTransformerReinforcement LearningImage
🎯 What it does: Improve the process material parameter prediction in image conditioning using reinforcement learning (RL), where a RL-trained Transformer model generates parameters to reconstruct the target image.
ProcessPainter: Learning to draw from sequence data
Yiren Song, Mike Zheng Shou
GenerationData SynthesisTransformerSupervised Fine-TuningVision Language ModelDiffusion modelImageVideoTextSequential
🎯 What it does: Propose ProcessPainter, a text-to-video model that can generate a step-by-step painting process based on textual prompts, and introduce the Artwork Replication Network to achieve painting process control, image segmentation, and completion of the artwork with arbitrary frame input.