SIGGRAPH Asia 2024 Papers — Page 3
ACM SIGGRAPH Asia (Transactions on Graphics) · 265 papers
Projected Walk on Spheres: A Monte Carlo Closest Point Method for Surface PDEs
Ryusuke Sugimoto (University of Waterloo), Christopher Batty (University of Waterloo)
OptimizationComputational EfficiencyPoint CloudMeshPhysics Related
🎯 What it does: Developed a Projected Walk on Spheres (PWoS) algorithm, which uses Monte Carlo recursion and nearest point projection to solve surface PDEs (Laplace, Poisson, screened Poisson, etc.) embedded in three-dimensional space in a point-based, mesh-free manner.
PuzzleAvatar: Assembling 3D Avatars from Personal Albums
Yuliang Xiu (Max Planck Institute for Intelligent Systems), Michael J. Black (Max Planck Institute for Intelligent Systems)
GenerationData SynthesisTransformerSupervised Fine-TuningPrompt EngineeringVision Language ModelDiffusion modelScore-based ModelImageTextPoint CloudMesh
🎯 What it does: Generate 3D human avatar portraits by synthesizing complete, personalized 3D characters directly from an individual's 'Today's Outfit' photo album;
PVP-Recon: Progressive View Planning via Warping Consistency for Sparse-View Surface Reconstruction
Sheng Ye (Tsinghua University), Wenping Wang (Texas A&M University)
OptimizationComputational EfficiencyRepresentation LearningConvolutional Neural NetworkTransformerDiffusion modelScore-based ModelNeural Radiance FieldContrastive LearningGaussian SplattingImagePoint CloudMesh
🎯 What it does: Propose a sparse view surface reconstruction system called PVP-Recon, which can gradually plan and add the most informative views during the reconstruction process, ultimately obtaining a high-quality 3D mesh.
Q3T Prisms: A Linear-Quadratic Solid Shell Element for Elastoplastic Surfaces
Juan Sebastian Montes Maestre, Bernhard Thomaszewski
Physics Related
🎯 What it does: A technique utilizing a quadratic thickness-direction solid shell element (Q3T) to simulate elastic-plastic surfaces is proposed.
Quad mesh mechanisms
Caigui Jiang, J. Wallner
OptimizationMesh
🎯 What it does: Provides computational tools for modeling and designing quadrilateral mesh mechanisms, facilitating continuous flexible deformation.
Quark: Real-time, High-resolution, and General Neural View Synthesis
John Flynn (Google), Ryan Overbeck (Google)
GenerationComputational EfficiencyRepresentation LearningConvolutional Neural NetworkTransformerDiffusion modelNeural Radiance FieldAuto EncoderGaussian SplattingOptical FlowImageVideo
🎯 What it does: Propose a real-time high-resolution general neural view synthesis model named Quark, which can reconstruct 3D scenes in real-time from sparse multi-view images and generate 1080p views.
Real-time Large-scale Deformation of Gaussian Splatting
Lin Gao, Yu-Kun Lai
Gaussian SplattingMesh
🎯 What it does: Developed an interactive deformation method called GaussianMesh based on Gaussian Splatting, introducing a mesh-oriented Gaussian representation and large-scale Gaussian deformation techniques;
Reconstructing translucent thin objects from photos
Xi Deng, A. Weidlich
OptimizationComputational EfficiencyData-Centric LearningDiffusion modelScore-based ModelNeural Radiance FieldOptical FlowImage
🎯 What it does: Proposed an affordable and fast acquisition pipeline for simultaneously capturing spatially varying reflectance and transmittance, and achieved joint reconstruction of shape and appearance of thin semi-transparent objects through a two-stage optimization (first initializing geometry with traditional visual methods and fitting a simple and fast appearance model, then refining with a more expensive volumetric model); meanwhile, introduced parameter sensitivity analysis to noise for optimizing measurement selection, and adopted weighted L2 loss instead of camera iteration to select useful pixels.
Refined Inverse Rigging: A Balanced Approach to High-fidelity Blendshape Animation
Stevo Rackovi'c (Instituto Superior Técnico), Cláudia Soares (NOVA School of Science and Technology)
OptimizationDiffusion modelAuto EncoderMesh
🎯 What it does: A full-sequence optimization method based on fourth-order nonlinear blendshape, l1 sparse regularization, and second-order difference temporal roughness regularization is proposed to address the inverse rig problem of facial blendshape, achieving high-fidelity, sparse, and temporally smooth animation generation.
Representing Long Volumetric Video with Temporal Gaussian Hierarchy
Zhen Xu (Zhejiang University), Xiaowei Zhou (Zhejiang University)
CompressionComputational EfficiencyRepresentation LearningNeural Radiance FieldGaussian SplattingOptical FlowVideoPoint Cloud
🎯 What it does: Propose Temporal Gaussian Hierarchy (time Gaussian hierarchy) to efficiently compress and train long-term volumetric videos, maintaining constant GPU memory and storage, supporting real-time rendering;
ReVersion: Diffusion-Based Relation Inversion from Images
Ziqi Huang (Nanyang Technological University), Ziwei Liu (Nanyang Technological University)
GenerationData SynthesisTransformerSupervised Fine-TuningPrompt EngineeringDiffusion modelContrastive LearningImageTextBenchmark
🎯 What it does: This paper proposes the Relation Inversion task, which learns relational prompt tokens ⟨R⟩ from a few example images using a pre-trained text-to-image diffusion model, and applies them to generate new scenes.
Robot Motion Diffusion Model: Motion Generation for Robotic Characters
Agon Serifi, Switzerland Moritz Bächer
GenerationRobotic IntelligenceTransformerSupervised Fine-TuningReinforcement LearningVision-Language-Action ModelDiffusion modelText
🎯 What it does: Proposed a robot motion diffusion model (RobotMDM) that combines motion generation models with physical control, using a reward agent to predict the performance of non-differentiable control tasks, and utilizing this agent to fine-tune the baseline generation model, enabling the generated text-conditioned motion to be both diverse and physically constrained.
Robust Dual Gaussian Splatting for Immersive Human-centric Volumetric Videos
Yuheng Jiang (ShanghaiTech University), Lan Xu (ShanghaiTech University)
CompressionDiffusion modelAuto EncoderGaussian SplattingOptical FlowVideoPoint Cloud
🎯 What it does: Propose DualGS dual Gaussian profile, achieving robust tracking and high-fidelity rendering of human performance volumetric videos, with 120 times compression;
Robust Symmetry Detection via Riemannian Langevin Dynamics
Jihyeon Je (Stanford University), Leonidas J. Guibas
RecognitionDiffusion modelScore-based ModelImagePoint CloudMeshStochastic Differential Equation
🎯 What it does: Proposes a symmetry detection method based on Riemannian Langevin dynamics, which can robustly identify local and global reflective symmetry in noisy shapes.
RoMo: A Robust Solver for Full-body Unlabeled Optical Motion Capture
Xiaoyu Pan (Zhejiang University), Xiaogang Jin (Zhejiang University)
Pose EstimationOptimizationGraph Neural NetworkTransformerDiffusion modelScore-based ModelAuto EncoderGenerative Adversarial NetworkContrastive LearningOptical FlowPoint CloudMeshGraph
🎯 What it does: Developed the RoMo framework for robust markerization and solving of full-body markerless optical motion capture data.
SD-πXL: Generating Low-Resolution Quantized Imagery via Score Distillation
Alexandre Binninger (ETH Zurich), O. Sorkine-Hornung
GenerationData SynthesisTransformerPrompt EngineeringDiffusion modelScore-based ModelImageText
🎯 What it does: Propose a differentiable image generation framework based on score distillation called SD/u1D70B XL, used to generate low-resolution quantized images in the style of pixel art under strict color palettes and resolution constraints;
SGEdit: Bridging LLM with Text2Image Generative Model for Scene Graph-based Image Editing
Zhiyuan Zhang (City University of Hong Kong), Jing Liao (City University of Hong Kong)
Image TranslationImage HarmonizationSegmentationGenerationTransformerLarge Language ModelPrompt EngineeringDiffusion modelScore-based ModelImageTextGraphRetrieval-Augmented Generation
🎯 What it does: Proposed an image editing framework based on scene graphs, SGEdit, which utilizes large language models (LLMs) to construct and edit scene graphs, and combines text-to-image diffusion models to achieve high-quality object addition, deletion, replacement, and relationship modification.
Sharpening and Sparsifying with Surface Hessians
Dylan Rowe, Oded Stein
OptimizationMesh
🎯 What it does: Proposes a fully intrinsic discretization of the L1 Hessian energy and applies it to tasks such as stylization, denoising, interpolation, hole filling, and segmentation of triangle meshes.
SIGGesture: Generalized Co-Speech Gesture Synthesis via Semantic Injection with Large-Scale Pre-Training Diffusion Models
Qingrong Cheng (Tencent AI Lab), Xinghui Fu (Tencent AI Lab)
GenerationData SynthesisPose EstimationTransformerLarge Language ModelSupervised Fine-TuningDiffusion modelTextMultimodalityRetrieval-Augmented GenerationAudio
🎯 What it does: Under audio-driven conditions, a high-quality, rhythm-synchronized, and semantically accurate 3D gesture is generated using a diffusion model, and controllable co-speech gesture synthesis is achieved by injecting semantics through semantically generated gesture segments from an LLM.
SING: Stability-Incorporated Neighborhood Graph
Diana Marin, Pooran Memari
Anomaly DetectionRepresentation LearningGraph Neural NetworkContrastive LearningPoint Cloud
🎯 What it does: Proposed Stability Embedding Neighborhood Graph (SING), a novel density-aware structure for capturing the intrinsic geometric properties of point sets, and improved the spherical influence graph by incorporating additional features to provide more flexible nearest-neighbor encoding and local density capture.
SKEL-Betweener: a Neural Motion Rig for Interactive Motion Authoring
Dhruv Agrawal, Martine Guay
GenerationPose EstimationDiffusion modelNeural Radiance FieldAuto EncoderGenerative Adversarial NetworkSequential
🎯 What it does: Propose a neural motion retargeting tool called SKEL-Betweener, which can generate long-term motion sequences from just two poses, and achieve joint-level fine editing of intermediate frames through neural motion curves, suitable for interactive motion creation.
Skeleton-Driven Inbetweening of Bitmap Character Drawings
Kirill Brodt, Mikhail Bessmeltsev
GenerationPose EstimationDiffusion modelAuto EncoderGenerative Adversarial NetworkOptical FlowImage
🎯 What it does: Proposed a bitmap character drawing intermediate frame generation system based on skeletal driving, supporting tight and sparse intermediate frame interpolation;
Sketching With Your Voice: "Non-Phonorealistic" Rendering of Sounds via Vocal Imitation
Matthew Caren (MIT), Karima Ma (MIT)
GenerationData SynthesisRetrievalTransformerDiffusion modelScore-based ModelAuto EncoderAudio
🎯 What it does: Proposed a system for automatically generating human-like sound imitations, capturing the essence of the sound and converting it into an acoustically interpretable description for humans;
Solid-Fluid Interaction on Particle Flow Maps
Duowen Chen (Georgia Institute of Technology), Bo Zhu (Georgia Institute of Technology)
Diffusion modelScore-based ModelOptical FlowPoint CloudMeshPhysics RelatedStochastic Differential EquationOrdinary Differential Equation
🎯 What it does: Proposed a solid-fluid coupling method under the Particle Flow Map framework, unifying elastic solids and impulse fluids as flow maps of different lengths, and achieving the conversion from impulse to velocity and the accumulation of coupling forces through path integration.
Solving Inverse PDE Problems using Grid-Free Monte Carlo Estimators
Ekrem Fatih Yilmazer (École Polytechnique Fédérale de Lausanne), Wenzel Jakob (École Polytechnique Fédérale de Lausanne)
OptimizationTabularPhysics RelatedStochastic Differential EquationOrdinary Differential Equation
🎯 What it does: This paper proposes a mesh-free Monte Carlo estimator based on Walk on Spheres for solving inverse PDE problems, and provides a corresponding differentiable implementation.
SpaceMesh: A Continuous Representation for Learning Manifold Surface Meshes
Tianchang Shen, Nicholas Sharp
GenerationRepresentation LearningConvolutional Neural NetworkTransformerDiffusion modelScore-based ModelGenerative Adversarial NetworkContrastive LearningPoint CloudMesh
🎯 What it does: Propose a continuous representation method called SpaceMesh, which can directly generate polygon meshes satisfying manifold constraints from the output of a neural network.
SPARK: Self-supervised Personalized Real-time Monocular Face Capture
Kelian Baert (Technicolor Group), A. Boukhayma
RestorationPose EstimationRepresentation LearningConvolutional Neural NetworkTransformerNeural Radiance FieldAuto EncoderContrastive LearningOptical FlowVideoMesh
🎯 What it does: Propose a self-supervised personalized real-time monocular facial capture framework called SPARK based on multi-video, which can reconstruct high-precision facial geometry and material from multiple unconstrained videos. On this basis, a fast expression and pose regression network is trained to achieve real-time high-fidelity reconstruction on new images.
Spatiotemporal Bilateral Gradient Filtering for Inverse Rendering
Wesley Chang, Tzu-Mao Li
OptimizationOptical Flow
🎯 What it does: Proposed a spatiotemporal bilateral gradient filter to accelerate gradient descent optimization in inverse rendering
SRIF: Semantic Shape Registration Empowered by Diffusion-based Image Morphing and Flow Estimation
Mingze Sun, Ruqi Huang
GenerationPose EstimationOptimizationComputational EfficiencyGraph Neural NetworkTransformerDiffusion modelFlow-based ModelGaussian SplattingOptical FlowImagePoint CloudMesh
🎯 What it does: Propose the SRIF framework to achieve semantic shape registration and continuous shape interpolation without annotations or priors
StableNormal: Reducing Diffusion Variance for Stable and Sharp Normal
Chongjie Ye (Chinese University of Hongkong Shenzhen), Xiaoguang Han (Chinese University of Hongkong Shenzhen)
Pose EstimationDepth EstimationConvolutional Neural NetworkTransformerDiffusion modelScore-based ModelContrastive LearningImagePoint Cloud
🎯 What it does: Propose StableNormal, which estimates high-quality, stable, and sharp surface normals from monocular color images using diffusion models.
Still-Moving: Customized Video Generation without Customized Video Data
Hila Chefer (Google DeepMind), Inbar Mosseri (Google DeepMind)
GenerationData SynthesisTransformerPrompt EngineeringDiffusion modelImageVideoText
🎯 What it does: Proposes the Still-Moving framework, which can seamlessly inject custom text-image (T2I) model weights into text-video (T2V) models without using any custom video data, achieving video personalization, stylization, and conditional generation;
Stochastic Normal Orientation for Point Clouds
Guojin Huang, Xiao-Ming Fu
OptimizationPoint Cloud
🎯 What it does: Proposed a normal orientation method for point clouds
Stripe Embedding: Efficient Maps with Exact Numeric Computation
Marco Livesu
Computational EfficiencyDiffusion modelScore-based ModelFlow-based ModelRectified FlowNeural Radiance FieldAuto EncoderGenerative Adversarial NetworkContrastive LearningGaussian SplattingSimultaneous Localization and MappingWorld ModelOptical FlowMeshStochastic Differential EquationOrdinary Differential Equation
🎯 What it does: Designed an algorithm based on stripe embedding that maps surface meshes with disk topology to boundary-constrained convex domains, achieving precise numerical computation.
Style-NeRF2NeRF: 3D Style Transfer from Style-Aligned Multi-View Images
Haruo Fujiwara (University of Tokyo), Tatsuya Harada (University of Tokyo)
Image TranslationGenerationDepth EstimationTransformerDiffusion modelScore-based ModelNeural Radiance FieldImagePoint CloudMeshStochastic Differential Equation
🎯 What it does: Proposes a 3D style transfer pipeline based on deep diffusion models. First, it generates multi-view consistent stylized images using a deep conditional style-aligned image-to-image diffusion model, and then fine-tunes the existing NeRF with a sliced Wasserstein loss, enabling the 3D scene to exhibit a specified text style.
StyleCrafter: Taming Artistic Video Diffusion with Reference-Augmented Adapter Learning
Gongye Liu, Yujiu Yang
GenerationTransformerSupervised Fine-TuningPrompt EngineeringDiffusion modelImageVideoRetrieval-Augmented Generation
🎯 What it does: Propose the StyleCrafter method, which generates videos in any style by adding a style control adapter to a pre-trained T2V model and using a reference image.
StyleTex: Style Image-Guided Texture Generation for 3D Models
Zhiyu Xie (Zhejiang University), Xiaogang Jin (Zhejiang University)
Image TranslationGenerationTransformerPrompt EngineeringVision Language ModelDiffusion modelScore-based ModelNeural Radiance FieldContrastive LearningImageTextMesh
🎯 What it does: This paper proposes a 3D texture generation method based on a single reference style image and text prompts, capable of generating textures for any untextured mesh model that are consistent with the style of the reference image and match the model's geometry.
Surface Reconstruction Using Rotation Systems
Ruiqi Cui (Technical University Of Denmark), J. A. Bærentzen (Technical University Of Denmark)
Diffusion modelScore-based ModelAuto EncoderGenerative Adversarial NetworkContrastive LearningPoint CloudMesh
🎯 What it does: A composite surface reconstruction algorithm based on a rotational system is proposed, which constructs a tree from point clouds and gradually inserts edges to obtain a high-quality triangular mesh with controllable topology.
Synchronize Dual Hands for Physics-Based Dexterous Guitar Playing
Pei Xu (Stanford University), Ruocheng Wang (Stanford University)
Robotic IntelligenceTransformerReinforcement LearningAuto EncoderGenerative Adversarial NetworkTabularTime SeriesSequentialPhysics RelatedAudio
🎯 What it does: A reinforcement learning framework capable of physically realistically controlling both hands to play the guitar was constructed. First, the left-hand fingering strategy and the right-hand picking strategy were trained separately, and then hand coordination and synchronization were achieved by adjusting the latent space.
TALK-Act: Enhance Textural-Awareness for 2D Speaking Avatar Reenactment with Diffusion Model
Jiazhi Guan (Tsinghua University), Ziwei Liu (Nanyang Technological University)
GenerationPose EstimationTransformerPrompt EngineeringDiffusion modelVideo
🎯 What it does: Propose a 2D talking head reconstruction framework called TALK-Act based on diffusion models, which can achieve full-body and hand motion synchronization from extremely short videos.
Taming 3DGS: High-Quality Radiance Fields with Limited Resources
Saswat Subhajyoti Mallick (Carnegie Mellon University), Fernando de la Torre (Carnegie Mellon University)
Computational EfficiencyScore-based ModelNeural Radiance FieldGaussian SplattingImagePoint Cloud
🎯 What it does: Propose a budget-constrained 3D Gaussian splitting training method to control model size and resource consumption;
Tencers: Tension-Constrained Elastic Rods
Liliane-Joy Dandy, Mark Pauly
OptimizationDiffusion modelPhysics Related
🎯 What it does: Study the equilibrium state of elastic rods forming three-dimensional spatial curves under the tension of a small number of inextensible ropes, and propose an inverse design optimization algorithm;
TEXGen: a Generative Diffusion Model for Mesh Textures
Xin Yu (University of Hong Kong), Xiaojuan Qi (University of Hong Kong)
GenerationData SynthesisConvolutional Neural NetworkGraph Neural NetworkTransformerVision Language ModelDiffusion modelScore-based ModelAuto EncoderGenerative Adversarial NetworkContrastive LearningImageTextPoint CloudMesh
🎯 What it does: Trained and released a large-scale diffusion model (TEXGen) that can directly generate high-resolution (e.g., 1024×1024) texture maps in the UV space, and supports conditional inputs from text and single-view images;
Text-guided Controllable Mesh Refinement for Interactive 3D Modeling
Yun-Chun Chen (University of Toronto), Alec Jacobson (University of Toronto)
GenerationOptimizationComputational EfficiencyGraph Neural NetworkTransformerPrompt EngineeringVision Language ModelDiffusion modelScore-based ModelNeural Radiance FieldContrastive LearningGaussian SplattingImageTextPoint CloudMesh
🎯 What it does: Refine rough 3D meshes with text prompts to generate high-quality geometric details and controllable 3D meshes.
Text-Guided Texturing by Synchronized Multi-View Diffusion
Yuxin Liu (Chinese University of Hong Kong), Tien-Tsin Wong (Chinese University of Hong Kong)
GenerationData SynthesisTransformerPrompt EngineeringVision Language ModelDiffusion modelAuto EncoderImageTextMesh
🎯 What it does: By sharing UV texture information across multiple views in each denoising step through a synchronized multi-view diffusion model, generating 3D object textures based on text descriptions, solving the problems of texture fragments and seams caused by traditional progressive filling methods.
TextToon: Real-Time Text Toonify Head Avatar from Single Video
Luchuan Song (University of Rochester), Chenliang Xu (University of Rochester)
GenerationData SynthesisTransformerPrompt EngineeringVision Language ModelDiffusion modelContrastive LearningGaussian SplattingImageVideoText
🎯 What it does: Generate animatable cartoon avatars driven by text instructions using a single monocular video, achieving real-time rendering.
The Lips, the Teeth, the tip of the Tongue: LTT Tracking
Feisal Rasras, Javier Romero
GenerationData SynthesisDiffusion modelScore-based ModelNeural Radiance FieldAuto EncoderGenerative Adversarial NetworkImageVideoPoint CloudMesh
🎯 What it does: Proposed a grid-based generative model that covers upper and lower teeth, gums, tongue, and their placement within the human head, achieving highly accurate avatar reconstruction with human-specific details through oral scanning and multi-camera captured facial expressions data.
ThermOuch: A Wearable Thermo-Haptic Device for Inducing Pain Sensation in Virtual Reality through Thermal Grill Illusion
Haichen Gao, Kening Zhu
Human-Computer InteractionMultimodality
🎯 What it does: Developed a wearable thermo-tactile device called ThermOuch, which uses thermal brush illusion to simulate pain perception in VR, and evaluated its effectiveness through user perception experiments.
Time-Gated Polarization for Active Non-Line-Of-Sight Imaging
Oscar Pueyo-Ciutad, A. Redo-Sanchez
Optical FlowPhysics Related
🎯 What it does: Proposes a non-line-of-sight (NLOS) imaging method that combines polarization and time-of-flight measurements
ToonCrafter: Generative Cartoon Interpolation
Jinbo Xing (Chinese University of Hong Kong), Tien-Tsin Wong (Chinese University of Hong Kong)
GenerationData SynthesisConvolutional Neural NetworkTransformerSupervised Fine-TuningDiffusion modelAuto EncoderImageVideo
🎯 What it does: Proposed ToonCrafter, which achieves generative cartoon video interpolation based on a pre-trained live-action video diffusion model, capable of handling nonlinear motion and occlusion problems.
Towards Unified 3D Hair Reconstruction from Single-View Portraits
Yujian Zheng (Chinese University of Hong Kong, Shenzhen), Xiaoguang Han (Chinese University of Hong Kong, Shenzhen)
GenerationData SynthesisTransformerDiffusion modelScore-based ModelGaussian SplattingImageMesh
🎯 What it does: This paper proposes a unified single-view 3D hair reconstruction method, leveraging 3D Gaussian representations and diffusion model priors to achieve 3D reconstruction of different hairstyles (including braided and non-braided hair).
Trading Spaces: Adaptive Subspace Time Integration for Contacting Elastodynamics
Ty Trusty, Danny M. Kaufman
Physics Related
🎯 What it does: Built an adaptive subspace simulator that balances solution improvement with system scale, and proposed an adaptive subspace oracle, model, and parallel time-step solver.
TrailBlazer: Trajectory Control for Diffusion-Based Video Generation
W. Ma, W. Kleijn
GenerationData SynthesisTransformerPrompt EngineeringVision Language ModelDiffusion modelVideoText
🎯 What it does: Propose a trajectory control method called TrailBlazer based on a pre-trained video diffusion model, allowing users to control the motion trajectory, size changes, and appearance transitions of one or more objects in a video by only providing bounding boxes (bbox) and corresponding text prompts at keyframes.
Trust-Region Eigenvalue Filtering for Projected Newton
Honglin Chen (Columbia University), Changxi Zheng (Columbia University)
OptimizationMesh
🎯 What it does: Propose an adaptive eigenvalue filtering strategy based on the trust region for stabilizing and accelerating the projection Newton method in solving the non-convex optimization of Neo-Hookean energy;
Tune-It: Optimizing Wire Reconfiguration for Sculpture Manufacturing
Qibing Wu, Haisen Zhao
OptimizationDiffusion modelScore-based ModelFlow-based ModelRectified Flow
🎯 What it does: A novel metal wire sculpture manufacturing strategy is proposed, which first adjusts the target shape into a collision-free, machinable form, and then manually bends it back to the target shape to achieve minimal number of bends.
TurboEdit: Text-Based Image Editing Using Few-Step Diffusion Models
Gilad Deutch, Daniel Cohen-Or
Image TranslationImage HarmonizationRestorationGenerationTransformerPrompt EngineeringVision Language ModelDiffusion modelScore-based ModelImageText
🎯 What it does: This paper proposes TurboEdit, a text-driven image editing framework based on few-step diffusion models, which can complete edits within only 3–4 steps while maintaining high quality;
UFO Instruction Graphs Are Machine Knittable
Jenny Han Lin, James Mccann
OptimizationComputational EfficiencyRepresentation LearningAI Code AssistantPrompt EngineeringTextRetrieval-Augmented GenerationChain-of-Thought
🎯 What it does: Propose instruction graphs as an intermediate representation to implement an editor and compiler, which automatically convert high-level knitting descriptions into low-level machine instructions while ensuring semantic equivalence.
URAvatar: Universal Relightable Gaussian Codec Avatars
Junxuan Li (Meta), Shunsuke Saito (Meta)
GenerationCompressionTransformerSupervised Fine-TuningNeural Radiance FieldAuto EncoderGaussian SplattingImagePoint CloudMesh
🎯 What it does: Constructing a realistic, real-time relit and animatable head avatar using smartphone scans, supporting eye rotation, neck rotation, and potential expression control.
V^3: Viewing Volumetric Videos on Mobiles via Streamable 2D Dynamic Gaussians
Penghao Wang (ShanghaiTech University), Lan Xu (ShanghaiTech University)
CompressionGaussian SplattingOptical FlowVideoPoint Cloud
🎯 What it does: Propose the V3 method, which compresses dynamic 3D Gaussian point cloud sequences into multi-dimensional 2D videos, utilizes hardware video encoders and decoders to achieve streaming, and enables high-quality, real-time voxel video playback on mobile devices.
VidPanos: Generative Panoramic Videos from Casual Panning Videos
Jingwei Ma (University of Washington), Forrester Cole (Google DeepMind)
GenerationData SynthesisTransformerSupervised Fine-TuningDiffusion modelGenerative Adversarial NetworkSimultaneous Localization and MappingOptical FlowVideo
🎯 What it does: Propose a framework based on generative video models that projects arbitrarily captured translational videos onto a panoramic canvas and completes unknown spatial and temporal regions, generating a complete panoramic video.
Volume Scattering Probability Guiding
Kehan Xu, Markus Gross
OptimizationComputational EfficiencyNeural Radiance FieldPoint CloudBiomedical DataMagnetic Resonance ImagingComputed Tomography
🎯 What it does: Proposes an unbiased volume rendering algorithm based on directly controlling the volume scattering probability (VSP), and provides a data-driven guidance framework.
Volumetric Homogenization for Knitwear Simulation
Chun Yuan (University of Utah), Yin Yang (University of Utah)
OptimizationComputational EfficiencyDiffusion modelGaussian SplattingOptical FlowPoint CloudMeshPhysics Related
🎯 What it does: Propose a knitting garment simulation method based on volumetric homogenization, mapping yarn motion at the filament level through a volumetric mesh, learning elastic parameters for each voxel, thereby achieving fast approximation of full-scale yarn-level simulation;
VOODOO XP: Expressive One-Shot Head Reenactment for VR Telepresence
Phong Tran (Mohamed Bin Zayed University of Artificial Intelligence), Hao Li (Pinscreen)
Image TranslationRestorationGenerationPose EstimationTransformerDiffusion modelNeural Radiance FieldAuto EncoderGenerative Adversarial NetworkContrastive LearningImageVideoMesh
🎯 What it does: Propose VOODOO XP, a real-time head reenactment system based on 3D-aware neural rendering, which can quickly generate personalized 3D avatars from a single photo and achieve bidirectional real-time facial animation in VR remote communication.
WaveBlender: Practical Sound-Source Animation in Blended Domains
Kangrui Xue, Doug L. James
Physics RelatedAudio
🎯 What it does: A GPU-accelerated finite difference time domain (FDTD) acoustic solver named WaveBlender was developed for simulating animated sound sources on uniform grids, and a scheme for time-domain hybridization on continuously moving and deforming interfaces was proposed.
World-Grounded Human Motion Recovery via Gravity-View Coordinates
Zehong Shen (Zhejiang University), Xiaowei Zhou (Zhejiang University)
Pose EstimationTransformerVideo
🎯 What it does: Recover 3D human motion in world coordinates from monocular video.
XPBI: Position-Based Dynamics with Smoothing Kernels Handles Continuum Inelasticity
Chang Yu (University of California Los Angeles), Chenfanfu Jiang (University of California Los Angeles)
Diffusion modelOptical FlowPoint CloudPhysics RelatedStochastic Differential EquationOrdinary Differential Equation
🎯 What it does: Propose XPBI — a framework that combines Position-Based Dynamics (XPBD) with updated Lagrangian deformation gradient and smoothing kernel, used for simulating continuous plastic, viscoplastic, sand, snow, metal, and other physical materials, and achieving seamless coupling with traditional PBD materials (such as cloth and fluid).