These 37 SIGGRAPH 2024 papers come with a code repository. Each shows an AI one-line summary below β get the verified repo link + the full 6-part summary (innovation, method, data, results, limitations) and search every SIGGRAPH 2024 paper, free trial on arXivSub.
3D Gaussian Splatting with Deferred Reflection
Keyang Ye (Zhejiang University), Kun Zhou (Zhejiang University)
π― What it does: This paper proposes an efficient dynamic scene novel view synthesis method based on 4D rotation operators (Rotor) β 4D-Rotor Gaussian Splatting.
π― What it does: Proposed the first large-scale multi-modal string performance dataset (String Performance Dataset, SPD), and designed an audio-guided unlabeled motion capture framework based on this dataset, which can accurately recover finger positions on the strings through audio (pitch) information, achieving fine-grained hand motion capture;
π― What it does: A bilateral grid-based radiance field processing method is proposed. During the NeRF training phase, a differentiable 3D bilateral grid is used to separate the camera post-processing differences across views. Subsequently, a low-rank 4D bilateral grid is utilized to migrate 2D edits from a single view to a 3D scene, achieving a complete process without floating points and with enhanced visualization.
π― What it does: Propose using time-varying correlated noise (especially blue noise) in deterministic diffusion models to improve image generation quality.
π― What it does: By unifying the geometry and topology of B-rep into a structured latent tree and using a Transformer-based diffusion model for layer-by-layer denoising, high-quality B-rep CAD models are directly generated.
π― What it does: Proposed and implemented ColorVideoVDP, a full-reference video and image quality assessment metric based on a visual perception model, which can simultaneously consider the visible differences in chrominance, temporal, and spatial frequency domains.
Conditional Mixture Path Guiding for Differentiable Rendering
Zhimin Fan, Jie Guo
CodeData SynthesisOptimizationComputational EfficiencyMixture of ExpertsDiffusion modelScore-based ModelNeural Radiance FieldImage
π― What it does: Proposed and implemented conditional hybrid path guiding, targeting ray path sampling in differentiable rendering, significantly reducing the variance of Monte Carlo estimation through real-time computation of optimal weights.
π― What it does: This paper proposes an algorithm based on boundary energy maximization, which gradually restores the global consistent normals of point clouds under a random method.
π― What it does: Propose an unsupervised 3D shape co-segmentation method called DAE-Net, which reconstructs each shape by learning deformable part templates and assembling them in needed subsets, achieving fine-grained and semantically consistent segmentation;
π― What it does: This paper proposes a differentiable geodesic distance framework on triangular meshes, utilizing geodesic length as the objective function to achieve intrinsic optimization, including problems such as elastic curve networks, elastic triangular membranes, bidirectional coupling, and geodesic Voronoi diagrams.
π― What it does: Implicitly model cell-level mechanical systems using differentiable Voronoi diagrams (power diagrams), where a single Voronoi site represents each cell, enabling continuous topological changes and cell deformations;
π― What it does: This study proposes a fully automatic, text-prompt-based generation method that can produce non-square 2D texture grids that perfectly tile the plane and contain only foreground objects;
π― What it does: Propose a hand-object interaction controller (HOIC) based on deep reinforcement learning, which reconstructs hand-object interaction movements in real-time using a single RGBD camera.
π― What it does: This paper proposes a continuous collision-free trajectory generation framework based on the SDF of an implicit swept volume (Swept Volume), which can generate continuous trajectories satisfying dynamic constraints for objects of arbitrary shapes.
π― What it does: Propose an incremental GAN inversion framework that quickly generates high-fidelity, animatable 3D head avatars using multi-frame input.
LGTM: Local-to-Global Text-Driven Human Motion Diffusion Model
Haowen Sun (Shenzhen University), Ruizhen Hu (Shenzhen University)
CodeGenerationPose EstimationTransformerLarge Language ModelPrompt EngineeringDiffusion modelTextMultimodalityRetrieval-Augmented Generation
π― What it does: Propose LGTM, a local-global text-driven human motion generation framework that first decomposes textual descriptions into part-level semantics and then gradually generates local motions using diffusion models, ultimately fusing them into a complete motion through a full-body optimizer.
π― What it does: This paper proposes N-BVH, a ray query compression structure that integrates neural networks with traditional BVH, used to efficiently replace traditional triangle mesh queries in ray tracing.
π― What it does: Construct and train an N-dimensional Gaussian Mixture Model (GMM) to explicitly approximate functions with high-dimensional parameter spaces, achieving fast training and efficient rendering.
π― What it does: Proposed a Jacobi-preconditioned nonlinear conjugate gradient (PNCG) method for real-time simulation of internal point hyperelastic models.
π― What it does: Propose a supersampling neural radiance field (Rip-NeRF) based on Ripmap encoding and Platonic Solid Projection, achieving high-fidelity view synthesis.
Zhimin Fan (Nanjing University), Ling-Qi Yan (University of California Santa Barbara)
CodeDiffusion modelScore-based ModelNeural Radiance FieldContrastive LearningGaussian SplattingOptical FlowImageVideoPhysics Related
π― What it does: A method is proposed that transforms the solving of mirror chains into a polynomial system and solves all feasible paths through resultants elimination, achieving deterministic light path search without Newton iterations, and applying it to the rendering of high-frequency effects such as glints and caustics.
π― What it does: Proposed a spatial-temporally consistent 4D Gaussian expansion model called ST-4DGS for high-quality and efficient dynamic scene rendering
π― What it does: Proposed a hierarchical 3D Gaussian splatting renderer to achieve view-consistent real-time rendering and eliminate popping artifacts when the camera rotates.
π― What it does: Propose a roughness-aware tensorized SDF representation method called TensoSDF, which can simultaneously reconstruct geometry and material from multi-view images.
π― What it does: Propose TexSliders, which achieves texture editing based on diffusion models by constructing sliders in the CLIP image embedding space;
CodeGenerationData SynthesisTransformerLarge Language ModelDiffusion modelText
π― What it does: A machine learning method based on text description is proposed, which uses conditional diffusion models to generate text-guided agent distribution fields and velocity fields, and combines them with local navigation algorithms to control multiple agents, thereby synthesizing diverse dynamic crowd animation scenes; at the same time, a large language model is used to standardize general scripts into structured sentences to improve training stability and scalability.
π― What it does: Studied the problem of unstructured and unlabeled optical motion capture (UUO mocap), proposing to utilize synchronized monocular video to generate body priors and combine them with unlabeled markers to achieve simultaneous reconstruction of full/local body pose and shape.
Ultra Inertial Poser: Scalable Motion Capture and Tracking from Sparse Inertial Sensors and Ultra-Wideband Ranging
Rayan Armani (ETH ZΓΌrich), Christian Holz (ETH ZΓΌrich)
CodePose EstimationRecurrent Neural NetworkGraph Neural NetworkContrastive LearningSimultaneous Localization and MappingOptical FlowPoint CloudGraphTime Series
π― What it does: Propose the Ultra Inertial Poser method, which combines sparse inertial sensors with UWB ranging to achieve real-time full-body pose estimation.
π― What it does: By acquiring reflectance-transmittance images of fabrics and combining them with a new two-layer BSDF model, the inversion and reconstruction of fabric parameters are achieved.
π― What it does: This paper proposes X-Portrait, a zero-shot Stable Diffusion-based portrait animation framework that can generate high-fidelity, expressive, and temporally coherent video from a single static portrait;
π― What it does: Propose the ZeroGrads framework, which online self-supervisedly learns a locally differentiable surrogate (neural network) and utilizes its gradient for parameter optimization in black-box graphics pipelines where gradients cannot be directly obtained.