These 31 SIGGRAPH Asia 2023 papers come with a code repository. Each shows an AI one-line summary below — get the verified repo link + the full 6-part summary (innovation, method, data, results, limitations) and search every SIGGRAPH Asia 2023 paper, free trial on arXivSub.
A Locality-based Neural Solver for Optical Motion Capture
Xiaoyu Pan (Zhejiang University), Xiaogang Jin (Zhejiang University)
🎯 What it does: Propose a method for cleaning and motion solving of optical motion capture data based on locality and heterogeneous graph neural networks.
🎯 What it does: Propose and study the CamP camera preprocessing method, which aims to improve the joint optimization of camera parameters in NeRF, thereby enhancing the quality of 3D scene reconstruction.
🎯 What it does: This paper proposes an automated method for cable layout design in tight-fitting clothing, aiming to minimize cable resistance during the wearer's movements.
🎯 What it does: Designed and implemented a numerically robust, parameter-independent 3D Constrained Delaunay Tetrahedralization (CDT) algorithm that can successfully construct CDT on all valid PLCs.
Daohan Lu (Carnegie Mellon University), Jun-Yan Zhu (Carnegie Mellon University)
CodeGenerationData SynthesisRetrievalRepresentation LearningTransformerVision Language ModelDiffusion modelScore-based ModelFlow-based ModelRectified FlowContrastive LearningImageTextMultimodalityBenchmarkAudio
🎯 What it does: This paper proposes a content-driven deep generative model retrieval task and constructs the Generative Model Zoo benchmark. It introduces a retrieval framework based on probabilistic and contrastive learning, which can retrieve text, images, sketches, and models themselves.
🎯 What it does: Constructed a full-body holographic avatar driven by sparse RGB-D input, capable of realistically reproducing the dynamics and details of loose clothing.
🎯 What it does: Proposes a novel differentiable memory table primitive called Differentiable Indirection (DIn), which replaces MLP computations in traditional neural networks to achieve efficient representation and compression for various graphics tasks such as geometry, texture, shading, and radiance fields.
EXIM: A Hybrid Explicit-Implicit Representation for Text-Guided 3D Shape Generation
Zhengzhe Liu, Chi-Wing Fu
CodeGenerationData SynthesisTransformerVision Language ModelDiffusion modelNeural Radiance FieldAuto EncoderImageTextMesh
🎯 What it does: This paper proposes a hybrid explicit-implicit representation method called EXIM, which generates high-fidelity 3D shapes guided by text and supports local editing.
🎯 What it does: By constructing a two-dimensional navigable exploration space, the high-dimensional latent space of a pre-trained 3D shape generation model is projected onto a two-dimensional plane, enabling smooth and natural interpolation among a given set of landmark shapes, and transferring the generated variations to high-quality meshes.
🎯 What it does: This paper proposes a domain-specific language called GarmentCode based on object-oriented programming, for generating programmable and interactive parameterized sewing patterns, and implements a garment configurator;
Interactive Story Visualization with Multiple Characters
Yuan Gong (Tsinghua University), Yujiu Yang (Tsinghua University)
CodeGenerationData SynthesisTransformerPrompt EngineeringVision Language ModelDiffusion modelContrastive LearningImageVideoText
🎯 What it does: A complete interactive story visualization system is proposed, which can generate multi-character images based on textual stories and produce animations.
🎯 What it does: Propose a GeGnn method based on graph neural networks to learn geodesic distance embeddings between two points on a surface, enabling constant-time distance queries after a single forward pass.
Multiple-bounce Smith Microfacet BRDFs using the Invariance Principle
Yuang Cui (Anhui Science and Technology University), Beibei Wang (Nankai University)
CodeDiffusion modelImageBenchmarkPhysics Related
🎯 What it does: Propose a multi-bounce Smith microsurface BRDF model derived using the principle of invariance, and provide a concise paragraph function.
🎯 What it does: Proposes a muscle-driven character simulation and control framework based on the Hill muscle model and the 3CC-r fatigue model, and designs a muscle space PD control and a VAE-based generator controller called MuscleVAE.
🎯 What it does: Propose a two-stage framework based on deep learning—Neural Gradient Learning (NGL) and Gradient Vector Optimization (GVO)—for directly estimating consistently oriented normals from noisy, dense, and diverse point clouds.
🎯 What it does: Proposed a neuro-morphological material network (NMN) for efficient inverse design of nonlinear morphological materials, directly optimizing structural parameters to match target stress-strain curves, directional stiffness, or Poisson's ratio curves.
🎯 What it does: Propose a random masking Poisson reconstruction framework based on neural networks, representing both mean and covariance with multi-layer perceptrons, supporting efficient iterative updates, next best view planning, and seamless integration into the scanning process.
🎯 What it does: This paper proposes a global energy function that treats each SDF sample as a sphere and introduces tangent point constraints, achieving reconstruction from SDF to explicit triangle mesh via geometric flow gradient descent;
🎯 What it does: Propose a multi-view learning framework that utilizes a 3D conditional convolutional network to reconstruct the 3D poses of closely interacting crowds from 2D heatmaps captured by multiple cameras, effectively handling occlusions and identity confusion without relying on real image-3D paired data.
ReparamCAD: Zero-shot CAD Re-Parameterization for Interactive Manipulation
Milin Kodnongbua, Adriana Schulz
CodeOptimizationComputational EfficiencyData-Centric LearningTransformerLarge Language ModelVision Language ModelTextMesh
🎯 What it does: Developed ReparamCAD zero-shot pipeline, which infers the meaningful variation space of shapes using pre-trained large language models and image models, and reparameterizes CAD programs based on this, enabling easy exploration along meaningful design axes.
🎯 What it does: Propose to decompose single-image novel view synthesis into two steps: pixel re-coloring and pixel re-localization. After training a specialized re-coloring network, combine it with existing re-localization methods to generate new views with perspective-dependent highlights that move naturally.
🎯 What it does: Propose a method called DoubleCoverUDF, which utilizes the double covering structure of r-offset and coverage mapping learning to robustly extract zero-level sets from the learned unsigned distance field (UDF), supporting both open and closed 3D surfaces.
CodeOptimizationDiffusion modelScore-based ModelNeural Radiance FieldOptical FlowImagePoint CloudPhysics Related
🎯 What it does: This paper proposes an end-to-end differentiable NLOS inverse rendering pipeline, capable of self-calibrating imaging parameters and reconstructing hidden scene surface points, normals, and albedo;
🎯 What it does: A example-driven deep generative network called ShaDDR is constructed, which can refine rough voxel models into high-resolution, textured 3D shapes in less than 1 second.
🎯 What it does: Propose a single-image view synthesis method called SinMPI, which generates high-quality, freely rotatable 3D-consistent novel views by constructing an extended multi-plane image (MPI).
🎯 What it does: Propose a single-image document de-warping method based on a dual-headed fully convolutional network, utilizing 3D meshes and 2D inverse mappings to jointly predict and achieve document de-warping.
🎯 What it does: Proposes a point cloud completion and neural rendering framework based on Visual Error Tomography (VET), which can automatically clean and complete sparse or missing geometric structures.
🎯 What it does: This paper systematically evaluates various automated evaluation metrics for the task of generating skeletal motions from natural language, and proposes a novel multimodal BERT-style evaluation model called MoBERT.