These 25 SIGGRAPH Asia 2024 papers come with a code repository. Each shows an AI one-line summary below — get the verified repo link + the full 6-part summary (innovation, method, data, results, limitations) and search every SIGGRAPH Asia 2024 paper, free trial on arXivSub.
Anim-Director: A Large Multimodal Model Powered Agent for Controllable Animation Video Generation
Yunxin Li (Harbin Institute of Technology), Min Zhang (Harbin Institute of Technology)
CodeGenerationData SynthesisTransformerLarge Language ModelAgentic AIPrompt EngineeringVision Language ModelDiffusion modelScore-based ModelImageVideoTextMultimodalityRetrieval-Augmented Generation
🎯 What it does: Developed Anim-Director, an automated animation generation agent based on a large-scale multimodal model, capable of automatically generating coherent and plot-complete long animation videos from short narratives.
🎯 What it does: Proposes the CBIL framework, which can directly learn the collective motion of fish schools from real videos without requiring three-dimensional motion trajectories.
🎯 What it does: Proposes a content-aware tile generation method based on exterior boundary inpainting, which can generate diverse and seamlessly mosaicked tile sets from text prompts or example images, and supports multiple tile types (self-mosaic tiles, random self-mosaic tiles, Escher tiles, Wang Ou tiles, Dual Wang tiles).
🎯 What it does: Propose an encoder-based text inversion method that encodes the rhythm and genre information of dance videos into pluggable pseudo-words, thereby enhancing pre-trained text-music models to achieve dance-synchronized music generation.
🎯 What it does: This paper proposes a rendering algorithm called DARTS, which efficiently generates time-resolved paths in uniform scattering media, achieving precise control over the flight time of light.
🎯 What it does: Propose a Jacobian network based on local rough input to achieve high-quality, detail-preserving shape deformation, and apply it to registration refinement, unsupervised correspondence, and interactive editing.
🎯 What it does: Propose the GaussianObject framework, which can achieve high-quality 3D object reconstruction and rendering using only four perspective images.
🎯 What it does: This paper proposes a complete method that can achieve high-quality animatable eyelid shape reconstruction and animation using only RGB video from a smartphone.
🎯 What it does: This paper proposes iSeg, an interactive 3D mesh segmentation method based on user clicks, which can generate fine-grained and user-intention-aligned segmentation results under one or multiple positive and negative clicks.
🎯 What it does: This study proposes a multi-sample multi-importance sampling (MIS) sample allocation framework called MARS, which utilizes fixed-point iteration combined with Russian roulette and splitting to adaptively allocate spatially varying numbers of samples to each sampling strategy through local estimation;
🎯 What it does: Propose a dynamic super-resolution method based on Mesh-Graph-Net and Hyper-Net, generating high-resolution detailed wrinkle details using low-resolution clothing simulation and body motion input.
🎯 What it does: Propose a method that utilizes graph neural networks to learn the point cloud Laplacian operator (NeLo), directly learning edge weights and mass matrices on the K-nearest neighbor graph, enabling geometric processing without the need to construct a triangular mesh.
🎯 What it does: Propose a product importance sampling method based on neural normalized flows, which improves sampling efficiency in rendering by approximating the product distribution of environment illumination and BRDF through cascading a small neural normalized flow (head warp) with a precomputed emitter tail warp.
🎯 What it does: Propose Occupancy-Based Dual Contouring (ODC), which improves 1D binary search and 2D point search, utilizing auxiliary points in 3D grids to construct QEF, specifically designed for occupancy functions;
🎯 What it does: This paper proposes a Prompt Aligned Personalization (PALP) method for text-to-image diffusion models, which can achieve subject personalization using only a small number of individual images, while maintaining high alignment with specified text prompts.
🎯 What it does: Pano2Room automatically generates complete 3D Gaussian Splatting scenes from a single indoor panoramic image and supports high-quality panoramic view synthesis.
Polar Interpolants for Thin-Shell Microstructure Homogenization
Antoine Chan-Lock (Universidad Rey Juan Carlos), M. Otaduy
CodeMeshTabularPhysics Related
🎯 What it does: Aiming at the macroscopic behavior of thin-shell microstructures, this paper proposes a conservative energy-based homogenized material model, and employs high-order RBF interpolation in polar coordinates to achieve precise fitting of the principal strain domain.
CodeOptimizationDiffusion modelNeural Radiance FieldAuto EncoderContrastive LearningOptical FlowImageVideoMultimodalityPhysics Related
🎯 What it does: This paper proposes a polarization BSSRDF acquisition method for dynamic faces, capable of simultaneously capturing facial geometry, spatially varying polarization appearance parameters, and biophysically based skin parameters.
ProcessPainter: Learning to draw from sequence data
Yiren Song, Mike Zheng Shou
CodeGenerationData SynthesisTransformerSupervised Fine-TuningVision Language ModelDiffusion modelImageVideoTextSequential
🎯 What it does: Propose ProcessPainter, a text-to-video model that can generate a step-by-step painting process based on textual prompts, and introduce the Artwork Replication Network to achieve painting process control, image segmentation, and completion of the artwork with arbitrary frame input.
🎯 What it does: Proposed a system for automatically generating human-like sound imitations, capturing the essence of the sound and converting it into an acoustically interpretable description for humans;
🎯 What it does: A composite surface reconstruction algorithm based on a rotational system is proposed, which constructs a tree from point clouds and gradually inserts edges to obtain a high-quality triangular mesh with controllable topology.