These 980 ECCV 2024 papers come with a code repository. Each shows an AI one-line summary below β get the verified repo link + the full 6-part summary (innovation, method, data, results, limitations) and search every ECCV 2024 paper, free trial on arXivSub.
"Close, But Not There: Boosting Geographic Distance Sensitivity in Visual Place Recognition"
Sergio Izquierdo (University of Zaragoza), Javier Civera (University of Zaragoza)
π― What it does: Proposed a new sampling strategy called CliqueMining to construct challenging batches in visual place recognition (VPR) training, thereby enhancing geospatial distance sensitivity (GDS).
π― What it does: Proposed an efficient motion diffusion model (EMDM) that can generate high-quality, diverse human motion with very few sampling steps.
π― What it does: This paper proposes a fully asynchronous, recursive, and sparse event convolutional network called FARSE-CNN, designed to efficiently process the sparse spatiotemporal streams generated by event cameras.
"Hyperion β A fast, versatile symbolic Gaussian Belief Propagation framework for Continuous-Time SLAM"
David Hug (ETH ZΓΌrich), Margarita Chli (ETH ZΓΌrich)
CodeOptimizationRobotic IntelligenceSimultaneous Localization and MappingImage
π― What it does: Proposed the Hyperion framework, combining fast continuous-time B-/Z-spline parameterization and efficient Gaussian Bayes propagation (GBP) algorithm, achieving real-time inference for distributed continuous-time SLAM;
"Not Just Change the Labels, Learn the Features: Watermarking Deep Neural Networks with Multi-View Data"
Yuxuan Li (Harbin Institute of Technology), Yunhui Guo (University of Texas at Dallas)
CodeSafty and PrivacyAdversarial AttackImage
π― What it does: Propose a deep learning model watermarking method called MAT, which utilizes multi-view data to construct a trigger set, thereby enhancing defense against model extraction attacks.
"On Calibration of Object Detectors: Pitfalls, Evaluation and Baselines"
Selim Kuzucu (Five AI Ltd.), Puneet Dokania
CodeObject DetectionImage
π― What it does: This paper analyzes the limitations of existing calibration evaluation methods for object detectors and proposes a more reasonable joint evaluation framework, verifying the effectiveness of post-calibration methods.
π― What it does: Proposes the ProCreate method, which introduces energy guidance during the diffusion model inference process to make generated image embeddings diverge from reference images, thereby enhancing sample diversity and creativity while preventing training data duplication.
"Refine, Discriminate and Align: Stealing Encoders via Sample-Wise Prototypes and Multi-Relational Extraction"
Shuchi Wu (NJUST), Tao Xiang (CQU)
CodeSafty and PrivacyRepresentation LearningAdversarial AttackContrastive LearningImageMultimodality
π― What it does: The paper proposes the RDA method, which efficiently steals pre-trained encoders under low query costs by using sample-level prototypes and multi-relationship extraction loss.
"SPHINX: A Mixer of Weights, Visual Embeddings and Image Scales for Multi-modal Large Language Models"
Ziyi Lin (Chinese University of Hong Kong), Hongsheng Li (Chinese University of Hong Kong)
CodeTransformerLarge Language ModelSupervised Fine-TuningVision Language ModelContrastive LearningImageTextMultimodality
π― What it does: Designed and trained the SPHINX multimodal large language model, integrating weighted mixing, visual embedding mixing, and multiscale high-resolution image processing to enhance multimodal understanding and fine-grained visual reasoning capabilities.
π― What it does: Aiming at the liquid surface estimation task for dual transparent liquids (both liquid and container are transparent) in AI-guided bio-laboratories, we propose the DTLD dataset and an end-to-end TCLD method for detecting liquid contact lines and estimating liquid surface height.
Liting Lin (Pengcheng Laboratory), Haibin Ling (Stony Brook University)
CodeObject TrackingTransformerVideo
π― What it does: In the visual object tracking task, the parameter-efficient fine-tuning technique LoRA is applied to a Transformer-based single-stream tracking framework, and a decoupled position embedding and token type embedding are introduced, along with a head network that uses only MLP, constructing a series of large-scale tracking models (LoRAT) that can be trained under low memory consumption.
"UniINR: Event-guided Unified Rolling Shutter Correction, Deblurring, and Interpolation"
Yunfan Lu (Hong Kong University of Science and Technology), Hui Xiong (Hong Kong University of Science and Technology)
CodeRestorationNeural Radiance FieldVideo
π― What it does: Propose the UniINR framework, which utilizes event camera data to simultaneously perform rolling shutter (RS) correction, deblurring, and high-frame-rate global shutter (GS) frame interpolation based on a single RS blurry frame, recovering clear GS frames at arbitrary time points.
π― What it does: Proposes Online-InReaCh, an algorithm capable of detecting and locating image anomalies online in real-time in an unsupervised manner.
"X-InstructBLIP: A Framework for Aligning Image, 3D, Audio, Video to LLMs and its Emergent Cross-modal Reasoning"
Artemis Panagopoulou (University of Pennsylvania), Juan Carlos Niebles (Salesforce AI Research)
CodeRepresentation LearningTransformerLarge Language ModelVision Language ModelContrastive LearningImageVideoMultimodalityPoint CloudAudio
π― What it does: Proposed the X-InstructBLIP framework, aligning multi-modal data such as images, 3D, audio, and videos with a frozen large language model (LLM), and achieved cross-modal reasoning capabilities.
β-Brush: Controllable Large Image Synthesis with Diffusion Models in Infinite Dimensions
Minh-Quan Le (Stony Brook University), Dimitris Samaras (Stony Brook University)
CodeGenerationData SynthesisConvolutional Neural NetworkTransformerDiffusion modelContrastive LearningImageBiomedical Data
π― What it does: Proposed a controllable large-image generation model β-Brush in an infinite-dimensional function space, achieving effective injection of conditional information through cross-attention neural operators.
3D Human Pose Estimation via Non-Causal Retentive Networks
Kaili Zheng (Tsinghua University), Ji Wu (Tsinghua University)
CodePose EstimationRecurrent Neural NetworkVideo
π― What it does: Propose a 3D human pose estimation model NC-RetNet based on RetNet, which performs inference by utilizing a large number of past frames and a small number of future frames.
3D Weakly Supervised Semantic Segmentation with 2D Vision-Language Guidance
Xiaoxu Xu, Xu Wang (Shenzhen University)
CodeSegmentationConvolutional Neural NetworkVision Language ModelPoint Cloud
π― What it does: Propose 3DSS-VLG, a weakly supervised 3D semantic segmentation method that uses a 2D vision-language model as a bridge to achieve implicit alignment between point clouds and text labels.
π― What it does: Propose a pixel-aligned implicit model called 3DFG-PIFu, which utilizes 3D feature grids to globally fuse multi-view information, refines the human mesh through an iterative mechanism in the backend, and supports an SDF-form SMPL-X prior.
4D Contrastive Superflows are Dense 3D Representation Learners
Xiang Xu (Nanjing University of Aeronautics and Astronautics), Qingshan Liu (Shanghai AI Laboratory)
CodeAutonomous DrivingRepresentation LearningConvolutional Neural NetworkTransformerVision Language ModelContrastive LearningMultimodalityPoint Cloud
π― What it does: Propose the SuperFlow framework, which utilizes continuous LiDAR-camera pairs for spatiotemporal contrastive learning to achieve dense 3D representation learning.
π― What it does: This paper proposes a network called TRG, which simultaneously estimates 6DoF head pose (translation, rotation) and dense 3D facial landmarks from a single image using an explicit bidirectional interaction structure;
A Cephalometric Landmark Regression Method based on Dual-encoder for High-resolution X-ray Image
Chao Dai (Tianjin University), Minpeng Xu (Tianjin University)
CodePose EstimationConvolutional Neural NetworkTransformerImageBiomedical Data
π― What it does: Propose a dual-encoder regression framework (D-CeLR) that detects all craniofacial measurement landmarks on high-resolution X-ray images in a single pass.
π― What it does: Propose a new GAN-based model inversion attack method called IF-GMI, which decomposes the StyleGAN2 generator and recovers private images through layer-by-layer optimization of intermediate features.
π― What it does: This paper conducts a benchmark comparison of five representative image restoration networks across five classic tasks (super-resolution, denoising, deblurring, deraining, and dehazing), and designs a more task-agnostic backbone network called X-Restormer based on the comparison results.
A Comprehensive Study of Multimodal Large Language Models for Image Quality Assessment
Tianhe Wu (Tsinghua University), Lei Zhang (OPPO Research Institute)
CodeLarge Language ModelPrompt EngineeringVision Language ModelImageMultimodalityChain-of-Thought
π― What it does: This paper systematically evaluates the performance of multimodal large language models (MLLM) in image quality assessment (IQA), explores nine prompting strategies, and designs a method for selecting difficult samples;
π― What it does: This paper proposes a coronary artery anatomy synthesis framework based on latent diffusion models, achieving controllable generation and editing through morphology and skeleton conditions.
π― What it does: Designed a probability-guided sampler that utilizes the probability density function derived from SDF during the training phase of neural implicit surface rendering. It first samples in the 3D image space (image coordinates + depth), then projects the sampled points into rays, and provides additional samples near each ray. A new surface reconstruction loss is also proposed, combining constraints from near-surface regions and free space.
π― What it does: Propose a rotation-invariant texture visual Transformer (SRRM-ViT) for fine-grained classification of esophageal cancer EUS images, achieving automatic selection of important texture tokens to avoid manual segmentation.
A Secure Image Watermarking Framework with Statistical Guarantees via Adversarial Attacks on Secret Key Networks
Feiyu CHEN, Antoni Chan (City University of Hong Kong)
CodeSafty and PrivacyAdversarial AttackImage
π― What it does: Propose a zero-bit watermarking framework based on secret key networks and adversarial attacks, training a network to output a standard multivariate normal distribution. During watermark embedding, the PGD attack is utilized to make the network output a specified vector, achieving detectable and non-replicable watermarks.
A Simple Baseline for Spoken Language to Sign Language Translation with 3D Avatars
Ronglai Zuo (Hong Kong University of Science and Technology), Xin Tong (Microsoft Research Asia)
CodeGenerationPose EstimationTransformerLarge Language ModelVideoTextMesh
π― What it does: Proposes a complete baseline system for Spoken2Sign translation using 3D avatars, incorporating lexicon-video dictionary construction, 3D sign estimation based on SMPLSign-X, text-lexicon translation, sign connector, and 3D rendering modules.
A Simple Low-bit Quantization Framework for Video Snapshot Compressive Imaging
Miao Cao (Zhejiang University), Xin Yuan (Westlake University)
CodeCompressionTransformerVideo
π― What it does: Proposes the Q-SCI low-bit quantization framework for end-to-end video snapshot compressive imaging reconstruction, significantly reducing computational cost while maintaining high reconstruction quality.
π― What it does: Proposed GLASS, a unified strategy for global and local anomaly synthesis, generating anomalies at both feature and image levels through gradient ascent and noise guidance to achieve industrial anomaly detection and localization.
π― What it does: Propose a watermark-conditional generation framework WaDiff based on diffusion models, embedding user-specific watermarks into images during the generation process to achieve IP protection, detection, and user tracking.
AccDiffusion: An Accurate Method for Higher-Resolution Image Generation
Zhihang Lin (Xiamen University), Rongrong Ji (Tencent)
CodeGenerationDiffusion modelImageText
π― What it does: Propose AccDiffusion, a training-free method for generating high-resolution images and effectively eliminating patch-level repetition issues.
Accelerating Image Super-Resolution Networks with Pixel-Level Classification
Jinho Jeong (Yonsei University Samsung Advanced Institute of Technology), Seon Joo Kim (Yonsei University Samsung Advanced Institute of Technology)
CodeSuper ResolutionImage
π― What it does: Proposed the PCSR model, which dynamically allocates computational resources in single-image super-resolution by leveraging pixel-level classification.
π― What it does: Proposed a retrieval-enhanced latent diffusion model, AV-LDM, which can generate realistic action audio from silent first-person videos and separate foreground action sounds from background environmental sounds.
π― What it does: Proposes ActionVOS, a reference video object segmentation framework based on action descriptions, which utilizes human action narratives as language prompts to build a model with a classification head, and introduces an action-aware annotation module and action-guided focal loss, focusing on segmenting active objects.
π― What it does: This paper proposes an image generation framework called ActGen based on active learning, which uses misclassified validation samples to guide the augmentation of training data.
AdaCLIP: Adapting CLIP with Hybrid Learnable Prompts for Zero-Shot Anomaly Detection
Yunkang Cao (Huazhong University of Science and Technology), Giacomo Boracchi (Politecnico di Milano)
CodeAnomaly DetectionTransformerPrompt EngineeringVision Language ModelImageBiomedical Data
π― What it does: Propose AdaCLIP, a zero-shot anomaly detection method based on CLIP, which significantly improves detection performance by combining static and dynamic learnable prompts.
π― What it does: Develop AdaGlimpse, proposing an active visual exploration framework that can adaptively select arbitrary position and scale windows in a continuous action space to achieve efficient environmental perception.
AdaIFL: Adaptive Image Forgery Localization via a Dynamic and Importance-aware Transformer Network
Yuxi Li (Peking University), Yuesheng Zhu (Peking University)
CodeSegmentationTransformerMixture of ExpertsImage
π― What it does: Designed and implemented a Transformer network called AdaIFL based on dynamic routing and importance-aware mechanisms for high-precision image forgery localization.
π― What it does: This paper proposes the AdaLog quantizer, achieving adaptive log-based quantization for Vision Transformers in post-training quantization, and extends it to activation layers after Softmax and GELU through bias reparameterization.
π― What it does: Propose AdaNAT, which improves the image generation quality of non-autoregressive Transformers (NAT) by learning adaptive generation strategies.
π― What it does: This paper addresses the scenario of cross-regional fine-grained view localization lacking precise ground truth, proposing a weakly supervised adaptation method based on knowledge self-distillation.
π― What it does: Propose a two-step adaptive bounding box uncertainty prediction framework, leveraging distribution-free conformal prediction to provide coverage guarantees for bounding boxes of multi-class detectors, and propagate classification uncertainty through class prediction sets.
π― What it does: This paper proposes an adaptive compressed sensing framework called AdaSense, which dynamically selects the most informative linear measurements during the measurement process by leveraging posterior sampling from pre-trained diffusion models.
π― What it does: Propose an Adaptive Multi-Head Contrastive Learning (AMCL) that improves representation learning by using multiple sets of projection heads and learning adaptive temperatures for each pair of samples.
Adaptive Multi-task Learning for Few-shot Object Detection
Yan Ren (Nanyang Technological University), Adams Wai-Kin Kong (Nanyang Technological University)
CodeObject DetectionKnowledge DistillationMeta LearningConvolutional Neural NetworkVision Language ModelImageMultimodality
π― What it does: Propose an adaptive multi-task learning framework to address the conflicts between classification and localization tasks in few-shot object detection;
Konstantinos P Alexandridis (Huawei Noah's Ark Lab), Shan Luo (King's College London)
CodeClassificationObject DetectionImage
π― What it does: Proposed and implemented an adaptive parameterized activation function APA, which unifies various activation forms and can be used across different layers.
π― What it does: Propose an adaptive selection sampling-reconstruction framework that selects the optimal sampling mask and corresponding reconstruction network for each input image in Fourier compressed sensing.
AddMe: Zero-shot Group-photo Synthesis by Inserting People into Scenes
Dongxu Yue (International Digital Economy Academy), Yu Li (International Digital Economy Academy)
CodeGenerationData SynthesisTransformerVision Language ModelDiffusion modelImageText
π― What it does: Propose AddMe, a zero-shot group photo generation framework that can insert personalized portraits into existing photos at any position using only one reference portrait and a user-provided mask, without requiring additional fine-tuning.
AddressCLIP: Empowering Vision-Language Models for City-wide Image Address Localization
Shixiong Xu (Chinese Academy of Sciences), Jieping Ye (Alibaba Cloud)
CodeClassificationTransformerVision Language ModelContrastive LearningImageText
π― What it does: Proposed an end-to-end vision-language model called AddressCLIP for predicting the readable text address of the location where the image was taken.
π― What it does: Proposed an end-to-end ADMap framework for online high-precision HD map vectorization construction, addressing point sequence jittering and sharpness phenomena.
Adversarial Prompt Tuning for Vision-Language Models
Jiaming Zhang (Beijing Jiaotong Univisity), Jitao Sang (Fudan Univisity)
CodeAdversarial AttackPrompt EngineeringVision Language ModelDiffusion modelContrastive LearningImageText
π― What it does: This paper proposes Adversarial Prompt Tuning (AdvPT), which enhances the robustness of the image encoder in CLIP against adversarial image attacks by fine-tuning the learnable prompt vectors in the CLIP text encoder.
Adversarial Robustification via Text-to-Image Diffusion Models
Daewon Choi (Korea Advanced Institute of Science and Technology), Jinwoo Shin (Korea Advanced Institute of Science and Technology)
CodeClassificationData SynthesisAdversarial AttackConvolutional Neural NetworkSupervised Fine-TuningVision Language ModelDiffusion modelImageText
π― What it does: Propose a 'denoising + classification' randomized smoothing framework based on text-to-image diffusion models without requiring training data, to achieve provable adversarial robustness for offline pre-trained classifiers.
π― What it does: This paper proposes the Agent Attention (Agent Attention) mechanism, integrating traditional Softmax attention with linear attention to form an efficient quadruple attention (Q, A, K, V) capable of global semantic modeling.
AID-AppEAL: Automatic Image Dataset and Algorithm for Content Appeal Enhancement and Assessment Labeling
Sherry X. Chen (University of California, Santa Barbara), Pradeep Sen (University of California, Santa Barbara)
CodeGenerationData SynthesisSuper ResolutionPrompt EngineeringVision Language ModelDiffusion modelImage
π― What it does: Designed and implemented an automated pipeline for generating the Image Content Attractiveness (ICAA) dataset, named AID-AppEAL. Proposed content attractiveness estimators based on relative and absolute evaluation, and developed enhanced methods combining text inversion with Stable Diffusion. Ultimately, generated datasets with over 70K images each in the food and interior design domains.
π― What it does: This paper proposes the AlignDiff framework, which utilizes a small number of real samples to guide text-to-image diffusion models in generating synthetic training data that is both aligned with target category instances and equipped with pixel-level annotations, thereby enhancing few-shot semantic segmentation performance.
π― What it does: This paper proposes the Emotional Face Representation with Audio Perspective (EAP) method, addressing the issues of input bias and emotional intensity saturation in audio-driven emotional speaker video generation. It first neutralizes arbitrary emotional faces through Audio-to-Neutral Translation (ANT), then extracts emotional context and frame-level emotional intensity from audio via Emotional Representation from Audio (ERA), ultimately achieving natural and emotionally accurate emotional speaker video generation.
Alternate Diverse Teaching for Semi-supervised Medical Image Segmentation
Zhen Zhao (Shanghai AI Lab), Luping Zhou (University of Sydney)
CodeSegmentationConvolutional Neural NetworkBiomedical Data
π― What it does: Propose the AD-MT framework, achieving semi-supervised medical image segmentation through a single student network and two sets of alternately updated teacher networks;
π― What it does: Proposed and implemented a Transformer-based heterogeneous and low-memory similarity estimation framework, AMES, for instance-level image retrieval, achieving approximately 1KB memory usage per image through techniques such as heterogeneous vectors, binary descriptors, and knowledge distillation.
π― What it does: On a web-crawled dataset, the paper proposes a Linear Separation Alternating (LSA) strategy by combining the linear separation from unsupervised contrastive learning with existing noise detection methods, and integrates it with the PLS algorithm to form PLS-LSA, significantly improving accuracy in image classification tasks under label noise.
π― What it does: Proposes an economical 6-DoF grasp detection framework called EconomicGrasp, aiming to maintain high grasp performance while reducing training resources.
CodeSegmentationTransformerVision Language ModelImageText
π― What it does: Proposes a multi-task visual localization framework (EEVG) based on Transformer Decoder, which utilizes the Decoder for vision-language fusion and significantly improves computational efficiency and localization accuracy through parameter-agnostic visual token elimination strategies and lightweight mask heads.
An Incremental Unified Framework for Small Defect Inspection
Jiaqi Tang (Hong Kong University of Science and Technology (Guangzhou)), Fugee Tsung (Hong Kong University of Science and Technology (Guangzhou))
CodeAnomaly DetectionTransformerAuto EncoderImage
π― What it does: This paper proposes an Incremental Unified Framework (IUF) that can progressively learn micro-defect detection for various industrial objects without storing feature memory, achieving image-level and pixel-level localization.
π― What it does: Proposes AnatoMask, a reconstruction-guided self-supervised masking method that improves the pre-training process for 3D medical image segmentation.
Any Target Can be Offense: Adversarial Example Generation via Generalized Latent Infection
Youheng Sun (University of Electronic Science and Technology of China), Jingkuan Song (University of Electronic Science and Technology of China)
CodeAdversarial AttackConvolutional Neural NetworkVision Language ModelImageMultimodality
π― What it does: Propose a generator-based adversarial attack method called GAKer, which can generate adversarial examples from any target object (regardless of whether it appears in the training set) by implanting source images with potential features of the target image, achieving attacks on both known and unknown categories.
π― What it does: Propose the Any2Point framework, which leverages any modality's pre-trained large models (e.g., language, vision, audio) to achieve 3D point cloud understanding through parameter-efficient fine-tuning.
Yaxin Luo, Gen Luo (International Digital Economy Academy)
CodeRecognitionObject DetectionConvolutional Neural NetworkPrompt EngineeringVision Language ModelContrastive LearningMultimodality
π― What it does: Propose an Anchor-based Prompt Learning framework (APL), which improves the weakly supervised Referring Expression Comprehension (REC) task by dynamically generating multimodal prompts (including position, color, and category) and fusing them into Anchor features.
π― What it does: This paper proposes a strong baseline B-AVSR based on optical flow guided recursive units, optical flow corrected cross-attention units, and hyper-decoupled hyper-sampling units. By incorporating multi-scale structural and texture priors, it forms ST-AVSR to achieve video super-resolution at arbitrary scales.
ARoFace: Alignment Robustness to Improve Low-quality Face Recognition
Mohammad Saeed Ebrahimi Saadabadi (West Virginia University), Nasser Nasrabadi (West Virginia University)
CodeRecognitionImage
π― What it does: This paper proposes the ARoFace method, which enhances the robustness of low-quality face recognition (LQ FR) by incorporating adversarial data augmentation with alignment error (FAE) during training.
CodeGenerationData SynthesisDiffusion modelAuto EncoderGenerative Adversarial NetworkImagePoint CloudBiomedical Data
π― What it does: This paper proposes a latent density score based on the inherent latent space of a generative model to evaluate the quality of individual generated samples, directly measuring quality by calculating the Gaussian kernel density between the latent code of the sample and the latent codes of the training set;
π― What it does: Propose AMSNet, a self-supervised denoising framework that employs single-mask training and multi-mask inference, compatible with various existing denoisers and avoiding limitations imposed by BSN.
Asynchronous Large Language Model Enhanced Planner for Autonomous Driving
Yuan Chen (Beihang University), Si Liu (Beihang University)
CodeAutonomous DrivingTransformerLarge Language ModelSupervised Fine-Tuning
π― What it does: Propose an asynchronous LLM-enhanced closed-loop planning framework called AsyncDriver, which uses LLM to extract scene-related instruction features to guide the real-time planner in generating more accurate and controllable trajectories.
Attention Beats Linear for Fast Implicit Neural Representation Generation
Shuyi Zhang (Zhejiang University), Haishuai Wang (Zhejiang University)
CodeRestorationGenerationTransformerMultimodality
π― What it does: Proposed a local attention-based implicit neural representation (ANR) model that generates instance feature vectors using a transformation network and fuses them with coordinate features through a local attention layer, achieving efficient and fast continuous signal reconstruction.
Attention-Challenging Multiple Instance Learning for Whole Slide Image Classification
Yunlong Zhang (Zhejiang University), Lin Yang (Westlake University)
CodeClassificationTransformerBiomedical Data
π― What it does: Proposed a multi-instance learning (MIL) framework called ACMIL for whole slide image (WSI) classification, aiming to alleviate overfitting caused by excessive attention concentration.
π― What it does: Propose the AttnZero framework, which leverages evolutionary search to automatically discover efficient linear attention modules, enhancing the performance and computational efficiency of Vision Transformers.
AUFormer: Vision Transformers are Parameter-Efficient Facial Action Unit Detectors
Kaishen Yuan (Tianjin University), Jingyu Yang (Tianjin University)
CodeRecognitionTransformerImage
π― What it does: Proposed AUFormer, a parameter-efficient facial action unit (AU) detection framework based on Vision Transformer, along with the MoKE collaboration mechanism and MDWA loss.
AugUndo: Scaling Up Augmentations for Monocular Depth Completion and Estimation
Yangchao Wu (UCLA Vision Lab), Alex Wong (Yale Vision Lab)
CodeDepth EstimationImage
π― What it does: Propose the AugUndo framework, enabling unsupervised depth completion and estimation to utilize richer geometric and photometric augmentations. The method reverses geometric transformations on the output depth, allowing the loss to be computed on the original input, thereby avoiding artifacts introduced by augmentations.
π― What it does: This paper proposes an automated proxy discovery framework, Auto-DAS, for rapidly evaluating the distillation performance of candidate student models in training-free distillation-aware architecture search.
π― What it does: This paper proposes Auto-GAS, a fully training-free generative model architecture search framework, which automatically discovers and evolves 'zero-cost proxies' to rapidly evaluate candidate generators and ultimately searches for efficient GAN architectures.
CodeTransformerLarge Language ModelPrompt EngineeringVision Language ModelVideoTextMultimodalityBenchmarkChain-of-Thought
π― What it does: This paper creates the AutoEval-Video benchmark, constructing 327 instances containing videos, open-ended questions, and dedicated evaluation rules. It uses GPT-4 as an automatic evaluator to assess the performance of large vision-language models on open-ended video question-answering tasks.
π― What it does: By jointly learning the camera trajectory during exposure and 3D Gaussian point cloud parameters, a virtual sharp image is generated and averaged using a physical motion blur model, ultimately achieving motion blur removal and high-quality 3D scene reconstruction with real-time rendering support.
Bayesian Detector Combination for Object Detection with Crowdsourced Annotations
Zhi Qin Tan (University of Surrey), Yunpeng Li (University of Surrey)
CodeObject DetectionImageBiomedical Data
π― What it does: This paper proposes a framework named Bayesian Detector Combination (BDC) for training object detection models on noisy data with multiple annotators, and can automatically infer the quality of each annotator.
BenchLMM: Benchmarking Cross-style Visual Capability of Large Multimodal Models
Rizhao Cai (Nanyang Technological University), Alex Kot (Nanyang Technological University)
CodePrompt EngineeringVision Language ModelImageBiomedical DataMagnetic Resonance ImagingComputed TomographyBenchmark
π― What it does: Propose BenchLMM benchmark for quantitatively assessing the reasoning robustness of large multimodal models across three visual styles (art, sensor, and application), and systematically evaluate existing open-source models and GPT-4V.
Benchmarking Spurious Bias in Few-Shot Image Classifiers
Guangtao Zheng (University of Virginia), Aidong Zhang (University of Virginia)
CodeClassificationVision Language ModelImageMultimodalityBenchmark
π― What it does: Proposed a framework named FewSTAB, an automated benchmark framework for systematically evaluating the robustness of few-shot image classifiers against spurious bias (dirty attributes).
π― What it does: This paper proposes a fully self-supervised video object segmentation (VOS) method BA, which leverages the attention maps from DINO pre-trained ViT, learns spatiotemporal correspondence through a single spatiotemporal Transformer block, and directly obtains object segmentation masks in videos using hierarchical clustering.
π― What it does: This paper proposes the semantic multi-object tracking (SMOT) task, which involves tracking target trajectories while simultaneously providing instance description, instance interaction recognition, and video-level description for each trajectory; based on this, a large-scale semantic multi-object tracking benchmark called BenSMOT is constructed, offering four types of annotations: trajectories, instance descriptions, interaction labels, and video descriptions; finally, an end-to-end trained baseline model named SMOTer is presented.
π― What it does: Propose a multi-scale patch multi-label classifier (MPMC) as a plug-in module to enhance contextual information in semi-supervised semantic segmentation models, and reduce noise generated by the teacher model through confidence learning to adaptively adjust pseudo-label weights.
π― What it does: This paper proposes a new comprehensive affordance (ComA) representation that can simultaneously model contact, relative pose, and spatial relationships in human-object interaction.
π― What it does: This paper proposes a two-stage vehicle maneuver prediction framework. It first learns maneuver representations using a balanced intersection viewpoint dataset, then learns geometric diversity using a self-vehicle perspective dataset. Finally, it calculates maneuver probabilities during inference using Gaussian kernel density estimation.
BI-MDRG: Bridging Image History in Multimodal Dialogue Response Generation
Hee Suk Yoon (Korea Advanced Institute of Science and Technology), Chang D. Yoo (Korea Advanced Institute of Science and Technology)
CodeGenerationTransformerVision Language ModelDiffusion modelImageTextMultimodality
π― What it does: In the multi-modal dialogue response generation task, this paper proposes the BI-MDRG model, which can simultaneously generate text and image responses, and enhances the image-rootedness of text responses and the consistency of image responses through bridging image history.
π― What it does: Propose a bidirectional test-time adaptation framework called Bi-TTA that performs single-instance adaptation using only user facial video during inference, addressing the generalization problem of remote photoplethysmography (rPPG) models in unseen domains.
Binomial Self-compensation for Motion Error in Dynamic 3D Scanning
Geyou Zhang (University of Electronic Science and Technology of China), Kai Liu (Sichuan University)
CodeDepth EstimationImageVideoPoint Cloud
π― What it does: To address motion errors in dynamic 3D scanning, this paper proposes a binomial self-compensation (BSC) algorithm based on four-step phase-shifting phase measurement (PSP), achieving cyclic projection under a set of high-frequency phase-shifting patterns and real-time point cloud reconstruction.