These 892 CVPR 2024 papers come with a code repository. Each shows an AI one-line summary below β get the verified repo link + the full 6-part summary (innovation, method, data, results, limitations) and search every CVPR 2024 paper, free trial on arXivSub.
3D Building Reconstruction from Monocular Remote Sensing Images with Multi-level Supervisions
Weijia Li (Sun Yat-Sen University), Conghui He (Shanghai AI Laboratory)
π― What it does: A multi-level supervised monocular remote sensing image building 3D reconstruction network (MLS-BRN) is proposed, which can be trained on samples with different levels of annotations to generate complete 3D models of building footprints, roofs, offsets, and heights from a single frame image.
π― What it does: This paper proposes a new loss function PRDL, which achieves geometric guidance for 3D face reconstruction by converting facial part segmentation into 2D point sets and aligning them with 3DMM reprojected point sets.
3D LiDAR Mapping in Dynamic Environments using a 4D Implicit Neural Representation
Xingguang Zhong (Center for Robotics, University of Bonn), Jens Behley (Center for Robotics, University of Bonn)
CodeAutonomous DrivingRobotic IntelligenceSimultaneous Localization and MappingPoint Cloud
π― What it does: Using a 4D implicit neural network for spatiotemporal TSDF modeling of continuous LiDAR point clouds, a complete static map is obtained and dynamic objects are automatically removed.
π― What it does: A motion recognition pipeline based on 3D point cloud sequences, called 3DinAction, is proposed, which focuses on constructing spatiotemporal features using local point cloud sets (t-patches) that evolve over time.
A Call to Reflect on Evaluation Practices for Age Estimation: Comparative Analysis of the State-of-the-Art and a Unified Benchmark
Jakub PaplhΓ‘m (Czech Technical University in Prague), Vojt?ch Franc
CodeClassificationRecognitionConvolutional Neural NetworkTransformerVision Language ModelImageBenchmark
π― What it does: Systematically evaluated and compared facial age estimation methods, proposed a unified evaluation protocol, and published data partitioning and code.
CodeDomain AdaptationOptimizationTransformerVision Language ModelContrastive LearningImage
π― What it does: The CLAP (Class-Adaptive Linear Probe) method is proposed, which uses zero-shot prototypes as priors to achieve efficient few-shot transfer of large-scale vision-language models without a validation set.
π― What it does: A point cloud upsampling method based on a conditional denoising diffusion probabilistic model (PUDM) is proposed, which directly generates high-quality dense point clouds from sparse point clouds as conditions.
π― What it does: This study systematically evaluates the robustness of 20 mainstream post-hoc OOD detection methods in the presence of label noise during classifier training, exploring how label noise affects the distinguishability between ID samples and OOD samples.
A Picture is Worth More Than 77 Text Tokens: Evaluating CLIP-Style Models on Dense Captions
Jack Urbanek (Meta), Adriana Romero-Soriano (Fair)
CodeSegmentationRetrievalTransformerLarge Language ModelSupervised Fine-TuningVision Language ModelContrastive LearningImageTextMultimodality
π― What it does: A dense annotated image dataset DCI has been constructed and made public, providing approximately 1000-word multi-level textual descriptions for each image, and based on this, a sub-image-caption matching task has been proposed to evaluate the fine-grained understanding ability of VLM.
A Semi-supervised Nighttime Dehazing Baseline with Spatial-Frequency Aware and Realistic Brightness Constraint
Xiaofeng Cong (Southeast University), Hao Shen (Hefei University of Technology)
CodeRestorationTransformerImage
π― What it does: A semi-supervised nighttime image dehazing baseline SFSNiD is proposed, utilizing a spatial-frequency information interaction module and local window brightness constraints to address issues of haze, halos, and noise caused by multiple light sources and low illumination.
A Subspace-Constrained Tyler's Estimator and its Applications to Structure from Motion
Feng Yu (University of Minnesota), Gilad Lerman (University of Minnesota)
CodeAnomaly DetectionOptimizationImage
π― What it does: A subspace-constrained Tyler estimator (STE) is proposed for efficiently recovering low-dimensional subspaces in the presence of a large number of outliers.
π― What it does: The researchers propose to predict complete human motion in a 3D scene using only the sparse tracking signals generated by a head-mounted display and dual hand controllers.
CodeDomain AdaptationSafty and PrivacyTransformerPrompt EngineeringImage
π― What it does: This paper proposes the A2XP method, which utilizes expert prompts and attention mechanisms to achieve domain generalization while maintaining the structural and parameter privacy of the target network.
π― What it does: A large-scale multimodal accident video dataset, MM-AU, is proposed, and based on this dataset, the AdVersa-SD framework is designed for self-supervised reasoning in accident video understanding and generation.
π― What it does: To address the low sampling efficiency of diffusion models, the authors propose a framework that accelerates sampling by optimizing the sampling time step.
Accurate Training Data for Occupancy Map Prediction in Automated Driving Using Evidence Theory
Jonas KΓ€lble (Bosch Center for Artificial Intelligence), Eddy Ilg (Saarland University)
CodeAutonomous DrivingPoint Cloud
π― What it does: A method for generating LiDAR occupancy grids based on evidence theory is proposed, which can explicitly handle occlusion, unobserved areas, and measurement errors, generating more accurate occupancy maps and uncertainty estimates for training image-driven occupancy prediction models.
π― What it does: Adversarial Consistency Training (ACT) is proposed, which incorporates a discriminator into the consistency model training to directly minimize the Jensen-Shannon distance between the generated distribution and the target distribution, thereby achieving efficient generation of a one-step diffusion model.
Action Scene Graphs for Long-Form Understanding of Egocentric Videos
Ivan Rodin (University of Catania), Giovanni Maria Farinella (University of Catania)
CodeObject DetectionRepresentation LearningGraph Neural NetworkTransformerLarge Language ModelVideoGraph
π― What it does: A new long-term first-person video understanding representation called Egocentric Action Scene Graphs (EASG) is proposed, and this graph representation has been manually annotated on the Ego4D dataset.
π― What it does: A proactive target detection framework KAD is proposed, utilizing ternary prior knowledge (semantic interaction, fine-grained vision, spatial prior);
Jihwan Bang (Korea Advanced Institute of Science and Technology), Jae-Gil Lee (Michigan State University)
CodeClassificationData-Centric LearningTransformerPrompt EngineeringVision Language ModelImage
π― What it does: Active prompt learning for pre-trained vision-language models is conducted to reduce labeling costs and enhance adaptability to new tasks.
AdaBM: On-the-Fly Adaptive Bit Mapping for Image Super-Resolution
Cheeun Hong (Seoul National University), Kyoung Mu Lee (Seoul National University)
CodeRestorationSuper ResolutionImage
π― What it does: For the task of image super-resolution, this paper proposes a quantization framework called AdaBM that can adaptively configure bit width in real-time during inference, completing bit mapping configuration in seconds and avoiding the massive training required by traditional QAT.
π― What it does: This paper proposes a method for cross-domain few-shot segmentation (CD-FSS) that does not require a pre-trained segmentation network. Instead, it directly achieves segmentation by inserting 1Γ1 adapters into each bottleneck layer of an ImageNet pre-trained ResNet-50 backbone during testing, and utilizes view consistency contrastive learning and class prototype contrastive loss for task adaptation of the adapters.
π― What it does: Systematically evaluated and improved adapters on visual Transformers, proposing a more efficient Adapter+ for parameterized transfer learning.
Adapting Short-Term Transformers for Action Detection in Untrimmed Videos
Min Yang (Nanjing University), Limin Wang (Nanjing University)
CodeObject DetectionTransformerAuto EncoderVideo
π― What it does: Proposes the ViT-TAD framework, using a pre-trained pure ViT as a unified long video Transformer for end-to-end temporal action detection.
CodeAnomaly DetectionTransformerVision Language ModelContrastive LearningImageBiomedical DataMagnetic Resonance ImagingComputed Tomography
π― What it does: Adapting the pre-trained CLIP visual-language model, a multi-layer visual feature adaptation and comparison framework is proposed to achieve zero/few-shot anomaly detection in medical images.
π― What it does: This paper proposes an Adaptive Bidirectional Displacement (ABD) framework to enhance the consistency learning effect in semi-supervised medical image segmentation.
π― What it does: This paper proposes an adaptive fusion network for monocular and multi-view depth estimation, named AFNet, aimed at enhancing the robustness and accuracy of depth estimation in autonomous driving scenarios.
π― What it does: Proposes an adaptive multimodal cross-entropy loss and a primary modality resolution estimator to improve the distribution supervision and result estimation of stereo matching networks.
π― What it does: The AdaSlot framework is proposed, which adaptively determines the number of slots during object discovery and segmentation, avoiding the limitations of traditional Slot Attention that requires a preset number of slots.
π― What it does: This paper studies the Softassign algorithm in graph matching and proposes Adaptive Softassign and Hadamard-Equipped Sinkhorn formulas to automatically adjust parameters and improve stability and efficiency.
AdaRevD: Adaptive Patch Exiting Reversible Decoder Pushes the Limit of Image Deblurring
Xintian Mao (Shanghai Key Laboratory of Multidimensional Information Processing East China Normal University), Yan Wang (Shanghai Key Laboratory of Multidimensional Information Processing East China Normal University)
CodeRestorationAuto EncoderImage
π― What it does: For the image deblurring task, the authors propose AdaRevD, a reversible sub-decoder with an adaptive patch exit mechanism that enhances decoding capability while being memory-friendly.
π― What it does: Automatically construct an Automatic Data Factory (ADF) using NeRF to generate high-quality optical flow annotations from RGB images captured by a single camera, and use them for unsupervised training of optical flow networks.
π― What it does: Utilizing Neural Radiance Fields (NeRF) to elevate 2D semantic and instance segmentation labels from a drone's perspective to 3D space, achieving semantic and building-level instance segmentation at an urban scale.
π― What it does: A training-free detection method called AEROBLADE is proposed, which utilizes the reconstruction error of a latent diffusion model (LDM) autoencoder to distinguish between real images and generated images.
Affine Equivariant Networks Based on Differential Invariants
Yikang Li (Peking University), Zhouchen Lin (Peking University)
CodeImageOrdinary Differential Equation
π― What it does: An affine equivariant network based on differential invariants and symmetric partial differential equations (InvarPDEs-Net and InvarLayer) is constructed, which does not require discretization or sampling of the affine group.
Aligning and Prompting Everything All at Once for Universal Visual Perception
Yunhang Shen (Tencent), Rongrong Ji (Xiamen University)
CodeObject DetectionSegmentationTransformerPrompt EngineeringVision Language ModelImageTextMultimodality
π― What it does: A unified visual perception model APE has been constructed, capable of detecting, segmenting, and localizing all targets, segmented areas, and natural language descriptions in an image at once, using an instance-level sentence-object matching framework.
π― What it does: This paper studies the black-box knowledge distillation (B2KD) problem from cloud to edge, proposing a two-step process: first, deprivatization is achieved through a generator, and then distillation is performed using mapping emulation.
AllSpark: Reborn Labeled Features from Unlabeled in Transformer for Semi-Supervised Semantic Segmentation
Haonan Wang (Hong Kong University of Science and Technology), Xiaomeng Li (Hong Kong University of Science and Technology)
CodeSegmentationTransformerImage
π― What it does: The AllSpark module is proposed to reshape labeled features by mapping unlabeled features to labeled features in semi-supervised semantic segmentation, alleviating the issue of label data dominance.
Alpha Invariance: On Inverse Scaling Between Distance and Volume Density in Neural Radiance Fields
Joshua Ahn (University of Chicago), Greg Shakhnarovich (TTI-Chicago)
CodeNeural Radiance FieldPoint Cloud
π― What it does: This paper addresses the inverse relationship between volume density and distance caused by the uncertainty of scene size scale in NeRF, proposing the concept of alpha invariance and providing corresponding mathematical analysis.
π― What it does: By fusing visual foundation models such as CLIP, DINOv2, and SAM through multi-teacher distillation, a RADIO visual model is trained that can handle arbitrary resolutions and is compatible with multiple tasks.
π― What it does: The AMU-Tuning method is proposed to enhance the performance of CLIP in few-shot learning by learning effective logit biases, and it provides a unified analysis of existing methods from the perspective of logit biases.
π― What it does: This paper proposes an interactive navigation method based on effect-driven affordance maps (ADIN), achieving long-term goal navigation in complex dynamic environments.
π― What it does: This paper proposes a high-efficiency knowledge transfer scheme called FedKTL, which uses a server-side pre-trained generator to generate image-vector pairs related to client tasks, aiding the transfer of global knowledge to client models in heterogeneous federated learning.
Analyzing and Improving the Training Dynamics of Diffusion Models
Tero Karras (NVIDIA), Samuli Laine (NVIDIA)
CodeGenerationData SynthesisDiffusion modelImage
π― What it does: Improved the training dynamics of the diffusion model, redesigned network layers to maintain the expected values of activations, weights, and update magnitudes, and proposed a post-EMA method to accurately adjust the model average;
π― What it does: A framework for open-set supervised anomaly detection based on simulated heterogeneous anomaly distributions (AHL) is proposed, which achieves unified modeling of seen and unseen anomalies by generating diverse anomaly distributions and performing collaborative differentiable learning in a simulated open-set environment.
π― What it does: This paper proposes an image dataset API based on anime production by analyzing the anime production process, and designs a prediction-guided compression module and pseudo-realistic line enhancement technology to address compression distortion and hand-drawn line blur.
AssistGUI: Task-Oriented PC Graphical User Interface Automation
Difei Gao, Mike Zheng Shou
CodeTransformerLarge Language ModelAgentic AITextMultimodalityBenchmark
π― What it does: Proposed the ASSISTGUI benchmark, covering 100 Windows PC GUI automation tasks, and introduced the AutoPC multi-agent collaboration framework to accomplish these tasks;
π― What it does: Using Stable Diffusion combined with ControlNet to generate realistic underwater images and corresponding accurate depths, and based on this dataset to train and enhance the performance of monocular depth estimation models.
π― What it does: A weakly supervised learning-based chemical structure recognition model, AtomLenz, has been developed, capable of achieving atomic-level localization and constructing molecular graphs using only SMILES-labeled image data.
π― What it does: DisenDiff is designed to learn and generate various personalized concepts in a text-to-image model using an attention calibration mechanism under the condition of having only a single image, supporting both independent and combined generation of concepts.
Attention-Propagation Network for Egocentric Heatmap to 3D Pose Lifting
Taeho Kang (Seoul National University), Youngki Lee (Seoul National University)
CodePose EstimationTransformerImage
π― What it does: A heatmap-based 3D pose upsampling method called EgoTAP is proposed for high-precision 3D human pose estimation in a binocular first-person camera scenario.
Tengda Han (University of Oxford), Andrew Zisserman (University of Oxford)
CodeGenerationTransformerLarge Language ModelVision Language ModelVideoMultimodalityAudio
π― What it does: This paper proposes two pixel-level movie audio description (AD) datasets (CMD-AD and HowTo-AD), and based on this, designs the Movie-BLIP2 and Movie-Llama2 models based on Q-former to achieve full pixel-level AD generation.
BA-SAM: Scalable Bias-Mode Attention Mask for Segment Anything Model
Yiran Song (Shanghai Jiao Tong University), Lizhuang Ma (La Trobe University)
CodeSegmentationTransformerImage
π― What it does: This paper addresses the performance degradation of the Segment Anything Model at different image resolutions by proposing a scalable Bias Attention Mask (BA-SAM), achieving length extrapolation without altering the structure.
π― What it does: This paper introduces triggers during the prompt learning phase of CLIP, constructing a trigger-aware context generator to perform backdoor attacks on the model;
Balancing Act: Distribution-Guided Debiasing in Diffusion Models
Rishubh Parihar (Indian Institute of Science), R. Venkatesh Babu (Indian Institute of Science)
CodeGenerationData SynthesisDiffusion modelImage
π― What it does: This paper proposes a method for achieving fair generation without retraining the pre-trained diffusion model, by using Distribution Guidance to make the attribute distribution of generated images approximate the user-specified reference distribution.
π― What it does: A method for 3D reconstruction of indoor scenes is proposed using a series of depth images, capable of simultaneously recovering visible surfaces and occluded (invisible) surfaces, achieving the generation of a complete 3D mesh.
Benchmarking Segmentation Models with Mask-Preserved Attribute Editing
Zijin Yin (Beijing University of Posts and Telecommunications), Jun Guo (Beijing University of Posts and Telecommunications)
CodeSegmentationGenerationTransformerLarge Language ModelDiffusion modelImageTextBenchmark
π― What it does: A mask-preserved attribute editing pipeline based on diffusion models has been constructed, which accurately edits local and global attributes of images while keeping the original segmentation labels unchanged, generating editable images that can be used to evaluate the robustness of semantic segmentation.
π― What it does: This paper addresses the vulnerability of temporal information in videos by constructing two sets of time corruption robustness benchmarks based on THUMOS14 and ActivityNet v1.3. It systematically evaluates the performance of seven mainstream temporal action detection models under varying degrees of time corruption and proposes a robust training strategy through FrameDrop data augmentation and Temporal-Robust Consistency (TRC) loss.
π― What it does: A super-resolution framework SR4IR is proposed, which utilizes task-driven perceptual loss (TDP) and cross-quality patch mixing (CQMix) to enhance the performance of low-resolution images in high-level vision tasks such as semantic segmentation, object detection, and image classification.
Beyond Text: Frozen Large Language Models in Visual Signal Comprehension
Lei Zhu (Peking University), Yanye Lu (Peking University)
CodeImage TranslationRestorationGenerationTransformerLarge Language ModelVision Language ModelGenerative Adversarial NetworkContrastive LearningImageTextMultimodality
π― What it does: Proposes the Vision-to-Language Tokenizer (V2L), which maps images to the vocabulary of LLM to generate discrete tokens, allowing the frozen LLM to directly perform visual understanding and image denoising.
Beyond Textual Constraints: Learning Novel Diffusion Conditions with Fewer Examples
Yuyang Yu, Shengfeng He
CodeGenerationData SynthesisDiffusion modelImage
π― What it does: The study investigates how to enable a text-to-image diffusion model based on Stable Diffusion to learn new control conditions under the constraint of only 100 samples, achieving high-quality and structurally consistent generation.
π― What it does: A dual-stream network BMCNet is proposed, which processes positive and negative events separately and achieves complementarity through a bidirectional information exchange module, thereby completing the super-resolution of event streams.
π― What it does: This paper proposes a deep completion network based on bilateral propagation, BP-Net, which utilizes sparse depth measurements and synchronized color images for early-stage depth propagation, thereby avoiding the sensitivity of later convolution operations to sparse data.
π― What it does: This paper proposes BlockGCN, an improved graph convolutional network designed to better capture spatial-temporal features in skeletal action sequences.
π― What it does: This paper proposes an unsupervised blur-to-blur translation framework called Blur2Blur, which first converts the unknown blur images captured by the target camera into a known blur domain, and then uses a pre-trained deblurring network to recover clear images.
π― What it does: This paper proposes a Rademacher complexity analysis based on the Fisher-Rao norm, introducing a new logit variable Ξ_ce, and further designs a stage-wise logit-oriented adversarial training framework (LOAT) to enhance the robustness of the model while maintaining or improving standard accuracy.
Boosting Continual Learning of Vision-Language Models via Mixture-of-Experts Adapters
Jiazuo Yu (Dalian University of Technology), You He (Tsinghua University)
CodeTransformerMixture of ExpertsVision Language ModelAuto EncoderImageMultimodality
π― What it does: A parameter-efficient continual learning framework is proposed, utilizing Mixture-of-Experts (MoE) adapters for dynamic expansion on a frozen CLIP model, and automatically distinguishing seen and unseen data through a Distribution-Discriminative Auto-Selector (DDAS), balancing memory retention and zero-shot transfer capability.
Boosting Image Quality Assessment through Efficient Transformer Adaptation with Local Feature Enhancement
Kangmin Xu (Wuhan University), Weisi Lin (Nanyang Technological University)
CodeConvolutional Neural NetworkTransformerImage
π― What it does: The LoDa method is proposed, which combines multi-scale distortion features extracted from a pre-trained ViT and CNN for no-reference image quality assessment, achieving parameter-efficient adaptation.
π― What it does: This paper proposes a general enhancement framework that improves the reconstruction, compression, repair, and interpolation capabilities of implicit video representation (INR) models by introducing a temporal-aware conditional decoder (TAT) and a sine activation NeRV-like module (SNeRV).
Boosting Object Detection with Zero-Shot Day-Night Domain Adaptation
Zhipeng Du (King's College London), Jiankang Deng (Imperial College London)
CodeObject DetectionDomain AdaptationImage
π― What it does: A zero-shot day-night domain adaptation framework DAI-Net is designed, utilizing reflectance representation learning to enhance object detection performance in low-light scenarios.
π― What it does: Proposes Supernet Shifting in NAS, combining architecture search and supernet fine-tuning to enhance global and local ranking consistency.
π― What it does: A deep network based on multi-order differential spike timing fusion (DSFT) and multi-granularity alignment is proposed to reconstruct high-quality images from high-frequency, noise, and quantization error-prone pulse sequences captured by spike cameras.
π― What it does: Pre-train a radar perception model through self-supervised learning to learn features from a large amount of unlabeled radar data, improving the performance of radar single target detection.
π― What it does: By inheriting the transformer parameters of large pre-trained visual backbone models (such as AugReg and CLIP) and only training a small number of dedicated focus transformers and intermediate layer weights, a rapid bootstrapping of SparseFormer is achieved.
BrainWash: A Poisoning Attack to Forget in Continual Learning
Ali Abbasi (Vanderbilt University), Soheil Kolouri (Vanderbilt University)
CodeOptimizationAdversarial AttackImage
π― What it does: A backdoor data poisoning attack named BrainWash is proposed, which allows continual learning models to forget previously learned knowledge after learning new tasks.
π― What it does: A pre-trained text-to-video diffusion model is optimized through score distillation to automatically animate a single subject's vector sketch using text prompts, outputting short videos that retain vector form.
Bridging Remote Sensors with Multisensor Geospatial Foundation Models
Boran Han (Amazon Web Services), Markus Reichstein (Max-Planck-Institute for Biogeochemistry)
CodeClassificationRestorationSegmentationTransformerMixture of ExpertsImage
π― What it does: A multi-sensor geospatial foundation model named msGFM is proposed, capable of jointly learning image features from four types of sensors: RGB, Sentinel-2, SAR, and DSM, and achieving unified modeling of paired and unpaired data through cross-sensor pre-training.
π― What it does: A unified video understanding framework UVCOM is proposed, jointly addressing the tasks of moment retrieval (MR) and highlight detection (HD).
π― What it does: This paper proposes a weakly supervised 3D instance segmentation framework called BSNet, which utilizes SAFormer to generate pseudo-labels and train the instance segmentation network.
BT-Adapter: Video Conversation is Feasible Without Video Instruction Tuning
Ruyang Liu (Peking University), Ge Li (Peking University)
CodeRecognitionRetrievalComputational EfficiencyTransformerVision Language ModelContrastive LearningVideoMultimodality
π― What it does: This paper proposes BT-Adapter, a method for efficiently transferring from image-language models to video-dialogue models by inserting a branch spatio-temporal attention module while keeping the CLIP visual encoder frozen.
π― What it does: Proposes the CSC (Coherent Semantic Consistency) framework, which utilizes cross-scene semantic consistency for unsupervised pre-training of 3D representations.
π― What it does: A reference image super-resolution method based on a conditional diffusion model, Ref-Diff, is proposed, utilizing land cover change priors to guide denoising, enhancing the content authenticity and texture transfer of large-scale remote sensing images.
C3: High-Performance and Low-Complexity Neural Compression from a Single Image or Video
Hyunjik Kim (Google DeepMind), Emilien Dupont (Google DeepMind)
CodeCompressionImageVideo
π― What it does: This paper presents C3, a low-complexity neural compression method for single images/videos, trained frame by frame to achieve high compression performance.
π― What it does: Proposed the Cam4DOcc benchmark, established a unified 4D occupancy prediction data format and standard evaluation protocol, and implemented four baselines (static world, point cloud projection, BEV instance replication, end-to-end OCCNet).
π― What it does: This paper proposes CAMEL, which utilizes learnable motion prompts and causal motion to enhance attention, achieving motion coherence and visual consistency in text-driven video editing.
π― What it does: For 48 ImageNet models trained using ResNet-50, we systematically measured their texture/shape bias, spectral bias, and critical band characteristics, and correlated these biases with various generalization benchmarks (ID, robustness, concept transfer, adversarial robustness).
Can I Trust Your Answer? Visually Grounded Video Question Answering
Junbin Xiao (National University of Singapore), Tat-Seng Chua (National University of Singapore)
CodeRecognitionOptimizationTransformerVision Language ModelContrastive LearningVideoMultimodalityBenchmark
π― What it does: A weakly supervised visual-oriented video question answering (NExT-GQA) dataset was constructed, and existing visual-language models (VLMs) were evaluated on it. The NG+ method was proposed, which combines differentiable Gaussian masking optimization with cross-modal self-supervision to enhance the visual credibility of answers.
CAPE: CAM as a Probabilistic Ensemble for Enhanced DNN Interpretation
Townim Faisal Chowdhury (Australian Institute for Machine Learning, University of Adelaide), Zhibin Liao (SA Pathology)
CodeExplainability and InterpretabilityKnowledge DistillationConvolutional Neural NetworkTransformerImage
π― What it does: Proposes the CAPE method, which reinterprets CAM as a probabilistic ensemble for visualizing DNN attention areas and providing comparable explanations.
π― What it does: Proposes the CapHuman framework, which generates high-fidelity portraits under various poses, expressions, and lighting changes using a single reference face image.