These 743 ICCV 2023 papers come with a code repository. Each shows an AI one-line summary below β get the verified repo link + the full 6-part summary (innovation, method, data, results, limitations) and search every ICCV 2023 paper, free trial on arXivSub.
3D Human Mesh Recovery with Sequentially Global Rotation Estimation
Dongkai Wang (Peking University), Shiliang Zhang (Peking University)
π― What it does: A Sequentially Global Rotation Estimation (SGRE) method is proposed for 3D human mesh recovery from monocular RGB images, directly predicting the global rotation matrices of each joint.
π― What it does: A self-supervised 3D Implicit Transporter is proposed, capable of discovering spatiotemporally consistent key points from continuous point cloud sequences, and utilizing these key points for goal-driven 3D object manipulation.
π― What it does: A 3D Semantic Subspace Traverser is proposed, which can achieve 3D shape generation and semantic editing under implicit function representation.
3DMiner: Discovering Shapes from Large-Scale Unannotated Image Datasets
Ta-Ying Cheng (University of Oxford), Niki Trigoni (University of Oxford)
CodeObject DetectionSegmentationRepresentation LearningTransformerContrastive LearningSimultaneous Localization and MappingImage
π― What it does: On a large-scale unlabeled image dataset, without the need for 3D annotations, camera information, or key points, the 3DMiner pipeline automatically mines and reconstructs 3D shapes;
π― What it does: A 3D Point Pose Encoding (3DPPE) is proposed for Transformer-based multi-camera 3D object detection, utilizing depth estimation to project pixels into 3D space to obtain accurate point position information and embed features.
π― What it does: A 4D myocardial reconstruction method based on implicit functions is proposed, which can predict the complete myocardial shape and its temporal evolution from sparse CMR slice point clouds.
π― What it does: This paper constructs a realistic Chinese-English text image super-resolution benchmark dataset, Real-CE, and proposes a dual supervision learning method based on text edges to enhance the reconstruction quality of Chinese characters.
π― What it does: This paper proposes a complete design framework for constructing forward diffusion processes that satisfy a given target distribution, and introduces a new diffusion model called Phase Space Langevin Diffusion (PSLD) within this framework.
π― What it does: An end-to-end unified system named FUS3D has been designed and implemented, capable of achieving real-time 3D object detection and multi-object tracking on edge devices using only depth maps.
π― What it does: A conditional discrete diffusion model Pix2SeqD is proposed for unified processing of panoramic segmentation tasks for images and videos.
π― What it does: A new video action recognition benchmark BEAR is proposed, collecting action datasets from 18 sources across 5 domains (anomaly, gesture, daily, sports, teaching), and systematically evaluating 6 mainstream video models under various settings such as standard fine-tuning, few-shot fine-tuning, unsupervised domain adaptation, and zero-shot learning.
π― What it does: This paper explores why adding classification loss can improve performance in deep regression tasks, and clarifies that its main role is to compensate for the impact of uneven sample distribution.
π― What it does: A general Learning-Accumulation-Integration (LAE) framework is proposed, which utilizes any parameter-efficient tuning (PET) modules (such as Adapter, LoRA, Prefix) for continual learning on pre-trained models, and achieves memoryless continual learning through the integration of online and offline PET modules.
π― What it does: The A2Q method is proposed, which trains quantized neural networks to use low-precision accumulators during inference without overflow.
AccFlow: Backward Accumulation for Long-Range Optical Flow
Guangyang Wu (University of Electronic Science and Technology of China), Wenyi Wang (University of Electronic Science and Technology of China)
CodeOptical FlowVideo
π― What it does: This paper proposes a recursive framework named AccFlow for estimating long-range optical flow by backward accumulating local optical flow.
π― What it does: Generate video subtitles directly in the compressed domain (I-frames, motion vectors, and residuals) in an end-to-end manner, eliminating the need to decode video frames or offline extract multimodal features.
π― What it does: This paper addresses the issue of model calibration failure on out-of-distribution (OOD) data by proposing a method based on Adaptive Calibrator Ensemble (ACE);
π― What it does: Using raw RAW images instead of traditional sRGB images for shadow detection, an Adaptive Illumination Mapping (AIM) module is proposed to generate sRGB images with different intensity ranges, and a feedback mechanism is employed to guide AIM in producing images with more shadow contrast, thereby improving shadow detection accuracy.
π― What it does: Conditional editing of the latent space of StyleGAN is achieved through gradual nonlinear transformations for controllable modification of facial attributes.
π― What it does: An Adaptive Rotational Convolution (ARC) module is proposed, which can dynamically rotate convolution kernels according to the orientation of targets in different images, and enhance the feature representation capability of the backbone by combining multiple rotated kernels through a conditional computation mechanism.
π― What it does: This paper focuses on the self-distillation framework in self-supervised representation learning, investigating the feasibility of using nearest neighbors (NN) to guide positive sample pairs, and proposes an adaptive similarity guidance method (AdaSim).
π― What it does: This paper proposes an adaptive spiral convolution layer suitable for 3D meshes, which can dynamically adjust the length and weights of the spiral path according to the mesh structure, thereby achieving efficient feature learning with a global receptive field and local refinement.
Irena Gao (Stanford University), Marco Tulio Ribeiro (Microsoft Research)
CodeClassificationObject DetectionRetrievalTransformerLarge Language ModelImageTextRetrieval-Augmented Generation
π― What it does: This paper presents AdaVision, a visual model testing process for human-computer interaction that helps users identify and fix semantically consistent failure modes of models.
π― What it does: Design and implement the ControlNet architecture, allowing users to finely control the spatial layout of generated images through additional conditional images (such as edge maps, pose skeletons, depth maps, etc.) without compromising the quality of the original large-scale pre-trained diffusion models (like Stable Diffusion).
π― What it does: This paper studies example exploitation in adversarial training, first proposing a robust confidence metric that divides samples into accuracy-critical (A-C) and robustness-critical (R-C) categories, and analyzes their different contributions to model accuracy and robustness. Subsequently, a new example handling method is designed, which reduces the robustness learning intensity for A-C samples and enhances the robustness learning intensity for R-C samples (through adaptive Ξ» or step size). This method is applied to both multi-step (TRADES, TEAT, etc.) and single-step (FastAT, GradAlign, etc.) adversarial training. Experiments show that it can simultaneously alleviate the accuracy-robustness trade-off, robustness overfitting, and catastrophic overfitting issues.
π― What it does: This paper proposes a Generalized Representation Segmentation (GRES) task for multi-image collections and constructs the corresponding GRD dataset and benchmark model GRSer.
π― What it does: This paper proposes a new single-source domain generalization method called Adversarial Bayesian Augmentation (ABA), which enhances model performance in unknown domains by introducing Bayesian neural networks in convolutional layers and combining them with adversarial training to generate diverse image augmentations.
π― What it does: For RGB-D image depth completion, a novel Attention Guided Gated Convolution Network (AGG-Net) is proposed to achieve fine recovery of missing depth.
π― What it does: A single-stage Transformer-based framework for person-object interaction detection, AGER, is proposed, which generates complete instance tokens internally in the encoder using text-guided dynamic clustering, thereby extracting complete instance-level features without the need for an additional object detector or instance decoder.
π― What it does: The AIDE driving perception dataset is proposed, and various baseline frameworks are constructed on this dataset, covering multi-view, multi-modal, and multi-task driving monitoring tasks.
π― What it does: A unified framework for visual task output space, AiT, is proposed, which discretizes the high-dimensional outputs of different visual tasks into tokens and uses autoregressive Transformers for prediction, supporting various tasks such as depth estimation, instance segmentation, and keypoint detection.
π― What it does: The ROBOSAC framework is proposed, utilizing the Random Sample Consensus (RANSAC) idea to achieve robustness against collaborative perception adversarial attacks;
π― What it does: This paper proposes AdaEA, which utilizes adaptive gradient modulation and a difference reduction filter to optimize multi-model ensemble attacks, enhancing the transferability of adversarial attacks between CNNs and ViTs.
π― What it does: This paper proposes a self-supervised learning backdoor attack method named CTRL, which can implant a backdoor by contaminating a small amount of training data.
π― What it does: This paper proposes a self-supervised learning framework called Alice, which utilizes cross-volume sampling to obtain positive samples of the same anatomical structure and performs semantic alignment within the same volume, thereby explicitly modeling anatomically invariant features.
π― What it does: An unsupervised anomaly detection method called GNL is proposed for scenarios with distribution shifts, and benchmarks are established on four commonly used datasets.
Anti-DreamBooth: Protecting Users from Personalized Text-to-image Synthesis
Thanh Van Le (VinAI Research), Anh Tran (VinAI Research)
CodeGenerationSafty and PrivacyAdversarial AttackDiffusion modelImage
π― What it does: In response to the potential misuse of the DreamBooth personalized text-image model, this paper proposes Anti-DreamBooth, which adds nearly invisible perturbations before users upload images, resulting in poor quality or distorted personalized images generated by any DreamBooth fine-tuning model trained on these images, thereby protecting user privacy.
Aperture Diffraction for Compact Snapshot Spectral Imaging
Tao Lv (Nanjing University), Xun Cao (Nanjing University)
CodeRestorationTransformerImage
π― What it does: A compact snapshot spectral imaging system ADIS, composed solely of an ultra-thin orthogonal aperture mask and a standard imaging lens, has been designed, and a spectral reconstruction algorithm CSST based on this system has been proposed.
AREA: Adaptive Reweighting via Effective Area for Long-Tailed Classification
Xiaohua Chen (Institute of Information Engineering), Weiping Wang (Institute of Information Engineering)
CodeClassificationImage
π― What it does: The AREA (Adaptive Reweighting via Effective Area) method is proposed, which improves the class imbalance problem in long-tail classification tasks by calculating the effective area for each category to achieve adaptive reweighting.
π― What it does: The ARNOLD benchmark has been constructed, which includes 8 language-driven robotic tasks aimed at continuous states, providing real 3D scenes, multi-camera observations, 10k expert demonstrations, and various generalization data splits; a systematic evaluation of language and state understanding has also been implemented.
Atmospheric Transmission and Thermal Inertia Induced Blind Road Segmentation with a Large-Scale Dataset TBRSD
Junzhang Chen (Beihang University), Xiangzhi Bai (Beihang University)
CodeObject DetectionSegmentationTransformerImage
π― What it does: This paper proposes a blind road semantic segmentation network based on thermal infrared images, and for the first time introduces two physical models, atmospheric transmission and thermal inertia effects, into the network, improving the accuracy of blind road detection in low-light environments.
AttT2M: Text-Driven Human Motion Generation with Multi-Perspective Attention Mechanism
Chongyang Zhong (Institute of Computing Technology Chinese Academy of Sciences), Shihong Xia (Institute of Computing Technology Chinese Academy of Sciences)
π― What it does: This paper proposes AttT2M, a two-stage multi-view attention mechanism for text-driven human action generation. The first stage utilizes a spatiotemporal encoder with body part attention and VQ-VAE to learn a discrete latent space; the second stage captures the cross-modal correspondence between text and actions through global-local attention (sentence-level conditional self-attention + word-level cross-attention) and generates action sequences using a generative Transformer.
π― What it does: This paper addresses the foreground drift problem in incremental object detection and proposes solutions including Augmented Box Replay (ABR) and Attentive RoI Distillation.
Augmenting and Aligning Snippets for Few-Shot Video Domain Adaptation
Yuecong Xu (Institute for Infocomm Research), Xiaoli Li (Institute for Infocomm Research)
CodeRecognitionDomain AdaptationTransformerVideo
π― What it does: A new method called SSA2lign is proposed to address the problem of video domain adaptation with only a small number of target video samples.
π― What it does: Proposes Auto-KD, which automates the design of knowledge distillation for the first time, constructing a unified tree search space and using Monte Carlo Tree Search for efficient searching;
π― What it does: This paper proposes an automatic channel pruning method called APIB based on the Information Bottleneck (IB) principle, which uses HSIC Lasso to solve the IB approximation and automatically determines the pruning ratio for each layer.
AutoReP: Automatic ReLU Replacement for Fast Private Network Inference
Hongwu Peng (University of Connecticut), Caiwen Ding (University of Connecticut)
CodeSafty and PrivacyComputational EfficiencyConvolutional Neural NetworkImage
π― What it does: We propose AutoReP, an automatic ReLU replacement framework for private inference that significantly reduces ReLU operations while maintaining high accuracy.
π― What it does: This paper proposes BANSAC, a variant of RANSAC that utilizes dynamic Bayesian networks to dynamically update the inlier probabilities of each matching point and performs probability-weighted sampling; it also provides a probability-based termination criterion.
CodeClassificationOptimizationKnowledge DistillationConvolutional Neural NetworkImageBiomedical Data
π― What it does: By combining Bayesian optimization with self-distillation, the BOSS framework is proposed to utilize the parameters and performance knowledge of previously trained models in each BO iteration to enhance the final model performance.
Bayesian Prompt Learning for Image-Language Model Generalization
Mohammad Mahdi Derakhshani (University of Amsterdam), Brais Martinez (Samsung AI Cambridge)
CodeClassificationDomain AdaptationTransformerPrompt EngineeringVision Language ModelImageText
π― What it does: By viewing prompt learning as Bayesian variational inference, a Bayesian prompt learning method is proposed to regularize the prompt space and enhance the generalization ability of unseen prompts.
π― What it does: This paper proposes a Non-Adversarial Backdoor (NAB) framework that defends against backdoor attacks by injecting non-adversarial backdoors into a small number of suspected samples to suppress original backdoor attacks.
π― What it does: Manually annotated the CIFAR-10/100 test set based on background color to generate the CIFAR-B dataset, studied subgroup performance differences, and proposed the FlowAug semantic data augmentation method to reduce subgroup differences.
BEVPlace: Learning LiDAR-based Place Recognition using Bird's Eye View Images
Lun Luo (Zhejiang University), Hui-Liang Shen (Zhejiang University)
CodeRecognitionPose EstimationAutonomous DrivingConvolutional Neural NetworkSimultaneous Localization and MappingPoint CloudBenchmark
π― What it does: The study uses Bird's Eye View (BEV) as a representation of LiDAR point clouds and proposes a rotation-invariant network called BEVPlace based on group convolution and NetVLAD for place recognition and location estimation of point clouds.
π― What it does: A Dual Multi-Modal Interaction (DMMI) network is proposed to address the segmentation difficulties when natural language descriptions point to multiple targets or no targets.
π― What it does: Proposes and evaluates a multidimensional skin color measurement method based on the CIELAB color space (L* represents skin color depth, h* represents skin color hue), and uses this method to detect skin color bias in image datasets and computer vision models.
π― What it does: A cross-domain object detection method named BiADT is proposed, which separates domain-invariant and domain-specific features for each token in the encoder and decoder of the Transformer, and achieves alignment of domain-invariant features and distinction of domain-specific features through bidirectional alignment.
Bird's-Eye-View Scene Graph for Vision-Language Navigation
Rui Liu (Zhejiang University), Yi Yang (Zhejiang University)
CodeObject DetectionRobotic IntelligenceTransformerVision Language ModelMultimodalityPoint Cloud
π― What it does: By constructing a scene graph based on Bird's Eye View (BEV), this paper achieves three-dimensional perception of indoor environments and utilizes the scene graph for visual-language navigation decision-making.
Black Box Few-Shot Adaptation for Vision-Language Models
Yassine Ouali (Samsung AI), Georgios Tzimiropoulos (Queen Mary University of London)
CodeDomain AdaptationPrompt EngineeringVision Language ModelImage
π― What it does: This paper proposes a novel black-box few-shot visual-language model adaptation method called LFA, which achieves cross-domain adaptation using only pre-computed image and text features without accessing model weights.
Bold but Cautious: Unlocking the Potential of Personalized Federated Learning through Cautiously Aggressive Collaboration
Xinghao Wu (Beihang University), Shaojie Tang (University of Texas at Dallas)
CodeFederated LearningImage
π― What it does: This paper proposes a personalized federated learning framework called FedCAC, which is based on parameter sensitivity and client data distribution similarity, enabling better collaboration and personalization among clients in non-IID scenarios.
BoMD: Bag of Multi-label Descriptors for Noisy Chest X-ray Classification
Yuanhong Chen (Australian Institute for Machine Learning, University of Adelaide), Gustavo Carneiro (Centre for Vision, Speech and Signal Processing, University of Surrey)
CodeClassificationConvolutional Neural NetworkLarge Language ModelImageBiomedical DataBenchmark
π― What it does: A two-stage method for handling noisy multi-label chest X-ray image classification is proposedβBag of Multi-Label Descriptors (BoMD).
π― What it does: This paper proposes the Gradient Relevance Attack (GRA), which enhances the transferability of adversarial samples through a gradient relevance framework and decay indicators.
π― What it does: This paper proposes a weakly supervised audio-video event parsing framework based on the Poisson-Binomial distribution, which improves the detection rate of positive sample segments through EM iteration, significantly enhancing event localization performance.
π― What it does: This paper studies a feature enhancement method without additional parameters or computational overheadβPartial Channel Shifting (PCS)βto improve the performance of single-image super-resolution models.
Borrowing Knowledge From Pre-trained Language Model: A New Data-efficient Visual Learning Paradigm
Wenxuan Ma (Beijing Institute of Technology), Gao Huang (Tsinghua University)
CodeClassificationDomain AdaptationData-Centric LearningTransformerLarge Language ModelContrastive LearningImageText
π― What it does: This paper proposes a method called BorLan that utilizes knowledge from pre-trained language models to enhance the learning efficiency of visual models in data-scarce tasks.
π― What it does: By training a detector on the outputs of original localization networks (such as DINO and CLIP visual encoders) and using the bounding boxes generated by the detector to refine the original network, improvements in localization are achieved under unsupervised and weakly supervised conditions.
π― What it does: A training-free method named BoxDiff is proposed for synthesizing images based on spatial conditions provided by users (such as boxes or doodles).
π― What it does: This paper proposes a framework for generating universal adversarial perturbations for videos based on image models and image data (BTC-UAP), utilizing image classification models for adversarial optimization on each frame, and attacking video models by minimizing the feature similarity between adjacent frames.
Bridging Cross-task Protocol Inconsistency for Distillation in Dense Object Detection
Longrong Yang (Zhejiang University), Xi Li (Zhejiang University)
CodeObject DetectionKnowledge DistillationImage
π― What it does: A knowledge distillation framework for dense object detection is proposed to address the issue of inconsistent cross-task protocols, with a binary classification distillation loss and an IoU localization distillation loss designed separately.
Bridging Vision and Language Encoders: Parameter-Efficient Tuning for Referring Image Segmentation
Zunnan Xu (Sun Yat-sen University), Guanbin Li (Sun Yat-sen University)
CodeObject DetectionSegmentationTransformerVision Language ModelContrastive LearningImageMultimodality
π― What it does: This study proposes a parameter-efficient tuning framework based on a dual-stream vision-language modelβBridgerβfor reference image segmentation tasks.
π― What it does: ClipFaceShop proposes a clipart-based facial photo editing method that can accurately transfer abstract clipart facial attributes (such as hairstyle, expression, beard, etc.) to real photos while maintaining the identity of the portrait.
π― What it does: The BT 2 method is proposed, which adds necessary dimensions through learnable basis transformations in backward-compatible training, maintaining the performance of the new model while being compatible with the old model.
CAD-Estate: Large-scale CAD Model Annotation in RGB Videos
Kevis-Kokitsi Maninis (Google Research), Vittorio Ferrari (Google Research)
CodeObject DetectionObject TrackingPose EstimationRetrievalOptimizationSimultaneous Localization and MappingVideoMesh
π― What it does: This paper proposes a semi-automated workflow that combines RGB videos with a CAD model database to generate globally consistent 9-DoF CAD model pose annotations for multi-object scenes in videos, constructing the CAD-Estate dataset with a scale of 20k videos, 101k instances, and 12k independent CAD models.
π― What it does: This paper proposes Cascade-DETR, a model that achieves high-quality cross-domain object detection through cascaded attention and IoU prediction recalibration based on DETR.
CASSPR: Cross Attention Single Scan Place Recognition
Yan Xia (Technical University of Munich), Daniel Cremers (Technical University of Munich)
CodeRecognitionRetrievalTransformerSimultaneous Localization and MappingPoint Cloud
π― What it does: This paper proposes CASSPR, a cross-attention transformer that combines point clouds and sparse voxels for scene recognition from single-frame LiDAR point clouds.
π― What it does: A semi-supervised segmentation method for medical imaging based on causal graphs, CauSSL, is proposed, which enhances model performance through algorithm independence.
π― What it does: The Continual Bias Adaptor (CBA) module is proposed, which dynamically compensates for catastrophic distribution drift through augmented classifiers and dual-layer optimization in online continual learning, thereby alleviating forgetting.
π― What it does: A cross-domain few-shot video action recognition method is proposed, which achieves feature balance between the source domain and the target domain by combining self-supervised pre-training and curriculum learning.
π― What it does: A decoupled point cloud registration framework based on object centers is proposed, estimating translation through center regression and predicting rotation using center-aligned point clouds, achieving 6D object pose estimation.
π― What it does: Two decision boundary black-box attack methods, CGBA and CGBA-H, are proposed. They efficiently generate adversarial samples with a low query budget by searching for decision boundary points along a semicircular trajectory on a two-dimensional constrained plane, and provide a better initial boundary point selection scheme.
π― What it does: The first large-scale multimodal dataset, Chaotic World, has been constructed to analyze human behavior in chaotic events, and a unified multitask model, IntelliCare, has been proposed.
ChartReader: A Unified Framework for Chart Derendering and Comprehension without Heuristic Rules
Zhi-Qi Cheng (Carnegie Mellon University), Alexander G. Hauptmann (Carnegie Mellon University)
CodeTransformerVision Language ModelTabular
π― What it does: A unified framework called ChartReader is proposed, integrating three major tasks: chart rendering (Chart-to-Table) and chart understanding (ChartQA, Chart-to-Text), forming an end-to-end unstructured learning process.
Chasing Clouds: Differentiable Volumetric Rasterisation of Point Clouds as a Highly Efficient and Accurate Loss for Large-Scale Deformable 3D Registration
Mattias P. Heinrich (University of Luebeck), Lasse Hansen (EchoScout GmbH)
π― What it does: Unsupervised and self-supervised registration of highly deformable 3D point clouds is performed, proposing a differentiable voxel rasterization loss that addresses the gradient sparsity and non-differentiability issues of traditional Chamfer/EMD in high-resolution and complex geometries.
π― What it does: A two-stage framework is designed: first, a CLIP pre-trained model aligned with printed character images and Chinese character IDS is used to learn the normative representation of Chinese characters, and then this representation is applied to a text recognition model to achieve zero-shot recognition of Chinese text.
CIRI: Curricular Inactivation for Residue-aware One-shot Video Inpainting
Weiying Zheng (South China University of Technology), Shengfeng He (Singapore Management University)
CodeRestorationSegmentationVideo
π― What it does: This study investigates a one-shot video inpainting framework that converts traditional video inpainting models to single annotation (only the first frame mask), addressing the problem of filling in missing areas in dynamic scenes.
CiT: Curation in Training for Effective Vision-Language Data
Hu Xu (Meta AI), Christoph Feichtenhofer (Meta AI)
CodeComputational EfficiencyRepresentation LearningData-Centric LearningTransformerVision Language ModelContrastive LearningImageTextMultimodality
π― What it does: This paper proposes an algorithm for dynamic data selection during the training processβCiT (Curation in Training). By utilizing a pre-trained text encoder, it matches the metadata of the target task with a vast number of image-text pairs for similarity, thereby automatically filtering out more relevant data in the training loop, significantly improving data utilization efficiency.
Class-Aware Patch Embedding Adaptation for Few-Shot Image Classification
Fusheng Hao (Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences), Jun Cheng (Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences)
π― What it does: In few-shot image classification, image representation and similarity computation are improved by making the patch embeddings of ViT category-related and defining a dense similarity matrix.
π― What it does: A class-incremental continual learning instance segmentation framework CL4WSIS based on image-level weak labels has been developed, which can gradually learn new categories while maintaining instance segmentation capabilities for old categories using only image labels.
π― What it does: A continuous learning framework named Class-Incremental Grouping Network (CIGN) is proposed for category-level semantic representation learning of audio and visual inputs in a multi-task environment, achieving audio-visual source classification.
π― What it does: This paper proposes a CLIP-based attribute hallucination framework (CLIP-Cluster) that generates features corresponding to various attributes (age, pose, expression) guided by text, and utilizes a neighbor-aware generative model to fuse these features to reduce attribute differences within the same identity, thereby achieving more compact facial clustering.
CLIP-Driven Universal Model for Organ Segmentation and Tumor Detection
Jie Liu (City University of Hong Kong), Zongwei Zhou (Johns Hopkins University)
CodeObject DetectionSegmentationConvolutional Neural NetworkVision Language ModelContrastive LearningImageBiomedical DataComputed Tomography
π― What it does: A general model based on CLIP has been proposed and trained, capable of segmenting and detecting 25 organs and 6 types of tumors across various abdominal CT datasets, and it can handle partially annotated data.
π― What it does: The CLIP2Point method is proposed, utilizing image-depth contrastive learning to pre-train a deep encoder, transferring CLIP's visual-text knowledge to 3D point cloud classification tasks.
CLIPTER: Looking at the Bigger Picture in Scene Text Recognition
Aviad Aberdam (Amazon Web Services), Ron Litman (Amazon Web Services)
CodeRecognitionTransformerVision Language ModelImage
π― What it does: This paper proposes the CLIPTER framework, which integrates global scene context into existing scene text recognizers by merging image-level features extracted from frozen vision-language models (such as CLIP/BLIP) with local features of cropped text images through gated cross-attention.