Researchers at the University of Surrey have developed a framework for object detection based on sketches, which can detect specific objects in a scene without requiring extra boundary boxes or class labels. The model is trained with a multi-category cross-entropy loss across the prototypes of all conceivable categories or instances in a weakly supervised object detection (WSOD) environment. The framework outperforms supervised and weakly supervised object detectors in a zero-shot setting and combines CLIP and Sketch-Based Image Retrieval (SBIR) to produce a sketch-aware detector that can function in a zero-shot fashion.
Facebook Research has released the Segment Anything Model (SAM) under the Apache 2.0 license, which uses machine learning to reliably figure out which pixels in an image belong to an object. SAM has been trained on a huge dataset of high-quality images and masks, making it very effective at what it does. Once an image is segmented, those masks can be used to interface with other systems like object detection and other computer vision applications.
Meta has released an AI model called "Segment Anything" that can detect objects in pictures and videos even if they weren't part of the training set. The model can work in tandem with other models and can limit the need for additional AI training. The AI model and dataset will be downloadable with a non-commercial license. While the model is flawed, it may help in situations where it's impractical to rely exclusively on training data.
