Labeling by similarity - SentiSight.ai

Labeling by similarity

How it works

Labeling by similarity refers to a way to assign labels to images using similarity search functionality. It can be used to automatically label images or assist in manual labeling.

Basically, it allows you to compare new images to existing images and automatically assign labels from the most similar images to the new ones. It does not require training a model and can be used as soon as you have some labeled images in your project.

  1. Upload some images.
  2. Label those images by hand
  3. Use Labeling by similarity to automatically label more images.

You can upload your images using the platform or via REST API. If you are using the platform, press the red Upload button to upload individual images, or Upload folder or Upload zip buttons to upload large numbers of images. If you are planning to use the API for image upload, read more about it here.

Note that labeling by similarity becomes more accurate the more labeled images you have in your projectg.

You can use Labeling by similarity both from web interface and using REST API.

Please note, that currently, we do not have a separate "Label by Similarity" offline model, but users can use our regular "Image similarity" model for those purposes. The best way to do it would be to store the image labels locally and once you get the similarity search results, calculate suggested labels manually. The suggested labels are actually labels of similarity hits multiplied by their similarity scores.

Similarity labeling with web platform

To use Labeling by similarity in web platform, go to Image Similarity - Label by Similarity and upload images you want to label.

You will see images matched by similarity to existing images, but the platform will also assign labels from the most similar images to the query images.

simlabel01simlabel01

To do so, just select the images you want labeled, right click on them and click AI Tool - Label by similarity. This will bring up Label by smiliarity dialog, allowing you to add or change the images' labels.

simlabel03simlabel03

In similarity labeling window you can change the number of images that each image will be compared to, as well as similarity threshold necessary for automatic labeling.

  • Number of results - how many already labeled images will each uploaded image be compared to.
  • Mark only top scoring label - only one label will be automatically ticked, useful for single-label classification training.
  • Threshold - similarity threshold, below which labels will not be automatically assigned.

You can also assign or remove suggested labels, allowing you to correct inaccuracies.

Please note that changing any values will discard any manually applied labels.

After you are done adjusting values and assigning labels, don't forget to click Add to dataset to save your changes and add any new images to your project.

simlabel02simlabel02

Similarity labeling with REST API

You can also use similarity labeling via REST API.

Set the "X-Auth-token" header to your API token string and set "Content-Type" header to "application/octet-stream". Set the body to your image file.

For more details, see the code samples below.

TOKEN="your_token"
PROJECT_ID="your_project_id"
IMAGE_FILENAME="your_image_name" # This is the filepath to the image on your PC
SAVE="false"
USE_FIRST="false"
THRESHOLD="10"

curl --location --request POST 'http://localhost:9080/rs/api/similarity-labels?project=1414&threshold=10&save=true&use-first=true' \
--header 'X-Auth-token: m4go96jsof5jhmq820kncfjln5' \
--header 'Content-Type: application/octet-stream' \
--data-binary '@$IMAGE_FILENAME'
package sentisight.api.sample;

import java.io.BufferedReader;
import java.io.DataOutputStream;
import java.io.File;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.net.HttpURLConnection;
import java.net.URL;
import java.nio.file.Files;

public class App {
    public static void main(String[] args) throws IOException {
        String token = "your_token";
        String projectId = "your_project_id";
        int threshold = 10;
        String imageFilename = "your_image_path";

        StringBuilder paramsString = new StringBuilder();
        paramsString.append("&save=false"); // to add results to dataset, use '&save=true'
        paramsString.append("&use-first=false"); // uses results with scores above threshold.
        // To get only the result with the highest score, use '&use-first=true'
        paramsString.append("&threshold=").append(threshold);

        URL url = new URL("https://platform.sentisight.ai/api/similarity-labels?project=" + projectId + paramsString);
        HttpURLConnection connection = (HttpURLConnection) url.openConnection();
        connection.setRequestProperty("Content-Type", "application/octet-stream");
        connection.setRequestProperty("X-Auth-token", token);
        connection.setRequestMethod("POST");
        connection.setDoOutput(true);
        DataOutputStream wr = new DataOutputStream(connection.getOutputStream());
        byte[] bytes = Files.readAllBytes(new File(imageFilename).toPath());
        wr.write(bytes);
        wr.flush();
        wr.close();

        InputStream stream = connection.getInputStream();
        BufferedReader in = new BufferedReader(new InputStreamReader(stream));
        String output;
        while ((output = in.readLine()) != null) {
            System.out.println(output);
        }
        in.close();
    }
}

<!DOCTYPE html>
<html lang="en" dir="ltr">
    <head>
        <meta charset="utf-8">
        <title>Sample</title>
        <script>
            const baseApiURL = 'https://platform.sentisight.ai/api/';

            function similaritySearch() {
                document.getElementById('button').disabled = true;
                var input = document.getElementById('upload');
                const file = input.files[0];
                var fr = new FileReader();
                fr.onload = function() {
                    performSimilaritySearch(fr.result);
                }
                if (file) {
                    fr.readAsArrayBuffer(file);
                }
            }

            function performSimilaritySearch(file) {
                var project = document.getElementById('project').value;
                var token = document.getElementById('token').value;
                var threshold = document.getElementById('threshold').value;
                var save = document.getElementById('save').checked;
		var first = document.getElementById('first').checked;
                var xmlHttp = new XMLHttpRequest();
                xmlHttp.open('POST',  baseApiURL + 'similarity-labels?project=' + project + "&threshold=" + threshold + "&save=" + save + "&use-first=" + first, true);
                xmlHttp.setRequestHeader('Content-Type', 'application/octet-stream');
                xmlHttp.setRequestHeader('X-Auth-token', token);
                xmlHttp.onreadystatechange = function () {
                    if (this.readyState == 4 && this.status == 200) {
                        document.getElementById('button').disabled = false;
                        document.getElementById('results').innerHTML = xmlHttp.responseText;
                    }
                }
                xmlHttp.send(file);
            }

        </script>
    </head>
    <body>
        Token: <input id="token" type="text">
        <br>
        Project id: <input id="project" type="number">
        <br>
        Threshold: <input id="threshold" type="number">
        <br>
        Save results: <input id="save" type="checkbox">
        <br>
        Use first result: <input id="first" type="checkbox">
        <br>
        Upload image: <input id="upload" type="file">
        <br>
        <button id="button" type="button" onclick="similaritySearch()">Perform similarity search</button>
        <br>
        <table id=results></table>
    </body>
</html>

import requests

token = "your_token"
project_id = "your_project_id"
image_filename = "your_image_path"
threshold = 10  # only uses results above this threshold (value in percent)
first = False
save = False

headers = {"X-Auth-token": token, "Content-Type": "application/octet-stream"}

request_url = "https://platform.sentisight.ai/api/similarity-labels?project={}".format(project_id)

request_url += "&save={}".format(save) # to add results to dataset, use '&save=true'
request_url += "&use-first={}".format(first) # uses results with scores above threshold.
# To get only the result with the highest score, use '&use-first=true'
request_url += "&threshold={}".format(threshold) # unused if 'use-first' is true

with open(image_filename, "rb") as handle:
    r = requests.post(request_url, headers=headers, data=handle)

if r.status_code == 200:
    print(r.text)
else:
    print("Error occured with REST API.")
    print("Status code: {}".format(r.status_code))
    print("Error message: " + r.text)

using System;
using System.IO;
using System.Linq;
using System.Net.Http;
using System.Net.Http.Headers;

namespace Sample
{
    class Program
    {
        static void Main()
        {
            const string token = "";
            const int projectId = 0;
            const string imageFilename = "";
            const double threshold = 10;
            
            var paramString = "";
            paramString += ("&save=false"); 
            paramString += ("&use-first=false");
            paramString += ("&threshold=" + threshold);
            
            var bytes = File.ReadAllBytes(imageFilename);
            var data = new ByteArrayContent(bytes);
            data.Headers.ContentType = MediaTypeHeaderValue.Parse("application/octet-stream");

            var uri = new Uri($"http://platform.sentisight.ai/api/similarity-labeling?project={projectId}{paramString}");
            var client = new HttpClient();
            client.DefaultRequestHeaders.Add("X-Auth-token", token);
            
            var response = client.PostAsync(uri, data);
            var result = response.Result.Content.ReadAsStringAsync().Result;
            Console.WriteLine(result);
        }
    }
}

Video tutorial

For more information, check out the video tutorial detailing labeling by similarity.

labeling-by-image-similarity-tutorial

Labeling by image similarity tutorial


Topics covered:
  • Labeling by image similarity feature
  • Changing parameters
  • Adjusting suggested labels manually
  • Performing AI-assisted labeling iteratively
  • Downloading classification labels
gdpr-image
We use cookies and other technologies to ensure that we give you the best experience on our website. If you continue to use this site we will assume that you are happy with it..
Privacy policy