Using similarity search

Using similarity search

Similarity search tool helps you to find similar images in your data set. Unlike other algorithms like image classification, object detection, and semantic/instance segmentation, similarity search tool does not require neither image labeling nor model training. You can use similarity search tool out-of-the-box!

There are two options how to use the similarity search tool: via SentiSight.ai web platform or via REST API. Additionally, there are two types of Similarity Search you can perform: 1vN that finds similar images to a single query image and NvN that finds most similar image pairs in your data set. The query image for 1vN similarity search can either be uploaded from your computer or selected from your data set on SentiSight.ai platform.

Using similarity search tool via SentiSight.ai web platform

To perform the similarity search, click on the “Similarity search” menu on the top of the screen. In the menu that pops up, you can select similarity search type 1vN or NvN. The 1vN type will find you similar images to the query image that you upload. The NvN type will look for similar image pairs in all of your data set. The result number field will adjust how many top matching images/image pairs are displayed and score threshold will determine what is the minimum similarity score for the matching images/image pairs to be displayed.

Note that you can also perform 1vN similarity search with an existing image in your data set. To do that, just right click on any image and select “Similarity search” option from a dropdown menu.

Before performing similarity search you can use “Filter by type” and “Filter by labels” tools on the lower left corner of the main screen. This will narrow down the search space. When filtering images by labels, you can choose between "and" and "or" operators.

Using similarity search tool via SentiSight.ai REST api

To begin using the similarity search tool via REST API you will need these details:

  • API token (available under "User profile" menu tab)
  • Project ID (available under "User profile" menu tab)

For 1vN similarity search you will also need an image file (either existing in your project or stored in your computer).

In all types of similarity search you can optionally specify the maximum number of results to be displayed and the similarity score threshold. You can also specify one or more image labels to filter the images in your data set prior to similarity search, hence, reducing the search space. You can also choose between "and" and "or" operators when filtering images by labels. All of these options can be specified as GET parameters when formatting the url for the request.

For more details see the code samples below.

1vN similarity search with an image from your computer

Set the "X-Auth-token" header to your API token string and set "Content-Type" header to "application/octet-stream". Set the body to your image file.


TOKEN="your_token"
PROJECT_ID="your_project_id"
IMAGE_FILENAME="your_image_path" # This is the filepath to the image on your PC
FILTER_LABELS="your labels here" # labels should be separated by spaces
AND_OPERATOR="true" # set to "false" to use "or" operator instead for filtering images by labels

LABEL_OP=''
for LABEL in $FILTER_LABELS ; do
  LABEL_OP+="&labels=$LABEL"
done

curl -H "X-Auth-token: $TOKEN" --data-binary @"$IMAGE_FILENAME" \
  -H "Content-Type: application/octet-stream" \
  -X POST "https://platform.sentisight.ai/api/similarity?project=$PROJECT_ID&label=$LABEL_OP&and=$AND_OPERATOR"


import java.io.BufferedReader;
import java.io.DataOutputStream;
import java.io.File;
import java.io.IOException;
import java.io.InputStreamReader;
import java.net.HttpURLConnection;
import java.net.URL;
import java.nio.file.Files;

public class App {
    public static void main(String[] args) throws IOException {
        String token = "";
        String projectId = "0";
        String imageFilename = "image.jpg";
        String[] labels = {"label1", "label2"};
        int limit = 10;
        double threshold = 0;

        String lstr = "";
        for (String label : labels) {
            lstr += ("&labels=" + label);
        }
        lstr += ("&and=true"); // to use 'or' operator for filtering labels remove this line or change it to '&and=false'
        lstr += ("&limit=" + limit);
        lstr += ("&threshold=" + threshold);

        URL url = new URL("https://platform.sentisight.ai/api/similarity?project=" + projectId + lstr);
        HttpURLConnection connection = (HttpURLConnection) url.openConnection();
        connection.setRequestProperty("Content-Type", "application/octet-stream");
        connection.setRequestProperty("X-Auth-token", token);
        connection.setRequestMethod("POST");
        connection.setDoOutput(true);
        DataOutputStream wr = new DataOutputStream(connection.getOutputStream());
        byte[] bytes = Files.readAllBytes(new File(imageFilename).toPath());
        wr.write(bytes);
        wr.flush();
        wr.close();

        BufferedReader in = new BufferedReader(new InputStreamReader(connection.getInputStream()));
        String output;
        while ((output = in.readLine()) != null) {
            System.out.println(output);
        }
        in.close();
    }
}


<!DOCTYPE html>
<html lang="en" dir="ltr">
    <head>
        <meta charset="utf-8">
        <title>Sample</title>
        <script>
            const baseApiURL = 'https://platform.sentisight.ai/api/';
            var token = '';
            var predictionId;
            var projectId;
            var images = [];

            function similaritySearch() {
                document.getElementById('button').disabled = true;
                token = document.getElementById('tokenfield').value;
                projectId = document.getElementById('project').value;
                var input = document.getElementById('upload');
                const file = input.files[0];
                var fr = new FileReader();
                fr.onload = function() {
                    performSimilaritySearch(fr.result);
                }
                if (file) {
                    fr.readAsArrayBuffer(file);
                }
            }

            function performSimilaritySearch(file) {
                var labelText = document.getElementById('labels').value;
                var labels = [];
                if (labelText.length) {
                    labels = labelText.split(',');
                }
                var lstr = '';
                labels.forEach(function(label) {
                    lstr += ('&labels=' + label);
                });
                if (document.getElementById('limit-on').checked) {
                    lstr += ('&limit=' + document.getElementById('limit').value)
                }
                if (document.getElementById('threshold-on').checked) {
                    lstr += ('&threshold=' + document.getElementById('threshold').value)
                }
                if (document.querySelector('input[name="and"]:checked').value == 'true') {
                    lstr += '&and=true';
                }
                var xmlHttp = new XMLHttpRequest();
                xmlHttp.open('POST',  baseApiURL + 'similarity?project=' + projectId + lstr, true);
                xmlHttp.setRequestHeader('Content-Type', 'application/octet-stream');
                xmlHttp.setRequestHeader('X-Auth-token', token);
                xmlHttp.onreadystatechange = function () {
                    if (this.readyState == 4 && this.status == 200) {
                        document.getElementById('button').disabled = false;
                        document.getElementById('results').innerHTML = xmlHttp.responseText;
                    }
                }
                xmlHttp.send(file);
            }

        </script>
    </head>
    <body>
        Token: <input id="tokenfield" type="text">
        <br>
        Project id: <input id="project" type="number">
        <br>
        Labels: <input id="labels" type="text">
        <br>
        <input type="radio" name="and" value="true"> And
        <input type="radio" name="and" value="false" checked> Or
        <br>
        <input id="limit-on" type="checkbox">Result number: <input id="limit" type="number">
        <br>
        <input id="threshold-on" type="checkbox">Score threshold: <input id="threshold" type="number" step="0.001">
        <br>
        Upload image: <input id="upload" type="file">
        <br>
        <button id="button" type="button" onclick="similaritySearch()">Perform similarity search</button>
        <br>
        <table id=results></table>
    </body>
</html>

import requests

token = "your_token"
project_id = "your_project_id"
image_filename = "your_image_path"
labels = ["your", "labels", "here"]
limit = 10  # limits result count to this number
threshold = 0  # only shows results above this threshold (value in percent)

headers = {"X-Auth-token": token, "Content-Type": "application/octet-stream"}

request_url = "https://platform.sentisight.ai/api/similarity?project={}".format(project_id)
for lbl in labels:
    request_url += "&labels={}".format(lbl)

request_url += "&and=true" # to use 'or' operator for filtering labels remove this line or change it to '&and=false'
request_url += "&limit={}".format(limit)
request_url += "&threshold={}".format(threshold)

with open(image_filename, "rb") as handle:
    r = requests.post(request_url, headers=headers, data=handle)

if r.status_code == 200:
    print(r.text)
else:
    print("Error occured with REST API.")
    print("Status code: {}".format(r.status_code))
    print("Error message: " + r.text)

1vN similarity search with an existing image in your data set

Set the "X-Auth-token" header to your API token string and set "Content-Type" header to "text/plain". Set the body to your query image name. The query image should be already uploaded to the SentiSight.ai platform.


TOKEN="your_token"
PROJECT_ID="your_project_id"
IMAGE_NAME="your_image_name" # this the name of an image which is already uploaded on platform.sentisight.ai
FILTER_LABELS="your labels here" # labels should be separated by spaces
AND_OPERATOR="true" # set to "false" to use "or" operator instead for filtering images by labels

LABEL_OP=''
for LABEL in $FILTER_LABELS ; do
  LABEL_OP+="&labels=$LABEL"
done

curl -H "X-Auth-token: $TOKEN" --data "$IMAGE_NAME" \
  -H "Content-Type: text/plain" \
  -X POST "https://platform.sentisight.ai/api/similarity?project=$PROJECT_ID&label=$LABEL_OP&and=$AND_OPERATOR"

import java.io.BufferedReader;
import java.io.DataOutputStream;
import java.io.IOException;
import java.io.InputStreamReader;
import java.net.HttpURLConnection;
import java.net.URL;

public class App {
    public static void main(String[] args) throws IOException {
        String token = "";
        String projectId = "0";
        String imageName = "image.jpg";
        String[] labels = {"label1", "label2"};
        int limit = 10;
        double threshold = 0;

        String lstr = "";
        for (String label : labels) {
            lstr += ("&labels=" + label);
        }
        lstr += ("&and=true"); // to use 'or' operator for filtering labels remove this line or change it to '&and=false'
        lstr += ("&limit=" + limit);
        lstr += ("&threshold=" + threshold);

        URL url = new URL("https://platform.sentisight.ai/api/similarity?project=" + projectId + lstr);
        HttpURLConnection connection = (HttpURLConnection) url.openConnection();
        connection.setRequestProperty("Content-Type", "text/plain");
        connection.setRequestProperty("X-Auth-token", token);
        connection.setRequestMethod("POST");
        connection.setDoOutput(true);
        DataOutputStream wr = new DataOutputStream(connection.getOutputStream());
        wr.writeBytes(imageName);
        wr.flush();
        wr.close();

        BufferedReader in = new BufferedReader(new InputStreamReader(connection.getInputStream()));
        String output;
        while ((output = in.readLine()) != null) {
            System.out.println(output);
        }
        in.close();
    }
}

<!DOCTYPE html>
<html lang="en" dir="ltr">
    <head>
        <meta charset="utf-8">
        <title>Sample</title>
        <script>
            const baseApiURL = 'https://platform.sentisight.ai/api/';
            var token = '';
            var predictionId;
            var projectId;
            var images;

            function similaritySearch(input) {
                document.getElementById('button').disabled = true;
                token = document.getElementById('tokenfield').value;
                projectId = document.getElementById('project').value;
                var input = document.getElementById('image-name').value;
                var labelText = document.getElementById('labels').value;
                var labels = [];
                if (labelText.length) {
                    labels = labelText.split(',');
                }
                var lstr = '';
                labels.forEach(function(label) {
                    lstr += ('&labels=' + label);
                });
                if (document.getElementById('limit-on').checked) {
                    lstr += ('&limit=' + document.getElementById('limit').value)
                }
                if (document.getElementById('threshold-on').checked) {
                    lstr += ('&threshold=' + document.getElementById('threshold').value)
                }
                if (document.querySelector('input[name="and"]:checked').value == 'true') {
                    lstr += '&and=true';
                }
                var xmlHttp = new XMLHttpRequest();
                xmlHttp.open('POST',  baseApiURL + 'similarity?project=' + projectId + lstr, true);
                xmlHttp.setRequestHeader('Content-Type', 'text/plain');
                xmlHttp.setRequestHeader('X-Auth-token', token);
                xmlHttp.onreadystatechange = function () {
                    if (this.readyState == 4 && this.status == 200) {
                        document.getElementById('button').disabled = false;
                        document.getElementById('results').innerHTML = xmlHttp.responseText;
                    }
                }
                xmlHttp.send(input);
            }

        </script>
    </head>
    <body>
        Token: <input id="tokenfield" type="text">
        <br>
        Project id: <input id="project" type="number">
        <br>
        Labels: <input id="labels" type="text">
        <br>
        <input type="radio" name="and" value="true"> And
        <input type="radio" name="and" value="false" checked> Or
        <br>
        <input id="limit-on" type="checkbox">Result number: <input id="limit" type="number">
        <br>
        <input id="threshold-on" type="checkbox">Score threshold: <input id="threshold" type="number" step="0.001">
        <br>
        Image name: <input id="image-name" type="text">
        <br>
        <button id="button" type="button" onclick="similaritySearch()">Perform similarity search</button>
        <br>
        <table id=results></table>
    </body>
</html>

import requests

token = "your_token"
project_id = "your_project_id"
image_name = "your_image_name".encode("utf-8")
labels = ["your", "labels", "here"]
limit = 10  # limits result count to this number
threshold = 0  # only shows results above this threshold  (value in percent)

headers = {"X-Auth-token": token, "Content-Type": "text/plain"}

request_url = "https://platform.sentisight.ai/api/similarity?project={}".format(project_id)
for lbl in labels:
    request_url += "&labels={}".format(lbl)
    
request_url += "&and=true" # to use 'or' operator for filtering labels remove this line or change it to '&and=false'
request_url += "&limit={}".format(limit)
request_url += "&threshold={}".format(threshold)

r = requests.post(request_url, headers=headers, data=image_name)

if r.status_code == 200:
    print(r.text)
else:
    print("Error occured with REST API.")
    print("Status code: {}".format(r.status_code))
    print("Error message: " + r.text)

NvN similarity search

Set the "X-Auth-token" header to your API token string and set "Content-Type" header to "text/plain".


TOKEN="your_token"
PROJECT_ID="your_project_id"
FILTER_LABELS="your labels here" # labels should be separated by spaces
AND_OPERATOR="true" # set to "false" to use "or" operator instead for filtering images by labels

LABEL_OP=''
for LABEL in $FILTER_LABELS ; do
  LABEL_OP+="&labels=$LABEL"
done

curl -H "X-Auth-token: $TOKEN" \
  -H "Content-Type: text/plain" \
  -X GET "https://platform.sentisight.ai/api/similarity?project=$PROJECT_ID&label=$LABEL_OP&and=$AND_OPERATOR"

import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
import java.net.HttpURLConnection;
import java.net.URL;

public class App {
    public static void main(String[] args) throws IOException {
        String token = "";
        String projectId = "0";
        String[] labels = {"label1", "label2"};
        int limit = 10;
        double threshold = 0;
        
        String lstr = "";
        for (String label : labels) {
            lstr += ("&labels=" + label);
        }
        lstr += ("&and=true"); // to use 'or' operator for filtering labels remove this line or change it to '&and=false'
        lstr += ("&limit=" + limit);
        lstr += ("&threshold=" + threshold);

        URL url = new URL("https://platform.sentisight.ai/api/similarity?project=" + projectId + lstr);
        HttpURLConnection connection = (HttpURLConnection) url.openConnection();
        connection.setRequestProperty("Content-Type", "text/plain");
        connection.setRequestProperty("X-Auth-token", token);
        connection.setRequestMethod("GET");

        BufferedReader in = new BufferedReader(new InputStreamReader(connection.getInputStream()));
        String output;
        while ((output = in.readLine()) != null) {
            System.out.println(output);
        }
        in.close();
    }
}

<!DOCTYPE html>
<html lang="en" dir="ltr">
    <head>
        <meta charset="utf-8">
        <title>Sample</title>
        <script>
            const baseApiURL = 'https://platform.sentisight.ai/api/';
            var token = '';
            var predictionId;
            var projectId;

            function similaritySearch(input) {
                document.getElementById('button').disabled = true;
                token = document.getElementById('tokenfield').value;
                projectId = document.getElementById('project').value;
                var labelText = document.getElementById('labels').value;
                var labels = [];
                if (labelText.length) {
                    labels = labelText.split(',');
                }
                var lstr = '';
                labels.forEach(function(label) {
                    lstr += ('&labels=' + label);
                });
                if (document.getElementById('limit-on').checked) {
                    lstr += ('&limit=' + document.getElementById('limit').value)
                }
                if (document.getElementById('threshold-on').checked) {
                    lstr += ('&threshold=' + document.getElementById('threshold').value)
                }
                if (document.querySelector('input[name="and"]:checked').value == 'true') {
                    lstr += '&and=true';
                }
                var xmlHttp = new XMLHttpRequest();
                xmlHttp.open('GET',  baseApiURL + 'similarity?project=' + projectId + lstr, true);
                xmlHttp.setRequestHeader('Content-Type', 'text/plain');
                xmlHttp.setRequestHeader('X-Auth-token', token);
                xmlHttp.onreadystatechange = function () {
                    if (this.readyState == 4 && this.status == 200) {
                        document.getElementById('button').disabled = false;
                        document.getElementById('result').innerHTML = xmlHttp.responseText;
                    }
                }
                xmlHttp.send();
            }

        </script>
    </head>
    <body>
        Token: <input id="tokenfield" type="text">
        <br>
        Project id: <input id="project" type="number">
        <br>
        Labels: <input id="labels" type="text">
        <br>
        <input type="radio" name="and" value="true"> And
        <input type="radio" name="and" value="false" checked> Or
        <br>
        <input id="limit-on" type="checkbox">Result number: <input id="limit" type="number">
        <br>
        <input id="threshold-on" type="checkbox">Score threshold: <input id="threshold" type="number" step="0.001">
        <br>
        <button id="button" type="button" onclick="similaritySearch()">Perform similarity search</button>
        <br>
        <table id="result"></table>
    </body>
</html>

import requests

token = "your_token"
project_id = "your_project_id"
labels = ["your", "labels", "here"]
limit = 10  # limits result count to this number
threshold = 0.5  # only shows results above this threshold (value in percent)

headers = {"X-Auth-token": token, "Content-Type": "text/plain"}

request_url = "https://platform.sentisight.ai/api/similarity?project={}".format(project_id)
for lbl in labels:
    request_url += "&labels={}".format(lbl)

request_url += "&and=true" # to use 'or' operator for filtering labels remove this line or change it to '&and=false'
request_url += "&limit={}".format(limit)
request_url += "&threshold={}".format(threshold)

r = requests.get(request_url, headers=headers)

if r.status_code == 200:
    print(r.text)
else:
    print("Error occured with REST API.")
    print("Status code: {}".format(r.status_code))
    print("Error message: " + r.text)