Object Detection User Guide - SentiSight.ai

How it Works

Object detection is an advanced form of machine learning that predicts which parts of an image contain certain elements that it has been trained to recognize.

The process is fairly straightforward:

  • First, you need to upload some images.
  • Then you need to label them by selecting parts of an image containing objects you want the neural network to recognize.
  • Then, you train your model—SentiSight.ai does this for you.
  • And finally, you can use the trained model to predict where in a given image the object can be found

Let's have a closer look at those steps and how you can perform them easily using SentiSight.ai

Uploading and Labeling images

To train an object detection model, you need a dataset of uploaded and labeled images.

Press the red Upload button to upload individual images, or Upload folder or Upload zip buttons to upload large numbers of images. You can also upload your images using our API which you can read more about here.

It is a good idea to label the images for classification as you upload them, because the classification labels will be the suggested label names for the bounding boxes so it will increase the speed of bounding box labeling, provided that classification labels and object detection labels match. You can see tips how to label images for classification here.

Labeling images

Labeling images for object detection is performed using the Labeling tool. Click on Label images button on the label panel on the left, or right click on an image and select Labeling from a dropdown menu.

Note that if you don't want to label all the images in the project, you can either select a number of them, or use filters—labeling tool will ignore images that are not selected or are filtered out. By default, labeling tool will iterate through all of the images.

Sentisight.ai labeling tool is a very powerful and versatile tool, which is useful not just for object detection, but also for segmentation labeling.

However, for the purposes of object detection labeling, we will need only one shape—Bounding box. Click on the bounding box icon on the labeling tool's toolbar or press B. Then drag a bounding box around the part of an image which contains an object you want to train the model on.

Then, you must enter object label name in the box on the right hand side. If any labels already exist in the project, they will be available through a dropdown.

Alternatively you can use shortkeys 1-9 to quickly change the label of selected object to one of existing labels. You can see which shortkey corresponds to which label in labeling settings (see "Labeling tool setting" section below). There you can also assign label names to shortkeys.

You can create multiple objects in a single image like that.

If you have classification labels assigned to the image already Use previous label name setting (see "Labeling tool setting" section below) will automatically assign the image's default classification label to the object. Likewise, if you are creating multiple objects in the same image, the same setting will ensure that each new object is automatically assigned the label that was assigned to the previously created object—this can potentially speed up labeling process drastically.

Press S or switch to the next image to save the label.

Labeling tool - od - coverLabeling tool - object detection

Labeling tool setting

Labeling tool has multiple settings that can make labeling process more convenient for you.

Click the small cog icon in the top right corner of the screen to bring up the settings window and select Labeling tab.

Immediately relevant settings are Fixed bounding box aspect ratio, which allows you to create uniform bounding boxes, and Allow labeling outside of image, which is useful if you want your bounding boxes to stay neatly inside image borders.

If you are planning to use the images for classification model training as well, Synchronize labels will ensure that all images with object labels will have matching classification labels.

Here you will also find two settings that we have already mentioned. The labeling shortcut settings allows you to quicker assign label names to bounding boxes. "Use previous label name" setting allows you to use classification labels as suggested object detection labels or use the last bounding box label as the suggested label name for a new bounding box in case classification labels are not available.

Other settings are more situational, or designed for segmentation objects.

Uploading labels

Alternatively, if you have a pre-labeled dataset, you can upload it to the SentiSight.ai platform. Click the Upload labels button in the left panel and choose Upload all image labels as JSON.

Below you can see the format for object detection labels.


Bounding box fields:

  1. name - image name
  2. boundingBoxes - array of bounding boxes
  3. label - label of the bounding box
  4. occluded - whether the object is obstructed by another object (optional)
  5. x0, y0 - coordinates of the top left corner of the bounding box
  6. x1, y1 - coordinates of the bottom right corner of the bounding box
  7. attributes - array of attributes (optional)
  8. keypoints - array of keypoints (optional)

[
    {
        "name": "image_name.jpg",
        "boundingBoxes": [
            {
                "label": "boundingBox1",
                "occluded": false,
                "x0": 342,
                "y0": 359,
                "x1": 565,
                "y1": 537,
                "attributes": [
                    {
                        "name": "attribute1"
                    },
                    {
                        "name": "attribute2"
                    }
                ],
                "keypoints": [
                    {
                        "name": "keypoint1",
                        "x": 456,
                        "y": 394
                    },
                    {
                        "name": "keypoint2",
                        "x": 383,
                        "y": 499
                    }
                ]
            }
        ]
    }
]
                                                                 

Images that have been successfully labeled with object detection labels will have a polygon icon on them.

Label icon - object detectionLabel icon

Training your object detection model

Once you have uploaded and labeled the images, you can start training the models. To do this, click on Train model and then choose object detection.

Here, you can set the model name, training time, and the stop time which determines for how long the model is going to continue training if there is no improvement. The improvement is measured by mean Average Precision (mAP), a common metric in object detection.

Please note that standard training time for an object detection model is significantly longer than that for a classification model.

The default training time for object detection models depends on the number of different classes in the training set (1-2 classes: 2 hours, 3-5 classes: 3 hours, 6-10 classes: 6 hours, 11+ classes: 12 hours).

  • For advanced users: additional parameters

    If you turn on advanced view, you can select more additional parameters. These include:
    Use user-defined validation set—whether model should use images marked for validation instead of automated percentage split. You can mark images for validation, by selecting some images, and selecting "add to validation set" on the right-click menu screen.
    Change the validation set size percentage—split between training and validation images
    Learning rate—dictates the rate at which the model weights are updated during the training.
    You can also choose to include unlabeled images as negative samples—this will help the model to better differentiate between background and objects.
    Also you can select what size of model you want to train—small, medium, or large.
    It is usually recommended to select large model, as the train time difference between small, medium and large is negligible, but the accuracy is often higher for larger models.
    However, the inference is quicker for the smaller models so if your primary concern is recognition speed rather than accuracy, you should go for a smaller model.

The training process

Once you are happy with the training parameters, set your model to start training by clicking Start. You can track the progress of the model training at the top of your screen.

  • Analyzing the learning curve and early stopping the training

    In object detection model training you can check the learning curves at any time, to see how the training is going.

    After the model is trained, you can find the final learning curves in model info window.

    The validation curves which show a learning curve for accuracy, and a learning curve for cross entropy loss. The red line on the learning curves show when the best model was saved. The best model is selected based on the lowest cross-entropy loss on the validation set. In some cases, (for example, when your validation set is small and lacks diversity) you might not want to trust this estimate, so you could choose to use the "last" model instead of the "best" model.

    After approximately 10 minutes of model training have elapsed, you can stop the training process. This will not delete the model—it will be saved and usable. You might do this if you noticed that accuracy is dropping in the learning curve, or because you need the model available as soon as possible.

Analyzing the model’s performance

After the model has been trained, you can view the model’s performance by clicking on View training statistics from the “Trained models” menu.

There you can see the ‘Train time’ taken, as well as the time when the Best Model was saved. The best model is chosen by the prediction error ("cross entropy loss") on the validation set.

The statistics window is divided into two tabs: Training and Validation. Training tab contains the statistics on the images that were used for the training, while the Validation tab contains the statistics for the images which where not used for the model training. Each tab contains the label count and global statistics, such as Accuracy, Precision, Recall, F1 and mAP. You can find the definition of each statistic by clicking a question mark next to it.

You can also click Show predictions to see the actual predictions for specific images, for either the train or validation set.

A prediction is judged to be correct if the predicted bounding box sufficiently overlaps with the ground truth bounding box. The amount these two boxes overlap is measured by a so called "Intersection over Union (IoU)" measure. By default for the prediction to be considered correct, the IoU should be more than 50%, but you can change this threshold. You can also filter the results to show only the correct or incorrect predictions.

Understanding best model

Users are able to choose whether to use the ‘best model’ or to use the ‘last model’. We usually recommend using the ‘best model’ but if your validation dataset is small and lacks diversity, you might consider using the last model instead.

Train statistic - od - coverModel stats - od
  • For advanced users: downloading predictions on train / validation sets

    If you would like to download the predictions, you can do so by clicking on Download Predictions, which will download a .zip file that includes the resized images, ground truth labels, and the predictions.

  • For advanced users: Analyzing the precision-recall curve.

    In the advanced view, you can see the precision-recall curve for your model. By default, this curve is showing statistics for all classes, however you can change this to show the precision-recall curve for specific labels.

    The precision is the percentage of predictions out of all of the predictions that were made. The recall is the percentage of correctly predicted instances of some class out of all instances of that class in the data set. F1 is the harmonic mean of the precision and the recall. The precision, recall and F1 values depend on the selected score threshold.

    The score threshold determines when the prediction is considered positive and a bounding box is drawn. For example, if the score threshold is 50%, all bounding boxes whose score is above 50% are drawn. When you increase the score threshold, fewer bounding boxes will be drawn, but they will be more likely correct, thus increasing the precision. On the contrary when you decrease the score threshold, more bounding boxes will be drawn, each of which will be less likely to be correct, but they will cover a larger amount of ground truth bounding boxes, thus increasing the recall.

    By default, we select the score threshold that optimizes F1 score, which is a good compromise. However, if you prioritize either precision or recall, you can adjust the score thresholds for each class manually.

    The intersection between the precision-recall curve and the dashed red line represents the response to the optimized score threshold. If you would like to change this threshold, uncheck the ‘Use optimized thresholds’ feature, and then you can enter your own threshold score in the box, or you can change the score threshold by clicking anywhere on the precision-recall curve. Once you have set the new threshold, the performance statistics will update. When setting your own score threshold, you want to choose as high a precision and recall as possible, so if you can increase one without compromising the other, you should always do that.

    Please, not that the score thresholds change simultaneously both for train and validation sets. The new score thresholds are also represented by changed vertical dashes in the “View predictions” window.

    If you want to set a uniform threshold for all classes, this can be achieved using "Set all” button.

Making predictions

Making predictions using web-interface

The easiest way to use object detection model is through SentiSight.ai web-interface.

Predictions on Uploaded images

There are two ways to launch a prediction with an uploaded image.

  • You can click Make a new prediction button in the Trained Models dropdown.
  • You can click Make a new prediction button in the Model statistics window.

This will open predictions window and you will have to click Upload images to choose which images to perform prediction on.

You can always click Upload images to start a new prediction.

You can download the results either as images with bounding boxes drawn on them, or, in JSON format.

Predict - od - coverPredict - object detection faster

Predictions on existing images

Alternatively, you can launch predictions on existing project images.

Right-click an image in the project and press Predict and then choose desired model.

This will open predictions window and immediately launch prediction with selected model.


Making predictions via REST API

Alternatively, you can make predictions by using your preferred scripting language vis REST API, allowing you to automate the process and make predictions from an app/software that you are developing.

To begin using your trained model via REST API you will need these details:

  • API token (available under "Wallet" menu tab)
  • Project ID (available in brackets next to project names)
  • Model name (shown in many places, for example, under "Trained models" menu)

Use this endpoint: https://platform.sentisight.ai/api/predict/{your_project_id}/{your_model_name}/

Please note that for each trained model we save two checkpoints: "the best"—the one with the highest validation mean Average Precision (mAP) and "the last"—the one from the last training iteration. By default we will use "the best" model checkpoint to make the predictions for queries via REST API interface. However, if you prefer to use "the last" model checkpoint to make your predictions, you can simply add "/last" to the endpoint: https://platform.sentisight.ai/api/predict/{your_project_id}/{your_model_name}/last

Making predictions using an image from your computer

Set the "X-Auth-token" header to your API token string and set "Content-Type" header to "application/octet-stream". Set the body to your image file.

For more details, see the code samples below.

TOKEN="your_token"
PROJECT_ID="your_project_id"
MODEL="your_model_name"
IMAGE_FILENAME="your_image_path"
curl -H "X-Auth-token: $TOKEN" --data-binary @"$IMAGE_FILENAME" \
  -H "Content-Type: application/octet-stream" \
  -X POST "https://platform.sentisight.ai/api/predict/$PROJECT_ID/$MODEL"
import java.io.BufferedReader;
import java.io.DataOutputStream;
import java.io.File;
import java.io.IOException;
import java.io.InputStreamReader;
import java.net.HttpURLConnection;
import java.net.URL;
import java.nio.file.Files;

public class App
{
   public static void main( String[] args ) throws IOException
   {
       if (args.length < 4) {
           System.out.println("Usage: java -jar sample.jar api_token project_id model_name file");
       }
       String token = args[0];
       String projectId = args[1];
       String modelName = args[2];
       String imageFilename = args[3];
       
       byte[] bytes = Files.readAllBytes(new File(imageFilename).toPath());
       
       URL url = new URL("https://platform.sentisight.ai/api/predict/" + projectId + "/" + modelName);
       HttpURLConnection connection = (HttpURLConnection)url.openConnection();        
       connection.setRequestProperty("Content-Type", "application/octet-stream");
       connection.setRequestProperty("X-Auth-token", token);
       connection.setRequestMethod("POST");
       connection.setDoOutput(true);
       DataOutputStream wr = new DataOutputStream(connection.getOutputStream());
        wr.write(bytes);
        wr.flush();
        wr.close();
        
        BufferedReader in = new BufferedReader(new InputStreamReader(connection.getInputStream()));
        String output;
        StringBuffer response = new StringBuffer();

        while ((output = in.readLine()) != null) {
            System.out.println(output);
            response.append(output);
        }
        in.close();
   }
}
<!DOCTYPE html>
<html lang="en" dir="ltr">
<head>
    <meta charset="utf-8">
    <title>Sample</title>
    <script type="text/javascript">
        const baseApiURL = 'https://platform.sentisight.ai/api/';
        let token = '';
        let predictionId;
        let results;
        let resultOutput;

        function predict() {
            token = document.getElementById('tokenfield').value;
            const projectId = document.getElementById('project').value;
            const modelName = document.getElementById('model').value;
            const input = document.getElementById('upload');
            const resultOutput = document.getElementById('output');
            const file = input.files[0];
            const fr = new FileReader();
            fr.onload = function() {
                results = apiPostRequest('predict/' + projectId + '/' + modelName, fr.result);
                let parsedResults = JSON.parse(results);
                resultOutput.innerText = results
                console.log(parsedResults);
            }
            fr.readAsArrayBuffer(file);
        }

        function apiPostRequest(request, body) {
            const xmlHttp = new XMLHttpRequest();
            xmlHttp.open( "POST",  baseApiURL + request, false );
            xmlHttp.setRequestHeader('Content-Type', 'application/octet-stream');
            xmlHttp.setRequestHeader('X-Auth-token', token);
            xmlHttp.send(body);
            console.log(xmlHttp.responseText);
            return xmlHttp.responseText;
        }
    </script>
</head>
<body>
Token: <input id="tokenfield" type="text" name="" value="">
<br>
Project id: <input id="project" type="number" name="" value="">
<br>
Model name: <input id="model" type="text" name="" value="">
<br>
Upload image: <input id="upload" type="file" name="" value="">
<br>
<button type="button" onclick="predict()">Predict</button>
<br><br><br>
<p id="output">Your results will go here!</p>
</body>
</html>
import requests

token = "your_token"
project_id = "your_project_id"
model = "your_model_name"
image_filename = "your_image_path"

headers = {"X-Auth-token": token, "Content-Type": "application/octet-stream"}

with open(image_filename, 'rb') as handle:
    r = requests.post('https://platform.sentisight.ai/api/predict/{}/{}/'.format(project_id,model), headers=headers, data=handle)

if r.status_code == 200:
    print(r.text)
else:
    print('Error occured with REST API.')
    print('Status code: {}'.format(r.status_code))
    print('Error message: ' + r.text)
using System;
using System.IO;
using System.Net.Http;
using System.Net.Http.Headers;

namespace Sample
{
    class Program
    {
        static void Main(string[] args)
        {
            if (args.Length < 4)
            {
                Console.WriteLine("Expected arguments: api_token project_id model_name file");
                return;
            }
            var token = args[0];
            var projectId = args[1];
            var modelName = args[2];
            var imageFilename = args[3];

            var bytes = File.ReadAllBytes(imageFilename);
            var data = new ByteArrayContent(bytes);
            data.Headers.ContentType = MediaTypeHeaderValue.Parse("application/octet-stream");
            
            var uri = new Uri($"https://platform.sentisight.ai/api/predict/{projectId}/{modelName}");
            var client = new HttpClient();
            client.DefaultRequestHeaders.Add("X-Auth-token", token);

            var response = client.PostAsync(uri, data);
            var result = response.Result.Content.ReadAsStringAsync().Result;
            Console.WriteLine(result);
        }
    }
}

Making predictions using an image URL

Using the REST API by providing an image URL is similar to the previous case of using REST API by uploading an image. The only differences are that you need to set the "Content-Type" header to "application/json" and set the body to a JSON formatted string with a "url" parameter specifying the image URL.

For more details, see the code samples below.

TOKEN="your_token"
PROJECT_ID="your_project_id"
MODEL="your_model_name"
IMAGE_URL="your_image_url"
curl --location --request POST "https://platform.sentisight.ai/api/predict/$PROJECT_ID/$MODEL" \
--header 'X-Auth-token: $TOKEN' \
--header 'Content-Type: application/json' \
--data-raw '{
    "url": "$IMAGE_URL"
}'
package sentisight.api.sample;

import java.io.BufferedReader;
import java.io.DataOutputStream;
import java.io.IOException;
import java.io.InputStreamReader;
import java.net.HttpURLConnection;
import java.net.URL;

public class App
{
    public static void main( String[] args ) throws IOException
    {
        String token = "";
        String projectId = "";
        String modelName = "";
        String image_url = "";

        String body = "{\r\n    \"url\": \"%s\"\r\n}".formatted(image_url);

        URL url = new URL("https://platform.sentisight.ai/api/predict/" + projectId + "/" + modelName);
        HttpURLConnection connection = (HttpURLConnection)url.openConnection();
        connection.setRequestProperty("Content-Type", "application/json");
        connection.setRequestProperty("X-Auth-token", token);
        connection.setRequestMethod("POST");
        connection.setDoOutput(true);
        DataOutputStream wr = new DataOutputStream(connection.getOutputStream());
        wr.writeBytes(body);
        wr.flush();
        wr.close();

        BufferedReader in = new BufferedReader(new InputStreamReader(connection.getInputStream()));
        String output;
        StringBuilder response = new StringBuilder();

        while ((output = in.readLine()) != null) {
            response.append(output);
        }
        System.out.println(response);
        System.out.println(connection.getResponseCode());
        in.close();
    }
}
<!DOCTYPE html>
<html lang="en" dir="ltr">
<head>
    <meta charset="utf-8">
    <title>Sample</title>
    <script type="text/javascript">
        const baseApiURL = 'https://platform.sentisight.ai/api/';
        let token = '';
        let results;
        let resultOutput;

        function predict() {
            token = document.getElementById('tokenfield').value;
            const projectId = document.getElementById('project').value;
            const modelName = document.getElementById('model').value;
            resultOutput = document.getElementById('output');
            const url = document.getElementById('url').value;
            const data = JSON.stringify({
                url
            });
            results = apiPostRequest('predict/' + projectId + '/' + modelName, data)
            let parsedResults = JSON.parse(results);
            resultOutput.innerText = results
            console.log(parsedResults);
        }

        function apiPostRequest(request, body) {
            const xmlHttp = new XMLHttpRequest();
            xmlHttp.open( "POST",  baseApiURL + request, false );
            xmlHttp.setRequestHeader('Content-Type', 'application/json');
            xmlHttp.setRequestHeader('X-Auth-token', token);
            xmlHttp.send(body);
            console.log(xmlHttp.responseText);
            return xmlHttp.responseText;
        }
    </script>
</head>
<body>
Token: <input id="tokenfield" type="text" name="" value="">
<br>
Project id: <input id="project" type="number" name="" value="">
<br>
Model name: <input id="model" type="text" name="" value="">
<br>
Url: <input id="url" type="text" name="" value="">
<br>
<button type="button" onclick="predict()">Predict</button>
<br><br><br>
<p id="output">Your results will go here!</p>
</body>
</html>
import requests
import json

token = "your_token"
project_id = "your_project_id"
model = "your_model_name"
image_url = "http://your-image-url.png"

payload = json.dumps({
    "url": image_url
})

headers = {"X-Auth-token": token, "Content-Type": "application/json"}

r = requests.post('https://platform.sentisight.ai/api/predict/{}/{}/'.format(project_id, model), headers=headers,
                  data=payload)

if r.status_code == 200:
    print(r.text)
else:
    print('Error occured with REST API.')
    print('Status code: {}'.format(r.status_code))
    print('Error message: ' + r.text)
using System;
using System.IO;
using System.Net.Http;
using System.Text;
using System.Text.Json;

namespace Sample
{
    class Program
    {
        static void Main(string[] args)
        {
            if (args.Length < 4)
            {
                Console.WriteLine("Expected arguments: api_token project_id model_name url");
                return;
            }
            var token = args[0];
            var projectId = args[1];
            var modelName = args[2];
            var imageUrl = args[3];
            
            using var ms = new MemoryStream();
            using var writer = new Utf8JsonWriter(ms);
            writer.WriteStartObject();
            writer.WriteString("url", imageUrl);
            writer.WriteEndObject();
            writer.Flush();
            var json = Encoding.UTF8.GetString(ms.ToArray());
            
            var data = new StringContent(json, Encoding.Default, "application/json");

            var uri = new Uri($"https://platform.sentisight.ai/api/predict/{projectId}/{modelName}");
            var client = new HttpClient();
            client.DefaultRequestHeaders.Add("X-Auth-token", token);

            var response = client.PostAsync(uri, data);
            var result = response.Result.Content.ReadAsStringAsync().Result;
            Console.WriteLine(result);
        }
    }
}

Making predictions using a Base64 encoded image

Using the REST API by providing a Base64 encoded image is very similar to the case of using REST API with an image URL. The only difference is that you need to change the JSON parameter name "url" to "base64".

For more details, see the code samples below.

TOKEN="your_token"
PROJECT_ID="your_project_id"
MODEL="your_model_name"
IMAGE_B64=""
curl --location --request POST "https://platform.sentisight.ai/api/predict/$PROJECT_ID/$MODEL" \
--header 'X-Auth-token: $TOKEN' \
--header 'Content-Type: application/json' \
--data-raw '{
    "base64": "$IMAGE_B64"
}'
package sentisight.api.sample;

import java.io.BufferedReader;
import java.io.DataOutputStream;
import java.io.IOException;
import java.io.InputStreamReader;
import java.net.HttpURLConnection;
import java.net.URL;

public class App
{
    public static void main( String[] args ) throws IOException
    {
        String token = "";
        String projectId = "";
        String modelName = "";
        String image_b64 = "";

        String body = "{\r\n    \"base64\": \"%s\"\r\n}".formatted(image_b64);

        URL url = new URL("https://platform.sentisight.ai/api/predict/" + projectId + "/" + modelName);
        HttpURLConnection connection = (HttpURLConnection)url.openConnection();
        connection.setRequestProperty("Content-Type", "application/json");
        connection.setRequestProperty("X-Auth-token", token);
        connection.setRequestMethod("POST");
        connection.setDoOutput(true);
        DataOutputStream wr = new DataOutputStream(connection.getOutputStream());
        wr.writeBytes(body);
        wr.flush();
        wr.close();

        BufferedReader in = new BufferedReader(new InputStreamReader(connection.getInputStream()));
        String output;
        StringBuilder response = new StringBuilder();

        while ((output = in.readLine()) != null) {
            response.append(output);
        }
        System.out.println(response);
        System.out.println(connection.getResponseCode());
        in.close();
    }
}
<!DOCTYPE html>
<html lang="en" dir="ltr">
<head>
  <meta charset="utf-8">
  <title>Sample</title>
  <script type="text/javascript">
    const baseApiURL = 'https://platform.sentisight.ai/api/';
    let token = '';
    let results;
    let resultOutput;

    function predict() {
      token = document.getElementById('tokenfield').value;
      const projectId = document.getElementById('project').value;
      const modelName = document.getElementById('model').value;
      resultOutput = document.getElementById('output');
      const base64 = document.getElementById('base64').value;
      const data = JSON.stringify({
        base64
      });
      results = apiPostRequest('predict/' + projectId + '/' + modelName, data)
      let parsedResults = JSON.parse(results);
      resultOutput.innerText = results
      console.log(parsedResults);
    }

    function apiPostRequest(request, body) {
      const xmlHttp = new XMLHttpRequest();
      xmlHttp.open( "POST",  baseApiURL + request, false );
      xmlHttp.setRequestHeader('Content-Type', 'application/json');
      xmlHttp.setRequestHeader('X-Auth-token', token);
      xmlHttp.send(body);
      console.log(xmlHttp.responseText);
      return xmlHttp.responseText;
    }
  </script>
</head>
<body>
Token: <input id="tokenfield" type="text" name="" value="">
<br>
Project id: <input id="project" type="number" name="" value="">
<br>
Model name: <input id="model" type="text" name="" value="">
<br>
Base64: <input id="base64" type="text" name="" value="">
<br>
<button type="button" onclick="predict()">Predict</button>
<br><br><br>
<p id="output">Your results will go here!</p>
</body>
</html>
import requests
import json

token = "your_token"
project_id = "your_project_id"
model = "your_model_name"
image_b64 = ""

payload = json.dumps({
    "base64": image_b64
})

headers = {"X-Auth-token": token, "Content-Type": "application/json"}

r = requests.post('https://platform.sentisight.ai/api/predict/{}/{}/'.format(project_id, model), headers=headers,
                  data=payload)

if r.status_code == 200:
    print(r.text)
else:
    print('Error occured with REST API.')
    print('Status code: {}'.format(r.status_code))
    print('Error message: ' + r.text)
using System;
using System.IO;
using System.Net.Http;
using System.Text;
using System.Text.Json;

namespace Sample
{
    class Program
    {
        static void Main(string[] args)
        {
            if (args.Length < 4)
            {
                Console.WriteLine("Expected arguments: api_token project_id model_name base64");
                return;
            }
            var token = args[0];
            var projectId = args[1];
            var modelName = args[2];
            var imageB64 = args[3];
            
            using var ms = new MemoryStream();
            using var writer = new Utf8JsonWriter(ms);
            writer.WriteStartObject();
            writer.WriteString("base64", imageB64);
            writer.WriteEndObject();
            writer.Flush();
            var json = Encoding.UTF8.GetString(ms.ToArray());
            
            var data = new StringContent(json, Encoding.Default, "application/json");

            var uri = new Uri($"https://platform.sentisight.ai/api/predict/{projectId}/{modelName}");
            var client = new HttpClient();
            client.DefaultRequestHeaders.Add("X-Auth-token", token);

            var response = client.PostAsync(uri, data);
            var result = response.Result.Content.ReadAsStringAsync().Result;
            Console.WriteLine(result);
        }
    }
}

SentiSight.ai Swagger specification

We have a Swagger specification of SentiSight.ai REST API here. There you can try out our REST API interactively and convert Swagger specification to code samples in many different languages.

Using the model offline—setting up your own REST API server

For a variety of reasons you might want to use a trained model independently from SentiSight.ai platform, or indeed without internet connection at all. You can do that by setting up your own REST API server with a model you trained on SentiSight.ai.

In order to set up your own REST API server, you will have to download an offline version of the model. To do that, click on the Download model button in the "View training statistics" page.

After the model is downloaded, follow the instructions in Readme.md to set up your local REST API server. You can make the client requests from the same PC on which you set up the server, so the model would be run completely offline. On the other hand, after you set up REST API server, you can also make client requests to this server from many different devices (including mobile) on your network. Note that the REST API server must be run on a Linux system, but the client devices can run on any operating system.

The offline version of the model can be run as a free trial for 30 days. After this period, if you like the offline version, you will have to buy a license from us. Note that we have three options for the speed of the offline model: slow, medium and fast. The license price depends on the selected speed. The free trial always runs on the slow speed mode.


Video tutorials

Below you can find a number of video tutorials that are relevant to this topic.

Object detection training tutorial

Object detection training tutorial


Topics covered:
  • Basics of bounding box labeling
  • Selecting parameters
  • Training object detection model
  • Analyzing learning curve
  • Analyzing statistics and predictions
  • Analyzing precision-recall curve
  • Changing score thresholds
  • Downloading model or using it online
Labeling tool tutorial

Labeling tool tutorial


Topics covered:
  • Labeling bounding boxes, polygons, bitmaps
  • Adding keypoints ant attributes
  • Converting bounding boxes and polygons to bitmaps and vice-versa
  • Labeling occluded objects
  • Using smart labeling tool
  • Downloading and uploading image labels