SentiSight.ai’s platform is constantly evolving and developing the image annotation tool features and capabilities it has available for its users. This article will guide you through the labeling processes within our company, showcasing how the platform can be used to solve real-world problems.
While labeling images is a very important step in a supervised machine learning project, it does not require any special equipment. Because of this, our current data annotation team consists of 7 people flexibly working from their location of choice – whether that be our local offices or off-site, the only requirements are having access to the online or offline labeling tool and the project’s dataset.
Labeling is a tedious job requiring annotators to sit in front of their computers throughout the day, yet with the right tools, it can be done without much difficulty.
Every project starts from an instruction set provided by the supervisor. It usually consists of a full description of the task:
This instruction set makes sure each team member is informed about the requirements so the resulting labels in the dataset do not vary too much. To paint a clearer picture, we have chosen a few of our past projects for further analysis.
One of our clients in Argentina came to us with a request for a solution that would recognize and classify crops and weeds. The dataset was not very large, therefore, they decided to label it themselves. SentiSight.ai’s annotation tool is very convenient for this purpose because it can be accessed from anywhere in the world and the client does not need to install any additional software to label the images.
As a pre-processing step, the background soil was removed from the images to get a clearer view of the crops and the images were passed to the client to label them. In total, it took around 20 hours to label all of the data. After that, an object detection model was trained to detect different types of corn, and whether there were any weeds in the field.
Another project was requested by SentiVeillance SDK – a software development product specializing in surveillance, people and vehicle tracking and classification, plus licence plate recognition proprietary software developed at Neurotechnology. The team of labelers received an instruction set requiring the following tasks to be completed:
A team of 5 people shared the tasks between them and finished labeling a dataset of 3269 images in 18 hours. Depending on the project’s complexity and the annotator’s experience, annotating an image took between 15 seconds and a few minutes.
Currently, the labeling team is working on several projects, one of them is labeling the human head pose. In three-dimensional space, pose can be described via three angles – roll, pitch and yaw. Usually, they are explained as aircraft principal axes, showcasing how an aircraft in flight is able to rotate. Rotation around the front-to-back axis is called roll, around the side-to-side axis pitch and around the vertical axis – yaw.
This labeling task requires fixing the existing roll, pitch and yaw angles on human faces, indicating their facing direction. Since the dataset is quite small, containing only around 8000 images, one person assigned to the task is able to successfully take care of them.
Needless to say, we do run into some obstacles, the most common being new annotators labeling so meticulously, therefore spending too much time on one task. Accurate labeling is a highly desired skill, yet there needs to be a balance between speed and quality to perform well on a job like this. Moreover, in some cases, machine learning algorithms are able to retrieve the required data without pixel-sized accuracy. Another issue we sometimes run into is varying comprehension of the task, although we are successfully solving this with more informative and simpler instructions given prior to the task.
Once the project has finished, the quality of work of each annotator is then judged on the following categories:
These parameters are then discussed during monthly performance reviews.
The SentiSight.ai platform provides numerous useful features for annotators and their supervisors to manage the team better:
Although labeling a large dataset is a labour intensive process, it does not feel that way when the most suitable labeling tool kit for the job is chosen. At SentiSight.ai we offer an intuitive image labeling platform designed for the best user experience, no matter whether it is for a beginner or an expert. Various annotation options allow labeling bounding boxes, polygons, points, polylines together with smart labeling tools for labelers while filtering and time tracking features ease the project manager’s job. Check out our image tabeling tool user guide for more information on the available intuitive features.