Imagine driving in foggy weather and seeing a figure on the side of the road. Is it a person, a traffic sign, or just a shadow? Autonomous vehicles encounter such situations daily, and they must reliably recognize their surroundings. This is made possible by data annotation, a process in which data fed into an AI model is preprocessed. With annotated data, an AI model learns to recognize shapes such as buildings or cars after training.
What is Data Annotation?
Data annotation is the process of labeling and classifying raw data (such as images, audio, or point clouds) for training machine learning models. To identify an obstacle ahead, a vehicle relies on machine learning models trained with previously annotated data. This data is processed either by humans or by models specifically designed for this purpose. During training, a machine learning model learns to recognize various objects, such as moving and stationary vehicles, pedestrians, and obstacles. Once the model has been trained on a sufficiently large amount of annotated data, it can make observations and predictions independently without direct comparison to example data.
Figure 1 shows a point cloud both annotated and unannotated. The unannotated point cloud (right) has no assigned classes, appearing blue in the image. The annotated point cloud (left) appears colorful, as each color represents a specific class. In practice, human annotation involves selecting a cluster of points that resemble an object and marking it with a desired class, as shown in Figure 2. When a sufficient number of examples have been annotated, the model learns to recognize obstacles ahead without human assistance. Each object visible in Figure 1 has its own class, which is listed in Figure 4.
Various types of data can be annotated, including audio, point clouds, images, and text, using different tools. Humans act as translators for devices that require annotated data to function safely and appropriately for their intended purpose.
The ROADVIEW Project
ROADVIEW is an EU-funded Horizon Europe innovation project aimed at developing sustainable and cost-effective perception and decision-making systems for connected and automated vehicles (ROADVIEW, 2024). The acronym ROADVIEW stands for Robust Automated Driving in Extreme Weather. The project focuses particularly on improving performance in challenging weather conditions and various traffic situations, such as heavy snow or rain. So far, most automated vehicles have been trained and tested primarily under optimal weather and road conditions with good visibility (FrostBit 2024). The project aims to develop an onboard perception and decision-making system based on enhanced sensing, localization, and more accurate object and human classification.
Data annotation is part of the ROADVIEW project’s development work. The project creates publicly shareable sample data of an environment where a car equipped with a LiDAR sensor drives different routes while generating point clouds of its surroundings. The annotator reviews this point cloud and marks obstacles around the vehicle. This annotated data is used to train machine learning models, enabling them to make accurate predictions even in challenging conditions. The focus is particularly on data collected in difficult weather conditions, such as heavy rain or snow.
Annotating a single point cloud takes approximately five hours. Before the data can be used for training, it must be reviewed again for errors. The data may be checked multiple times to enhance its accuracy and reliability. In practice, the review process takes nearly as much time as annotation itself.
Tools and Platforms
The ROADVIEW project uses Segments.ai for data annotation, a tool launched in 2020 by Otto Debals and Bert De Brabander (Segments.ai, 2024). Segments.ai was chosen for the project due to its ease of use in annotating 3D point cloud data. This tool enables the annotation of various data types, including images and point clouds.
Besides Segments.ai, several other annotation tools and platforms exist, such as Labelbox, CVAT, Supervisely, and Amazon SageMaker Ground Truth, each specializing in a specific data type. For example, Segments.ai focuses on 3D data like point clouds, while Labelbox specializes in image, video, text, and audio data. Different annotation tools offer diverse features. The table below compares the most popular tools and their areas of specialization.
Among these tools, Label Studio has been used in the BerryMachine -project, where an automated berry recognition and counting system was developed using artificial intelligence (FrostBit, 2021).
SemanticKITTI
The ROADVIEW project follows the class definitions of SemanticKITTI, a dataset developed through the collaboration of the Photogrammetry & Robotics Group, Computer Vision Group, and Autonomous Intelligent Systems Group at the University of Bonn. SemanticKITTI includes 28 classes in total. Figure 3 presents the SemanticKITTI classes.
In the ROADVIEW project, a few additional classes have been included to better describe obstacles in front of the vehicle, such as distinguishing between moving and stationary cars. While SemanticKITTI also has classes for moving objects, these are combined into their stationary class counterparts. For instance, a moving car is categorized as a “car” class.
What is a LiDAR Sensor?
LiDAR (Light Detection and Ranging) is a distance measurement technology that utilizes laser pulses reflecting off objects to measure distances and create precise three-dimensional point maps of the environment (National Oceanic and Atmospheric Administration, 2024). LiDAR technology is one of the key sensor technologies in the development of autonomous vehicles. It enables self-driving cars to accurately and in real-time perceive their surroundings.
Below, in Figure 5, is an example of a LiDAR sensor. There are various types of sensors, but the Ouster OS1 shown in the image is weather-resistant and operates even in temperatures as low as -40 degrees Celsius.
LiDAR sensors emit laser pulses that bounce off surrounding obstacles and return to the LiDAR detector. This process creates an accurate three-dimensional point cloud of the vehicle’s environment, enabling obstacle detection and safe driving even in challenging conditions.
Summary
Data annotation is a crucial part of autonomous vehicle development, as it enables AI models to accurately and reliably identify objects in various environments and weather conditions. The ROADVIEW project aims to enhance perception systems, especially in challenging weather conditions, with annotated data playing a key role in achieving this goal.
Tools such as Segments.ai allow for efficient and precisely controlled annotation, advancing the development of autonomous vehicles.
ROADVIEW is an EU-funded Horizon Europe project involving 16 partners from 7 countries. The total project budget is €9.7 million, with an implementation period from September 1, 2022, to August 31, 2026.
Sources
Behley, J., Garbade, M., Milioto, A., Quenzel, J., Behnke, S., Stachniss, C. & Gall, J. 2019. SemanticKITTI: A Dataset for Semantic Scene Understanding of LiDAR Sequences. Accessed 19.02.2025. https://arxiv.org/abs/1904.01416.
FrostBit. 2021. Berry Machine. Accessed 17.03.2024. FrostBit Portfolio, Berry Machine.
FrostBit. 2024. Roadview. Accessed 16.10.2024. FrostBit Portfolio, Roadview.
National Oceanic and Atmospheric Administration (NOAA). 2024. What is lidar? Accessed 17.04.2024. Oceanservice.noaa.gov.
ROADVIEW. 2024. Robust Automated Driving in Extreme Weather. Accessed 16.10.2024. https://roadview-project.eu/.
Segments.ai. 2024. Our History. Accessed 04.03.2025. https://segments.ai/company/.
University of Bonn. 2021. Overview. Accessed 16.10.2024. https://www.semantic-kitti.org/dataset.html.
The article publications are written by the professionals at FrostBit, related to the activities and results of the projects, as well as on other topics related to RDI activities and the ICT sector. The articles are evaluated by FrostBit’s publishing committee.
Sofie Väisänen
Sofie Väisänen, an expert and Bachelor of Science in Information and Communications Technology, works in the Digital Solutions competence group at Lapland University of Applied Sciences.