Sensor data annotations and quality check

1. Viewing and evaluating the sensor data • Review of individual sensor data or sensor platform data provided in encrypted form by DB Netz AG: - Lidar 3D point cloud data - Radar data (as image files) - Visual image data (resolution up to 20 Mio pixels; frame rates: up to 50fps; as image file) - IR image data (resolution: VGA; frame rates: up to 30fps; as image file) - GNSS data • Checking of single sensor data or sensor platform data regarding the following criteria and documentation of the outcome: - Quality, e.g. image noise, occlusions, blurring, etc. - Calibration, regarding the correct local synchronism of the reference systems of all sensors i.e. checking whether all single sensor data of the same scene in 2D/3D space are locally superimposed. • Immediate communication to the project team of DB Netz AG in case of: - Data quality problems - Content errors - Calibration errors - Other ambiguities 2. Annotation • Annotation of the classes listed in the attachments using semantic segmentation, 2D and 3D bounding boxes and/or 2D, 3D spline curves/polylines and/or 2D, 3D poly-gons (for example: person, train, rails, etc.). • The annotation requirements provided in the attachments should be able to be met by the applicants. General requirements as well as object-specific requirements shall be considered. Note, that the requirements may slightly change during the contract pe-riod. • Flexible annotation of further classes, which can be defined by DB Netz AG during the contract period. • Creation of an annotation guide, by means of which the annotation and the contents of the metadata can be traced. • The annotation data should be provided in a standardized format. For this purpose, the ASAM openLabel V1.0.0 format (or future releases, see attachments) shall be used. An example of its use is provided in the attachments. If newer versions of the standard are released, a change can also be made in the project. 3. Statistics regarding the data • The contractor visualizes and describes the status of the project to the client at regu-lar intervals. • This includes, among other things: a list of all labeled objects and object classes; number of annotated data vs. data still to be annotated, etc. 4. Review of the data • The client (i.e., DB Netz AG) must be provide the possibility of reviewing annotated data sets. Suitable tools should be available to the contractor for this purpose. • The provision of web- or browser-based tools is preferred. • These tools must have a comment function that allows the client (i.e., DB Netz AG) to leave comments on the annotations. • The contractor must be able to integrate the comments of the quality check results, by providing an interface or software tool to other companies (i.e. lot 2) for commenting. 5. Annotation accuracy and internal quality control • The annotation accuracies (for example pixel tolerance) are agreed on a class- and sensor-specific basis and are documented in the requirements • The applicant must present a concept or process for internal quality assurance/con-trol. This means that all data must go through an internal quality check process after the internal annotation process. This is to ensure the highest possible quality. 6. Transmission of the annotation results • Transmission of the annotations in a jointly agreed rhythm. 7. Integration of the results from the quality inspection • The annotation results (after internal quality assurance) are checked by another com-pany • The errors found in the annotation data must be corrected. • This is done iteratively until the quality of the data meets the specifications. The ear-lier the quality specifications are meet, the higher is the rating for the work done. • The quality check process is described in detail in the following chapters. 8. Data security requirements • Encrypted data transport and dispatch. • Deletion of the data after completion of the project. • Ensure compliance with the GDPR when processing personal data through the DB data processing agreement ["Auftragsdaten-Verarbeitungs-Vereinbarung" (AVV)]. • Viewing, processing, and using the data exclusively for the purpose defined by DB Netz AG. 9. Data anonymization (optional) • Anonymization of the faces and vehicle plates in the camera images. Estimated and maximum quantities or estimated and maximum values The estimated quantity of services that will be called for during the term of the framework service agreements (estimated quantity) is: for lot 1: 20.000.000 Annotations The maximum amount of services that may be called for during the term of the framework service agreements is: for lot 1: 25.000.000 Annotations

This task involves checking the annotation results of another company. The annotation company (i.e. lot 1) provides an interface for reviewing the annotations. Hence, the quality check company do not need own software tools for reading the data as well as the associated annotations and providing the comments. Nevertheless, own computer hardware, internet access, and properly trained employees should be available. If the company is able to check the results as well automatically, this would be a benefit. The annotation quality check includes several subtask packages: 1 Familiarize with the software tool provided by the annotation company • Familiarize the employees that should check the quality of the annotations with the tool of the annotation company. A user manual and support will be provided. 2 Documentation of the annotation errors. • Documentation of the annotation errors in the agreed format. 3 Reviewing the reworked annotation results. • The reworked results must be checked by the company according to the quality as-surance process, outlined in chapter "Quality assurance process check". 4 Providing of results • Results need to be provided within a jointly agreed frequency. • Listing and documentation of statistics on errors found. This means that it must be documented (human readable format with examples, i.e., pictures) and evaluated which types of errors occur and in which object classes or attributes they occur most frequently. • The error rate must be determined for each sequence. Documentation and descrip-tion of systematic errors should be provided in human-readable format, including a description (e.g., word document) and visualized examples (e.g., pictures, screen-shots, etc.). 5 Data security requirements. • Ensure compliance with the GDPR when processing personal data through the DB data processing agreement (see in attachments). • Viewing, processing, and using the data exclusively for the purpose defined by DB Netz AG. Estimated and maximum quantities or estimated and maximum values The estimated quantity of services that will be called for during the term of the framework service agreements (estimated quantity) is: for lot 2: 20.000.000 Annotations The maximum amount of services that may be called for during the term of the framework service agreements is: for lot 2 25.000.000 Annotations