Annotation

Annotation: The paper considers the task of recognition and interpretation of object images based on the mixed reality headset Microsoft Hololens 2 in the context of optimization of the process of identification of personal computer components (PCCs). To solve the task, the software tools (ST) with client-server architecture have been developed. The client part of the ST is located on the Microsoft Hololens 2 and is responsible for the graphical interface, generation of the PCC images, as well as sending requests to the server. The server part of the ST contains image annotation module, text description translation module, and also the database module with information about the PCC names, their text descriptions and verification images. The annotation and text translation modules are based on the application of deep learning neural network models such as BLIP and T5, respectively, which are transformer models. The finetuning of the BLIP model is performed on the dataset containing examples from the subject area in the form of pairs "image – annotation": it allowed to form accurate annotations of the PCCs in the process of image recognition. The developed STs can be used to inventorize the PCCs using Microsoft Hololens 2 to optimize the process of their identification, as well as in the training of personnel working with the PCCs.