Annotation

Annotation: Matrix transformations form the foundation of modern data mining and machine learning algorithms. The relevance of the study stems from the necessity to enhance the skills of novice programmers in software implementation of matrix transformations, as well as to strengthen their software testing skills for the early detection of defects and errors, in compliance with the guidelines established in GOST R 56920–2024 and GOST R 56939–2024. The aim of the research is the automation of the generation and assessment process of exercises on the software implementation and testing of matrix transformations. The scientific novelty of the method proposed in this paper consists in the synergistic combination of the following components: a term rewriting system with constraints tailored to the specific nature of matrix transformations; mutation analysis based on prepared programs with injected mutations used to evaluate the quality of students’ testing functions; interpretation of expression trees with both full and reduced precision of matrix operations to generate cross-platform test suites. The application of the proposed method in the educational process freed programming instructors from the burden of manually checking 12269 programs submitted by students over the course of a single semester. Incorporating mutation analysis as a distinct stage of the automated assessment pipeline has improved the robustness of exercises against solutions generated by large language models using the zero-shot approach, achieving an 80% resistance rate. Furthermore, it increased the absolute rejection rate of programs by 33.5% when compared to the automated assessment algorithm operating without mutation analysis. As a result of students’ work on enhancing the quality of their testing functions in initially rejected programs, the average number of assert statement executions in accepted programs increased by a factor of 2.24. The time required to generate unique exercises for over 1400 students using a software tool implemented in Python does not exceed several minutes, every exercise consisted of 5 tests for automated assessment and 3 modified programs for mutation analysis. The average time consumed for checking and mutation analysis of a single program within the sandbox environment, implemented as an isolated Docker container, is 8.4 seconds. The obtained results can be used in academic courses, in open online courses, and in programming competitions.