Annotation

Annotation: Regular collection of publications from open mass media sources is complicated by the heterogeneity of source interfaces, the volatility of page markup and the need to attest the provenance and immutability of the accumulated corpus. The aim of the study is to design and experimentally verify an application architecture in which adding a source does not affect the program core, the completeness of periodic feed polling is monitored under a variable publication rate, and the state of the collected corpus remains verifiable. A combined architecture is proposed, in which coarse-grained functional modules are complemented by a coordination layer formed by a message broker with durable, at-least-once delivery and a distributed cache of short-lived state. The application combines a versioned declarative source template with reparsing of stored responses, adaptive feed polling based on overlapping publication windows, and verifiable corpus attestation using a Merkle tree and signed checkpoints. These mechanisms form a reproducible sequence linking source responses, extraction rules and normalised records. The experimental implementation was compared with Scrapy at matching process counts. With one to eight workers, the ratio of mean batch throughputs ranged from 1.70 to 2.00. A separate experiment used a shared request-rate limiter to characterise operation under a constrained input flow. Stored HTML pages from two news resources were subjected to a controlled local markup transformation and parsed using revised templates. No additional HTTP requests were sent during reparsing; the non-empty field values obtained with the original template were recovered without differences. Message redelivery occurred in the worker, RabbitMQ and Redis failure tests, but each of the three episodes ended with all unique results stored. With one million records in the tree, the mean inclusion-path size was 640 bytes and verification took 24.50 microseconds. The reported timings were obtained on one node for the stated software versions, using the measurement intervals specified for each experiment.