The redundancy of a multichannel data transmission system increases its reliability. During the operation of the system, it is necessary to diagnose and switch failed channels. To solve this problem, the set of input signals of the system is considered as a vector signal, whose scalar components are the coordinates of the vector in a given dimensional space. The diagnosis is performed using a scalar criterion, whose relative simplicity is ensured by the linearity of the signal transformations applied. To minimize the total probability of diagnostic error, the task of optimizing the tolerance on the diagnostic parameter is solved. The possibility of technical implementation of the proposed method is shown based on matrix transformations of the system's input and output signals. The system efficiency was assessed according to the "reliability-cost" criterion. Scientific novelty of the work consists in the fact that analytical expressions for matrix transformations of input and output vector signals of a multichannel data transmission system have been developed. Realization of these transformations provides diagnostics of the system according to the developed scalar criterion both in the test mode and in the mode of functioning as intended.