Nowadays, computer software plays a significant role in all fields of our life. Essentially open-source software provides economic benefits for software companies such that it allows building new software without the need to create it from scratch. Therefore, it is extremely used, and accordingly, open-source software’s quality is a critical issue and one of the top research directions in the literature. In the development cycles of the software, checking the software reliability is an important indicator to release software or not. The deterministic and probabilistic models are the two main categories of models used to assess software reliability. In this paper, we perform a comparative study between eight different software reliability models: two deterministic models, and six probabilistic models based on three different methodologies: perfect debugging, imperfect debugging, and Gompertz distribution. We evaluate the employed models using three versions of a standard open-source dataset which is GNU’s Not Unix Network Object Model Environment projects. We evaluate the employed models using four evaluation criteria: sum of square error, mean square error, R-square, and reliability. The experimental results showed that for the first version of the open-source dataset SRGM-4 based on imperfect debugging methodology achieved the best reliability result, and for the last two versions of the open-source dataset SRGM-6 based on Gompertz distribution methodology achieved the best reliability result in terms of sum of square error, mean square error, and R-square.