Time delays in system states or control may result into unacceptable system operation or uncertainty in specialised technical systems like aircraft control, plant control, robotics etc. The issue of robustness, controllability, traceability, flexible management, reliability, and safety of such systems with time-delays, has been one of the primary research focuses of the last few decades. In parallel computing, different computing subunits share their tasks to balance loads to increase performance and throughput. In order to do so, subsystems have to communicate among themselves, adding further delay on top of existing system delay. It is possible to maintain performance and stability of the whole system, by designing observer for every subsystem in the system, overseeing the system state and compensating for existing time-delay. This paper reviews present literature to identify a linear time-delay system for load balancing and evaluates the stability and load balancing performance of the system with and without an observer. Stability is analysed in terms of oscillation in the system responses and performance is evaluated as the speed of load-balancing operation.