International Journal of Modern Education and Computer Science (IJMECS)
IJMECS Vol. 14, No. 2, 8 Apr. 2022
Cover page and Table of Contents: PDF (size: 561KB)
The Evaluation of the Academic Progress Information System SIsKA-NG Mobile Based on Heuristic and User Experience
Full Text (PDF, 561KB), PP.55-64
Views: 0 Downloads: 0
Author(s)
I Made Subrata Sandhiyasa
1,*
I Gede Aris Gunadi
2
Gede Indrawan
2
Index Terms
Evaluation, concurrent think aloud, heuristic, mobile application, system usability scale, user experience
Abstract
The limitations on the Academic Progress Information System (SIsKA) application, in terms of its interface and functionality, have a direct consequence on the quality of service and monitoring student progress in the Computer Science Department of Universitas Pendidikan Ganesha. Therefore, it is necessary to overcome these limitations through research that focuses on the development and evaluation of the next generation of SIsKA (SIsKA-NG) mobile application with real-time notification advantage compared to its counterpart existing web application. The heuristic evaluation of the first stage of this mobile application development identified twenty problems where eight problems with the highest severity (Catastrophic), six problems with high priority (Major), three problems with low priority (Minor) that need to be solved, and three problems that do not need to be fixed unless there is extra time. User satisfaction was measured by using the System Usability Scale (SUS) questionnaire and gave the final score of 82.25. That score was higher than the minimal SUS standard score of 68.00 and was graded as very good, which means that users are very satisfied with using the application. To maximize more of the SIsKA-NG mobile performance, usability testing was conducted to obtain qualitative data from the same SUS respondents by using the Concurrent Think Aloud (CTA) method. CTA gave a summary result of twenty-one criticisms or suggestions that were referred to by the second stage of the SIsKA-NG mobile development.
Cite This Paper
I Made Subrata Sandhiyasa, I Gede Aris Gunadi and Gede Indrawan, " The Evaluation of the Academic Progress Information System SIsKA-NG Mobile Based on Heuristic and User Experience", International Journal of Modern Education and Computer Science(IJMECS), Vol.14, No.2, pp. 55-64, 2022.DOI: 10.5815/ijmecs.2022.02.05
Reference
[1] A. A. I. I. Paramitha, G. R. Dantes, and G. Indrawan, “The evaluation of web based academic progress information system using heuristic evaluation and user experience questionnaire (UEQ),” in Proceedings of the 3rd International Conference on Informatics and Computing, ICIC 2018, 2018, doi: 10.1109/IAC.2018.8780430.
[2] G. Indrawan et al., “SIsKA: Mobile Based Academic Progress Information System,” 2017, doi: 10.2991/icirad-17.2017.24.
[3] G. Indrawan, I. G. A. Gunadi, and I M. S. Sandhiyasa, “REST API and Real-Time Notification of SIsKA-NG Mobile for the Academic Progress Information System,” in The Fifth International Conference on Information and Communication Technology for Competitive Strategies, ICTCS, 2020.
[4] S. E. Motlagh Tehrani, N. M. M. Zainuddin, and T. Takavar, “Heuristic evaluation for Virtual Museum on smartphone,” Proc. - 2014 3rd Int. Conf. User Sci. Eng. Exp. Eng. Engag. i-USEr 2014, no. Vm, pp. 227–231, 2015, doi: 10.1109/IUSER.2014.7002707.
[5] I. Almarashdeh and M. Alsmadi, “Heuristic evaluation of mobile government portal services: An experts’ review,” in 2016 11th International Conference for Internet Technology and Secured Transactions, ICITST 2016, 2017, doi: 10.1109/ICITST.2016.7856746.
[6] G. W. Sasmito, L. O. M. Zulfiqar, and M. Nishom, “Usability Testing based on System Usability Scale and Net Promoter Score,” in 2019 2nd International Seminar on Research of Information Technology and Intelligent Systems, ISRITI 2019, 2019, doi: 10.1109/ISRITI48646.2019.9034666.
[7] J. R. Lewis, “The System Usability Scale: Past, Present, and Future,” Int. J. Hum. Comput. Interact., 2018, doi: 10.1080/10447318.2018.1455307.
[8] S. Gupta, “A Comparative study of Usability Evaluation Methods,” Int. J. Comput. Trends Technol., 2015, doi: 10.14445/22312803/ijctt-v22p121.
[9] R. S. Pradini, R. Kriswibowo, and F. Ramdani, “Usability Evaluation on the SIPR Website Uses the System Usability Scale and Net Promoter Score,” in Proceedings of 2019 4th International Conference on Sustainable Information Engineering and Technology, SIET 2019, 2019, doi: 10.1109/SIET48054.2019.8986098.
[10] A. C. Wijaya, M. Wiranto Aris Munandar, and F. Utaminingrum, “Usability Testing of Augmented Reality for Food Advertisement Based on Mobile Phone Using System Usability Scale,” in Proceedings of 2019 4th International Conference on Sustainable Information Engineering and Technology, SIET 2019, 2019, doi: 10.1109/SIET48054.2019.8986118.
[11] C. Katsanos, N. Tselios, and M. Xenos, “Perceived usability evaluation of learning management systems: A first step towards standardization of the system usability scale in Greek,” in Proceedings of the 2012 16th Panhellenic Conference on Informatics, PCI 2012, 2012, doi: 10.1109/PCi.2012.38.
[12] K. A. Ericsson and H. A. Simon, Protocol analysis: Verbal reports as data (rev. ed.). 1993.
[13] O. Alhadreti and P. Mayhew, “Are Two Pairs of Eyes Better Than One? A Comparison of Concurrent Think-Aloud and Co-Participation Methods in Usability Testing,” J. Usability Stud., 2018.
[14] M. J. Van den Haak and M. D. T. De Jong, “Exploring Two Methods of Usability Testing: Concurrent versus Retrospective Think-Aloud Protocols,” in IEEE International Professional Communication Conference, 2003, doi: 10.1109/ipcc.2003.1245501.
[15] S. McDonald, H. M. Edwards, and T. Zhao, “Exploring think-alouds in usability testing: An international survey,” IEEE Trans. Prof. Commun., 2012, doi: 10.1109/TPC.2011.2182569.
[16] L. Nielsen and S. Madsen, “The usability expert’s fear of agility - An empirical study of global trends and emerging practices,” in NordiCHI 2012: Making Sense Through Design - Proceedings of the 7th Nordic Conference on Human-Computer Interaction, 2012, doi: 10.1145/2399016.2399057.
[17] W. O. Galitz, “The essential guide to user interface design,” SIGCHI Bull., 1997.
[18] G. Indrawan, I. M. A. O. Gunawan, and Sariyasa, “The usability evaluation of academic progress information system (SISKA-NG),” Adv. Sci. Technol. Eng. Syst., 2020, doi: 10.25046/aj050259.
[19] B. A. Mustikaningtyas, M. C. Saputra, and A. Pinandito, “Analisis Usability Pada Website Universitas Brawijaya Dengan Heuristic Evaluation [Usability Analysis on Brawijaya University Website with Heuristic Evaluation],” J. Teknol. Inf. dan Ilmu Komput., 2016, doi: 10.25126/jtiik.201633194.
[20] B. Yilmaz and P. O. Durdu, “Heuristic Evaluation of a mobile hand-writing learning application,” in 9th International Conference on Application of Information and Communication Technologies, AICT 2015 - Proceedings, 2015, doi: 10.1109/ICAICT.2015.7338621.
[21] N. Borovina, D. Bošković, J. Dizdarević, K. Bulja, and A. Salihbegović, “Heuristic based evaluation of Mobile Services web portal usability,” in 2014 22nd Telecommunications Forum, TELFOR 2014 - Proceedings of Papers, 2015, doi: 10.1109/TELFOR.2014.7034611.
[22] J. Nielsen and R. Mack, Heuristic Evaluation. 1994.
[23] S. Elling, L. Lentz, and M. De Jong, “Combining concurrent think-aloud protocols and eye-tracking observations: An analysis of verbalizations and silences,” IEEE Trans. Prof. Commun., 2012, doi: 10.1109/TPC.2012.2206190.
[24] J. Brooke, “SUS: a retrospective,” J. Usability Stud., 2013.
[25] Z. Sharfina and H. B. Santoso, “An Indonesian adaptation of the System Usability Scale (SUS),” in 2016 International Conference on Advanced Computer Science and Information Systems, ICACSIS 2016, 2017, doi: 10.1109/ICACSIS.2016.7872776.
[26] A. Belkhir, M. Abdellatif, R. Tighilt, N. Moha, Y. G. Gueheneuc, and E. Beaudry, “An observational study on the state of REST API uses in android mobile applications,” in Proceedings - 2019 IEEE/ACM 6th International Conference on Mobile Software Engineering and Systems, MOBILESoft 2019, 2019, doi: 10.1109/MOBILESoft.2019.00020.
[27] A. Madhuram, M, Kumar and M. Pandyamanian, “Cross Platform Development using Flutter,” Int. J. Eng. Sci. Comput., 2019.