Readiness of Medical Libyan Establishments for the Digital Era A Pre-requisite for Quality Assurance and Accreditation

Introduction: Integration of digital technologies is imperative across sectors like healthcare and education. This shift presents opportunities to enhance medical services and transform learning for future healthcare professionals. This study evaluates Libyan medical institutions’ readiness for the digital age, emphasizing its importance for quality assurance and accreditation.

Materials and methods: A quantitative survey was sent to all the medical faculties deans in Libya (Appendix 1 ).

Results: Surveys gathered responses from 47% of deans representing 17 medical faculties in Libya. Insights emerged on digital transformation dimensions. Positive aspects included an administrative commitment to digital transformation, the pedagogical shift towards student-centered learning, and teacher training support. Challenges included digital infrastructure, resource availability, budgetary constraints, and limited student access to digital tools, emphasizing the need for equitable access. Data security and privacy compliance were strong, with room for data quality improvements.

Discussion and conclusion: This study emphasizes leadership awareness among responders regarding digital readiness. Recommendations encompass investments in digital infrastructure, integration of high-quality digital learning materials, and ensuring equitable student access, community engagement, and staying current with emerging technologies are essential. It is mandatory to empower Libyan medical establishments to navigate digital transformation and to create a robust digital ecosystem to enhance patient care.

Introduction

Integration of digital technologies has become imperative for various sectors, including education and healthcare. The digital transformation has the potential to enhance the quality and efficiency of medical services while providing new learning opportunities for aspiring healthcare professionals [1,2].

The merging of the internet and a commitment to sharing intellectual property has ignited a global movement aimed at opening up knowledge and Educational Resources (OER) for everyone. The OER not only contributes to leveling global access to knowledge but also holds the potential to revolutionize educational practices [3]. Various resources play a pivotal role in supporting learning and educational methods.

In this paper, we explore the readiness of medical establishments in Libya for the digital era and its significance for quality assurance and accreditation. Our primary objectives for this study are to assess the readiness of medical establishments in Libya for the digital era and to identify the specific areas to improve for a smooth transition into the digital age.

As per the 2020 World Economic Forum’s Future of Jobs report, over 80% of global companies have expedited their digitalization efforts in response to the COVID-19 pandemic. The need for digital transformation is expected to become an even more prominent requirement for organizations across diverse industries shortly [4].

Digital transformation encompasses two dimensions. Firstly, it involves a cultural shift that begins at the organizational level and permeates to the individual level. Secondly, it necessitates a tangible technological change, entailing an infrastructure upgrade and the use of new technologies in processes and operations. Nonetheless, the true impact of digital transformation is rooted in the individuals who implement it [5].

Materials and methods

The assessment of digital age readiness involves the appraisal of both institutes and employees. This paper is concerned with the study of the digital readiness of medical institutes in Libya.

A questionnaire was prepared to assess the institution’s digital readiness. The survey was sent on 6 September 2023 digitally to the deans of the 17 public medical faculties (Benghazi, Tripoli, Misurata, Sabha, Sirte, Elmergib, AL Asmarya, Derna, Gharyan, Ajdabiya, Al Zintan, AL Zawiya, Omar Al Mukhtar, Tobruk, Sabratha, Azzaytuna, and Aljafra) in Libya, and feedback was collected over 3 weeks. The survey evaluated various dimensions of digital transformation within Libyan medical establishments, including infrastructure and resources, digital learning content, teacher training and development, student access and equity, data security and privacy, pedagogical approach, administrative support, assessment and evaluation, and community engagement. We calculated the proportions (percentage) among the responding Faculties.

Results

In this section, we provide the key findings from our study, shedding light on the strengths and weaknesses of the digital readiness landscape within Libyan medical establishments. We also highlight any significant challenges and opportunities identified during our research.

These findings could assist in optimizing the advantages of digitizing medical education and enable the formulation of a digital transformation strategy. This strategy would encompass the development of a competency framework, evaluating the workforce using Mercer | Mettl’s Digital Readiness Assessment (DRA) tool and framework, known as ‘An Inside-Out Approach,’ which consists of two parts. The first part involves the assessment of digital potential, evaluating the behavioral competencies and cognitive abilities of employees to adapt and manage the digital transformation process. The second part involves an assessment of digital proficiency, measuring how effectively an individual can manage and protect digital data and information, collaborate through digital technologies, ascertain the digital readiness level of each employee, and help initiate the training process [5]. The digital readiness assessment is a well-structured evaluation

Designed to gauge employees’ essential digital skills, both on an individual and interpersonal level, as well as their capacity for change management and innovation [6].

The responder faculties are; AL Asmarya, Derna, Gharyan, Ajdabiya, Al Zintan, AL Zawiya, Omar Al Mukhtar, and Al Marj (Figures 1-12).

Figure 1: Percentage of Responering Faculties: The responders were 8 out of 17 (47) % of medical faculties in Libya.

Figure 2: Infrastructure and Resources. In this figure, we evaluated the availability of infrastructure. Sixty-two percent (62%) provide high-speed internet access throughout their premises, and seventy-five % (75%) have sufficient and up-to-date hardware devices like computers, tablets, and interactive whiteboards for teaching and learning. Seventy-five % (75%) offer adequate software and digital learning resources for educators and students, and 62% have dedicated IT support and technical staff for maintenance and troubleshooting.

Figure 3: Digital Learning Content. Around ninety percent (87.5%) of faculties provide digital learning materials. However, only 50% integrate multimedia and interactive content. The most inconvenient shortcoming is the low percentage (12.5) of institutions providing access to Learning Management Systems (LMS).

Figure 4A,B: Teacher Training and Development. Seventy-five percent of faculties provide ongoing professional development (A) and support for teachers to integrate digital tools (B).

Figure 5A,B: Assessment of Student access and equity. This figure showed that only 12.5% of faculties provide resources and support for students with varying technology (A), while only 50 % have access to digital devices and internet (B).

Figure 6: Data Security and Privacy. The faculties have 100 % compliance with data privacy regulations and guidelines, and 87.5 % implement robust data security.

Figure 7: Pedagogical Approach. Three-quarters of faculties (75 %) encourage technology pedagogical shifts towards active and student-centered learning and promote technology for collaborative and project-based learning.

Figure 8: Administrative Support. All faculties have a (100%) Commitment from leadership to drive digital transformation. Less than 40% of faculties allocate budgets and resources for technology initiatives. Seventy-five (75%), have policies and procedures for the adoption and use of technology.

Figure 9: Assessment and Evaluation. A quarter of medical faculties (25%) developed methods to assess the impact of technology on teaching and learning, and 62% actively sought feedback from students, teachers and parents.

Figure 10: Community Engagement. (A): One Institution among responders (14%) involves parents and the broader community in the digital transformation process, and 75% have effective communication regarding the benefits and progress of digital initiatives (B).

Figure 11: Future Proofing. A quarter of faculties (75%) stay up-to-date with emerging technologies and trends in education and actively prepare for future changes and disruptions in the digital landscape.

Figure 12: Gap in actual technology provision. 12.5% of responding faculties reported offering technological support to students.

Despite initiatives to help teachers integrate digital tools and promote technology for collaborative learning, only 12.5% of responding faculties reported offering support for students with diverse technology needs. This reveals a notable resource gap for students, underscoring the necessity for an assessment of the underlying factors to enable increased investment in student technology support to promote a smooth transition to the digital era.

Discussion

The presented findings highlight the progress and challenges in Libya as it strives to embrace the digital era. These insights are critical for understanding the current state of digital readiness and charting a path toward more effective digital integration in healthcare and medical education.

One notable observation is the relatively low response rate from medical faculties, with only 47% of deans participating in the survey. This relatively low response rate may impact the representativeness of the findings and restrict the generalizability of the results. This response rate raises concerns about the level of engagement and awareness regarding digital readiness among medical leadership in Libya. To strengthen future research, it is recommended to implement strategies designed to enhance participation. Possible approaches may include targeted outreach to non-respondents, and ensuring that the survey process is both streamlined and user-friendly by addressing these limitations and adopting these strategies, subsequent studies can achieve higher response rates and provide more comprehensive insights.

Future efforts should aim to increase participation to obtain a more comprehensive perspective.

The study revealed that most faculties provide high-speed internet access, up-to-date hardware devices, and digital resources. It’s a positive indication of the efforts made in terms of infrastructure.

E-learning in medical education as a tool to achieve educational goals. Achieving the full potential of e-learning depends on the readiness of the medical institutes, both in terms of human resources and infrastructure, which isn’t always present in Low - and Middle - Income Countries (LMIC) [7]. This finding is consistent with ours, as the use of LMS is about 10 % of Libyan Faculties. However, there is room for improvement in providing access to LMS, which plays a crucial role in digital education by wise resource allocation to bridge this gap.

While most faculties offer digital learning materials, there is a need for greater integration of multimedia and interactive content. Interactive learning can significantly enhance engagement and comprehension. The low percentage of institutions providing access to LMS is a significant concern, as these platforms can centralize and streamline digital education efforts.

Expanding the role of digital inclusion policies and ensuring equitable access for students is essential for enabling all learners, regardless of socio-economic background, to thrive in a digital environment. These policies help bridge the digital divide by providing access to high-speed internet, devices, and digital literacy training, particularly in underserved communities. They also support the development of targeted assistance for students facing technological challenges.

To enhance the use of digital resources, institutions must invest in robust IT infrastructure, ensuring reliable internet access and device availability. Ongoing training for educators in integrating digital resources into their teaching can improve effectiveness and confidence. Additionally, incorporating interactive online content into the curriculum and engaging parents through digital literacy initiatives fosters a supportive learning environment.

Implementing blended learning models offers students varied experiences and flexibility. Establishing metrics to evaluate the effectiveness of these policies and collaborating with technology providers can further enhance access and support. By prioritizing digital inclusion and these strategies, educational institutions can create equitable environments that leverage digital resources effectively, ultimately improving educational outcomes for all students.

The positive finding that 75% of faculties provide ongoing professional development and support for teachers in integrating digital tools is commendable. It highlights a proactive approach to ensure educators are equipped with the necessary skills for effective digital teaching. In the context of healthcare digitalization, essential competencies encompass a profound grasp of digital technology and the requisite skills for providing top-notch patient care. It includes interpersonal communication and ethical considerations related to digitalization’s impact on patient care. Healthcare professionals must also display motivation and a willingness to gain practical experience with digital tools in their specific roles.

The support of colleagues and the institute plays a crucial role in shaping positive digitalization experiences for healthcare professionals [8].

The sustainability of the digital transformation journey relies on providing ongoing support and training to our educators. The e-learning approach, utilizing web-based spaced education for ongoing professional development, has demonstrated its significant benefits. This innovative curriculum incorporated principles like spaced learning, test-enhanced learning, and gamification theory, providing educators with flexible and engaging avenues for professional advancement [9].

The limited access to digital devices and the internet for students raises equity concerns. A thorough exploration of the principles underpinning the formation of an inclusive student community and the challenges encountered in implementing these initiatives are needed. A strategy that prioritizes continuous quality improvement, guided by core values such as social justice and constructive dialogue, fosters inclusion and elevates the quality of medical education. Addressing this issue is paramount to ensure that all students have equal opportunities to benefit from digital education. Strategies for providing access to technology and connectivity should be a priority [10].

This study showed high compliance with data privacy regulations and robust data security implementation. The primary obstacle revolves around enhancing data quality, providing AI models with contextual information, and enforcing privacy, security, and ethics.

Robust eHealth ecosystems, well-prepared stakeholders, standardized data methodologies, and sustained investment in infrastructure are also necessary for overcoming these obstacles and advancing healthcare technology [11]. Implementing a Bring-Your-Own-Device (BYOD) policy for personal mobile devices can save costs for medical schools and healthcare institutions by reducing the need for institution-owned devices. This policy aims to balance user convenience with security requirements [12].

The emphasis on using technology to encourage student-centered learning and collaborative approaches is a promising sign of the evolving pedagogical landscape.

Ejaz et al studied the state of artificial intelligence in medical education in 48 countries. The study suggests including AI education in the global medical student curriculum, emphasizing AI’s relevance in healthcare, especially in clinical medicine. It also encourages involving students in algorithm development. In LMIC, providing technology access and robust AI education is vital to enable healthcare innovation within these regions [13].

The move towards student-centered education with Information Communication Technology (ICT) integration is apparent, highlighted by a survey across three Irish universities involving medical faculty and students. Internet Skills Scale assessed five internet-related skills, garnering responses from 78 faculty members with a 45% response rate and 401 students with a 15% response rate. Of note, the response rate is comparable to ours! This emphasizes the need for faculty skills training, support, and enhancing creative abilities for educators and students in online and distance learning contexts, thus enhancing student-centered learning [14].

Active learning methods and collaborative projects can enhance the educational experience and better prepare future healthcare professionals.

Wever et al, study shows that a unique method for creating and distributing educational materials is viable in Africa. It involves medical students collaborating on an affordable orthopedic video project for their peers, resulting in substantial viewership and watch time on YouTube. Notably, this content is accessible to audiences of different income levels, spanning low, middle, and high-income countries. Over three years, the students’ educational videos consistently reached a worldwide audience [15].

Empathy is vital in patient-centered healthcare, benefiting patients and healthcare providers, but training healthcare trainees in clinical empathy lacks consensus. Research demonstrated that blended module design, which integrates a Massive Open Online Course (MOOC) with virtual learning, is promising and beneficial for the future and effectively improves empathy in dental undergraduate students [16].

In acknowledging the role of leadership, Health Data Research UK collaborated with Imperial and Edinburgh to initiate phase 2 of a program focused on leadership in digital health. The goal is to nurture a new generation of leaders capable of driving the NHS transformation through digitalization. This initiative provides participants with the necessary skills and expertise to lead change, enabling healthcare systems to leverage the many benefits and innovations of modern technology for patient care and organizational improvements [17].

To contextualize the findings on digital readiness in Libya, it is beneficial to compare them with studies from other developing countries that have faced similar challenges in digital integration within healthcare and education. For instance, a study in Kenya shows that despite infrastructural hurdles, investments in mobile health (mHealth) solutions have significantly improved patient outreach and healthcare delivery [18]. This focus on community engagement and local capacity building provides valuable lessons for Libya. Similarly, research from India highlights the effectiveness of telemedicine in expanding healthcare access in underserved regions, demonstrating how existing technologies can facilitate remote consultations and enhance healthcare outcomes [19].

In Bangladesh, the introduction of digital learning platforms has improved access to quality education, aided by collaborations with NGOs to provide training and resources [20]. This collaborative model could inform Libya’s efforts to boost digital literacy among educators and students. Nigeria’s experience with digital transformation in healthcare underscores the importance of policy frameworks and government support in promoting digital initiatives, with various policies implemented to increase access to health information and technology [21].

These comparative studies reveal that while Libya faces unique challenges, successful models from other countries can inform its digital integration efforts. The insights highlight the need for collaboration, community involvement, and governmental support to create a more effective digital landscape in healthcare and education, which will be crucial as Libya moves toward embracing the digital era.

While our results showed a strong commitment from leadership to drive digital transformation, the allocation of budgets and resources for technology initiatives remains a challenge. It is of utmost importance to secure the essential financial backing required to maintain digital progress in both medical education and healthcare. The research indicates the potential for enhancing the evaluation of technology’s influence on the teaching and learning processes. Consistent input from students, educators, and parents is indispensable for informed decision-making and ongoing enhancement.

Involving parents and the broader community in the digital transformation process can foster support and awareness.

The involvement of all stakeholders before a decision or change is primordial. Medical staff prioritize high accuracy in deep learning models, preferring specificity levels exceeding 90%, which significantly reduces overdiagnosis risk compared to FDA standards. However, residents prioritize doctor involvement in screening over model accuracy.

Medical staff favors AI technology when supervised by doctors, both semi and fully-automated models, while residents disapprove of unsupervised AI. This can emphasize the significance of apprenticeship and the transfer of accumulated implicit human knowledge and skills.

It’s recommended to use deep learning models under doctor supervision with high specificity, aiming to reduce repetitive tasks and enhance communication with residents through digital transformation [18].

The emphasis on staying up-to-date with emerging technologies and trends is commendable. Preparing for future changes and disruptions in the digital landscape is essential to ensure that medical establishments remain competitive and responsive to evolving healthcare needs. However, an examination of the state of e-government among local authorities in Libya revealed that the initiatives are insufficient with skill and funding shortages, limited organizational capabilities, lack of management support, and process inefficiencies. The transition to a digital government framework that leverages advanced technologies—such as artificial intelligence, big data, analytics, and cloud computing—remains out of reach [19].

Recommendations

Our study highlights key priorities for Libyan medical establishments, providing a comprehensive roadmap for digital transformation. To enhance the impact of these recommendations, we suggest specific strategies supported by relevant case studies from similar contexts. First, investing in robust IT infrastructure, including high-speed internet and necessary hardware, is crucial for ensuring universal access to Learning Management Systems (LMS). Additionally, developing high-quality digital learning materials tailored to local healthcare needs is essential; the University of Cape Town’s integration of digital tools into its medical curriculum has led to improved student engagement and learning outcomes [20,21].

Moreover, establishing comprehensive training programs focused on digital literacy and innovative teaching methods for educators and healthcare professionals will be vital. The digital training initiative at the University of Ghana significantly boosted faculty confidence and competence in using digital tools. Encouraging blended learning approaches, which combine online and in-person education, can further enhance educational experiences [22,23]; for example, Australia’s shift to blended learning in medical education has resulted in increased student satisfaction and better preparation for clinical settings [24].

Robust data security and privacy measures are also necessary. Implementing strict data governance policies and providing regular training on data protection, as seen with the NHS in the UK, can significantly reduce data breaches and safeguard sensitive information. Finally, creating an ongoing feedback loop involving students and faculty will enable continuous evaluation and assessment of digital initiatives, similar to the methods employed by the University of Toronto, ensuring programs remain relevant and effective.

By embracing these recommendations, Libyan medical establishments can embark on a successful digital transformation that leads to improved healthcare outcomes and the accreditation of medical programs, ultimately benefiting patients and the broader healthcare sector. Continuous adaptation and innovation in response to evolving technology and healthcare demands will be crucial for long-term success.

Conclusion

Our paper highlights the critical need to address obstacles to the digital transformation of medical establishments in Libya. The potential benefits of a robust digital ecosystem in healthcare and education, including improved patient care and enhanced medical education, are underscored. Prioritizing digital readiness is essential for fully realizing these advantages. Additionally, conducting a follow-up qualitative survey can further elucidate the strengths and weaknesses of Libyan medical institutions’ readiness for the digital era. This study offers a comprehensive assessment of the current state of readiness, acknowledging progress while identifying areas requiring additional attention and investment. Implementing the proposed strategies can empower Libyan medical establishments to overcome challenges and embrace the opportunities afforded by the digital age, ultimately contributing to better patient care and the accreditation of medical programs.

Ethical considerations

We with this declare that ethical considerations have been duly addressed in this study. Informed consent was obtained from all participants before their involvement in the research. The data were collected through the deans of the faculties, ensuring proper oversight and compliance with institutional protocols.

Appendix

Although AI-generated tools were used to generate this eBook/ Article, the concepts and central ideas it contains were entirely original and devised by a human writer. The AI merely assisted in the writing process, but the creative vision and intellectual property belong to the human author.

1. Iyamu I, Xu AXT, Gómez-Ramírez O, Ablona A, Chang HJ, Mckee G, et al. Defining digital public health and the role of digitization, digitalization, and digital transformation: scoping review. JMIR Public Health Surveill. 2021;7(11):e30399. Available from: https://doi.org/10.2196/30399
2. Li JO, Liu H, Ting DSJ, Jeon S, Chan RVP, Kim JE, et al. Digital technology, telemedicine and artificial intelligence in ophthalmology: a global perspective. Prog Retin Eye Res. 2021;82:100900. Available from: https://doi.org/10.1016/j.preteyeres.2020.100900
3. Smith MS. Opening education. Science. 2009;323(5910):89-93. Available from: http://dx.doi.org/10.1126/science.1168018
4. World Economic Forum. The future of jobs report 2020. Geneva: World Economic Forum; 2020. Available from: https://www3.weforum.org/docs/WEF_Future_of_Jobs_2020.pdf
5. Digital readiness assessment: the first step toward a future-ready workforce [Internet]. Mettl Blog. Available from: https://blog.mettl.com/digital-readiness/
6. Mettl digital readiness assessment to gauge the digital readiness of your workforce [Internet]. Mettl. Available from: https://mettl.com/test/digital-readiness-assessment/
7. Frehywot S, Vovides Y, Talib Z, Mikhail N, Ross H, Wohltjen H, et al. E-learning in medical education in resource-constrained low- and middle-income countries. Hum Resour Health. 2013;11:4. Available from: https://doi.org/10.1186/1478-4491-11-4
8. Konttila J, Siira H, Kyngäs H, Lahtinen M, Elo S, Kääriäinen M, Kaakinen P, Oikarinen A, Yamakawa M, Fukui S, Utsumi M, Higami Y, Higuchi A, Mikkonen K. Healthcare professionals' competence in digitalization: a systematic review. J Clin Nurs. 2019 Mar;28(5-6):745-761. Available from: https://doi.org/10.1111/jocn.14710
9. Hirsch AW, Nagler J. Spaced education through e-learning for ongoing professional development. AEM Educ Train. 2020;4(4):423-427. Available from: https://doi.org/10.1002/aet2.10446
10. Woodruff JN, Vela MB, Zayyad Z, Johnson TA, Kyalwazi B, Amegashie C, et al. Supporting inclusive learning environments and professional development in medical education through an identity and inclusion initiative. Acad Med. 2020;95(12S Addressing Harmful Bias and Eliminating Discrimination in Health Professions Learning Environments): S51-S57. Available from: https://doi.org/10.1097/acm.0000000000003689
11. López DM, Rico-Olarte C, Blobel B, Hullin C. Challenges and solutions for transforming health ecosystems in low- and middle-income countries through artificial intelligence. Front Med (Lausanne). 2022;9:958097. Available from: https://doi.org/10.3389/fmed.2022.958097
12. Kadimo K, Kebaetse MB, Ketshogileng D, Seru LE, Sebina KB, Kovarik C, et al. Bring-your-own-device in medical schools and healthcare facilities: a review of the literature. Int J Med Inform. 2018;119:94-102. Available from: http://dx.doi.org/10.2196/preprints.10764
13. Ejaz H, McGrath H, Wong BL, Guise A, Vercauteren T, Shapey J. Artificial intelligence and medical education: a global mixed-methods study of medical students' perspectives. Digit Health. 2022;8:20552076221089099. Available from: https://doi.org/10.1177/20552076221089099
14. O'Doherty D, Lougheed J, Hannigan A, Last J, Dromey M, O'Tuathaigh C, et al. Internet skills of medical faculty and students: is there a difference? BMC Med Educ. 2019;19(1):39. Available from: https://doi.org/10.1186/s12909-019-1475-4
15. Wever GS, Elliott I, McCaul J, Workman M, Laubscher M, Dunn RN, et al. Viewership footprint for a low-resource, student-centered collaborative video platform to teach orthopedics in southern Africa. S Afr Med J. 2020;110(6):532-536. Available from: https://doi.org/10.7196/samj.2020.v110i6.14348
16. Anishchuk S, Kubacki A, Howell Y, van Harten MT, Yarascavitch C, MacGiolla Phadraig C. Can a virtual learning module foster empathy in dental undergraduate students? Eur J Dent Educ. 2023;27(1):118-125. Available from: https://doi.org/10.1111/eje.12783
17. Inglis Humphrey W, Smithies A, Prestt J, Fletcher M, Cresswell K; NHS Digital Academy Digital Health Leadership Programme. The NHS Digital Academy Digital Health Leadership Programme. Stud Health Technol Inform. 2022 Aug 31;298:24-28. Available from: https://doi.org/10.3233/shti220901
18. Lin S, Li L, Zou H, Xu Y, Lu L. Medical staff, and resident preferences for using deep learning in eye disease screening: discrete choice experiment. J Med Internet Res. 2022;24(9):e40249. Available from: https://doi.org/10.2196/40249
19. Wynn M, Bakeer A, Forti Y. E-government and digital transformation in Libyan local authorities. Int J Teach Case Stud. 2021;12(2):119-139. Available from: http://dx.doi.org/10.1504/IJTCS.2021.116139
20. Gellisch M, Morosan-Puopolo G, Brand-Saberi B, Schäfer T. Adapting to new challenges in medical education: a three-step digitization approach for blended learning. BMC Med Educ. 2024;24:585. Available from: https://doi.org/10.1186/s12909-024-05503-1
21. Harle J, Lamptey RB, Mwangi A, Nzegwu F, Okere O. International Network for Advancing Science and Policy (INASP) Creating digital content and delivering digital learning in African universities: Stories of innovation from Ghana, Kenya, and Nigeria. 2021. Available from: https://www.inasp.info/sites/default/files/2021-10/INASP%20Digital%20Universities%20final%20report%202021-10-13.pdf
22. Boateng JK. Managing learning outcomes with technology in Ghanaian higher education. Cogent Soc Sci. 2023;9(2). appendAvailable from: https://doi.org/10.1080/23311886.2023.2282507
23. Singh J, Evans E, Reed A, Karch L, Qualey K, Singh L, Wiersma H. Online, hybrid, and face-to-face learning through the eyes of faculty, students, administrators, and instructional designers: lessons learned and directions for the post-vaccine and post-pandemic/COVID-19 world. J Educ Technol Syst. 2022;50(3):301-326. Available from: https://doi.org/10.1177/00472395211063754
24. Venkatesh S, Rao YK, Nagaraja H, Woolley T, Alele FO, Malau-Aduli BS. Factors influencing medical students' experiences and satisfaction with blended integrated e-learning. Med Princ Pract. 2020;29(4):396-402. Available from: https://doi.org/10.1159/000505210