Systematic Review

Rumor surveillance methods in outbreaks: A systematic literature review

Introduction

In recent years, despite many advances in health and vaccine availability, emerging and re-emerging epidemics threatens community health. Severe acute respiratory syndrome (SARS), influenza A (H1N1), avian flu, Ebola virus, Middle East respiratory, ¹ and recent COVID-19 are among the best examples of such epidemics.

The public and media reactions during an outbreak period are different. When emergencies occur, people seek information more than usual. Invalid and unclear information not controlled by the government and the media may lead to psychological and emotional tensions, public panic, and economic loss within in a community. ^2,3 But, if there are clear and useful information and a valid channel for information dissemination among the government, the media, and the public, the level of social panic may be curbed and the expansion and spread of adverse effects in the event may be prevented. At the same time, a unique information interactive platform can better establish the credibility of the government and the media. ⁴

It has been observed that the spread of misinformation and rumors regarding natural disasters and crises has increased. ⁵ As an instance, Towers et al found that the news media may help spread fear and misinformation during Ebola epidemics in 2014. ⁶ Also, during the SARS pandemics, there were network media information chaos, rumors, and public panic. There was also a high volume of misinformation about overbuying of medical-related products and the soaring prices of these products, which confused the community and caused many problems during disease control and prevention periods. ⁷ Also, harmful rumors negatively affected people’s health during the outbreak of the infectious diseases such as SARS in 2003. ⁸ Previous studies have highly emphasized the role of rumors’ management in critical periods of public health, like the Chernobyl nuclear accident in 1986, and the Ebola outbreak in Uganda in 2011. ^9,10 Although the management and monitoring of such rumors seems to be vital, identifying the quick and accurate sources of rumors, due to its complicated process and dynamic changes is highly challenging. ¹¹ The information diffusion in social sites opens many research trends like detection of misinformation and/or rumor checking, recognition of social bots, monitoring the spread of fake news, and prediction of future diffusion and source detection of rumors, as well. ¹² The results of a recent study on the use of health information in Senegal showed the necessity of investigating research methods for addressing health rumors. ¹³ In an effort to develop such methods, Ma et al proposed two recursive neural models based on bottom-up and top-down tree-structured neural networks for rumor representation learning and classification. ¹⁴

Besides, many efforts have been made to detect social media rumors by analyzing their content and social context using machine learning techniques. ¹⁵ A new study, conducted by Liu and Xu, surveys user features in an online media platform for the detection of rumors. They developed a new information propagation model based on a different user representation and modeling method. Applying the new method, we can differentiate rumors from credible messages by observing distinctions in their respective propagation patterns in social media. ¹⁶ All of these studies, therefore, shed light to a firm conclusion on the role of surveillance as the most efficient and effective way of handling rumors in outbreaks. Although the existing studies on rumor detection and surveillance focus on the theoretical modeling of classification and identification of rumors, there is still a need to understand the realistic methods used in previous outbreaks and epidemics.

There is a scarcity in the research done on the detection of rumors in outbreaks. To our knowledge, this is the first study that attempted to identify methods used for rumor detection in previous epidemics. Therefore, this study aimed to identify the methods and/or tools used for rumor detection and surveillance during outbreaks and to discuss their advantages and disadvantages.

Material and Methods

This study was a systematic review of studies performed on the development of rumor surveillance systems during outbreaks and epidemics in the world.

Information sources

We searched PubMed, EMBASE, Web of sciences, and Scopus databases for relevant papers in English. We searched the databases in November 2020 and included the related articles published from 2000 to 2020.

Strategy search

n searching these databases, we used a mixture of medical subject headings(MeSH) and keywords. Three groups of key terms were used: (A) key terms denoting rumor spreading, including rumor, gossip, misinformation, (B) intervention related terms (e.g. outbreaks OR epidemics), and (C) outcome related terms (surveillance OR detection OR gossip OR misinformation).

We used two different strategies to extract relevant articles from the databases, and the results of the two strategies were then combined. To search the databases, advanced search functions were used as follows: first, we used “OR” to combine terms in each group A, B and C separately; then, we combined results from the two groups using the “AND” operator to accumulate all the studies about the rumor in outbreaks. We also searched for grey literature using the first 200 hits from Google Scholar and examined the references of the included studies to identify other potentially relevant studies. We used identical search terms and combined them in different ways (See Appendix 1).

Eligibility criteria

Only the studies conducted on the problem of rumor spreading in outbreaks were considered for the review. Special emphasis was given to the interventional studies that developed a rumor surveillance system and applied it in an actual outbreak. Non-original and non-English papers, and the articles that focused on proposing and/or modeling a system, and those that evaluated such systems in drill or simulation were excluded from our study.

Quality assessment

The quality of studies was evaluated using the Mixed Methods Appraisal Tool (MMAT) ¹⁷ via answering two questions for all types of study, 5 questions for quantitative descriptive studies, and 5 questions for qualitative studies by two of the authors (See Appendix 2).

Data selection, extraction, and analysis

After searching databases, two reviewers independently screened the articles first based on title and abstract. The full texts of all potentially eligible articles were retrieved and independently assessed for eligibility by two authors. Then, according to the predefined inclusion and exclusion criteria, relevant articles were selected. Disagreements between the reviewers were resolved by a third reviewer. Finally, two reviewers extracted relevant information for each article. They abstracted the characteristics of eligible studies in a Microsoft Excel form. The characteristics were area, type of study, type of outbreaks, sources of rumor, type of surveillance system, data transfer method, and data collection method. Extracted data were summarized narratively and presented in a structured table, which were then analyzed using descriptive statistics.

Results

Study selection

A total of 890 articles were initially retrieved according to the patients/problem, intervention, comparison, and outcome (PICO) search strategy. After removing the duplicates, 615 articles remained for screening. Most of the articles did not meet the inclusion criteria and only five articles were included in the final analysis ().

Close

Figure 1. PRISMA flow chart for systematic literature review.

Study characteristics

Publication dates of the included articles ranged from 2005 to 2018. The literature on this topic began to show its presence in 2005 with an original article aimed to develop and implement a rumor surveillance system for avian influenza. All articles (n=5) were research-based and employed various methodologies. In terms of quality, all five articles were at a good level and met all MMAT criteria (one qualitative and four quantitative descriptive studies).

The studies included in this review had to match the predetermined criteria according to the patients/problem, intervention, comparison, outcome, and study design (PICOS) approach. Criteria for inclusion and exclusion are specified in Table 1.

Geographical focus

Most of the literature has focused on African nations (n=3). One study discussed the use of social media in the countries affected by Zika, without a specific geographical focus. The other one, with a specific geographic focus, addressed the rumor surveillance system in the western pacific region (WPR), including 37 countries around the world.

Types of outbreaks

The use of social media for informing about infectious diseases in outbreak situations was another common thread within this theme. The most specific type of outbreaks that addressed independently in the reviewed studies was influenza (two out of the five articles). One article specifically addressed more than one outbreak. One of these articles focused on the use of a rumor surveillance system within the recent Zika outbreak in 2016. Another article highlighted the development of the Surveillance and Outbreak Response Management System (SORMAS) based on the findings from the Nigeria’s 2014 Ebola outbreak. (Table 2)

Sources of rumor surveillance

A focus on the sources of rumor surveillance was the most prominent theme identified in the literature. One article addressed social media as a source of rumors which specifically focused on the Twitter platform. Two articles mentioned other forms of media including the traditional media such as newspapers, TV, radio, and websites. Detecting rumors from community, health workers, and health facilities was highlighted in two of five studies. (Table 2)

Type of surveillance system

The focus of most rumor surveillance systems was on human-computer interaction methods (n=3). Two papers specifically addressed the event-based surveillance (EBS) system, which was reported to require rapid detection, reporting, confirmation, and assessment of rare and new health events that can affect public health. These systems were integrated into the routine surveillance system. In another study, a software tool named SORMAS was used to support health workers in efficient handling of the infectious diseases’ outbreak situations, such as Ebola.

Another study designated a rumor surveillance officer to develop and implement the rumor surveillance system for avian influenza. This officer actively assessed media sources and email-based public health discussions and regularly contacted the World Health Organization (WHO) network to identify rumors. It was categorized as a manual method. The automatic method was only used in one of the studies. Machine learning technology for tracking health misinformation was used in the 2016 Zika outbreak.

Data transfer method

Our results showed that the data transmission method in three studies was asynchronous, within which data transfer took place over a while. On the contrary, synchronous (Real-time) methods were discussed in two studies. In this method, the time interval of data transmission is constant.

Discussion

Exposure to misinformation and rumors may harm those who receive and believe them. In this study, the major source of rumors was found to be social and traditional media, followed by the rumors come from community and health workers.

As seen in the recent outbreaks of measles, Zika virus, and Ebola, the public is exposed to a large amount of information from both official channels, such as WHO and local authorities, and unofficial channels, such as newspapers and social media. ^23,24 Banakar et al revealed that social media, like WhatsApp, Telegram, and Instagram as well as the national media such as TV and radio were the primary sources of the Covid-19 news for the participants, and print media was less common sources. ²⁵ A reason for such a difference might be that the acquisition of information from social media platforms is more time-saving and less-costly than from the conventional news media such as newspapers or television. Besides, chatting and sharing information is easy on social media. ²⁶

Methods for rumor surveillance

Different methods have been proposed for rumor detection and surveillance in previous outbreaks. According to our results in this review, in two studies, EBS was established along with the existing indicator-based surveillance systems for rumor surveillance. Design and implementation of such systems to retrieve and process the daily stream of misinformation is required for monitoring unofficial sources from the web regarding a wide range of health-related threats. Several EBS systems are currently monitoring unofficial sources on the web regarding a wide range of health threats. ²⁷ The EBS system is a simple-to-use and low-cost strategy that forms a cornerstone of public health surveillance and response, particularly in low-resource countries. Such systems use a combination of humans and computers for rumor and other event surveillance. Rumors and associated information are collected from different sources by health and/or surveillance officers are entered into Microsoft Excel 2013 for timely reporting and response.

As an instance of a rumor surveillance system, developed during the avian influenza outbreak in 2004, an officer actively assessed media sources and email-based public health discussions and contacted the WHO network, in a regular basis, to identify rumors. In this method, each rumor was followed up via an email or a telephone request to the relevant WHO country office to investigate its veracity.

An advantage of such a system is its low establishing cost, but it seems that the reporting time is high, especially in the situations of severe outbreaks when real-time response is critical. In the reviewed studies, the average time for rumor reporting was at a range from 3.8 to 10 days. Timely detection of health rumors can help public health officials in tackling the issue before extensive spreading. Delay in reacting to a rumor, may increase its damage through rapid spreading of its harmful misconceptions. For example, during the Ebola outbreak, a rumor that circulated on the Internet presented “drinking salty water” as an effective protective measure, which led to several deaths in the population. ²⁸ This may also reflect the surveillance of rumors at different levels of the health system. While in two studies ^19,22 rumors surveillance was at the national and international levels, in another study ¹⁸ the surveillance was conducted at the health centers level, which is much closer to the community. So, the reporting time is much shorter than those at the national level.

Machine learning is another method for health-related rumor discovery that was used for social media surveillance. Ghenai et al. building a machine learning tool for tracking the health misinformation on Twitter during Zika outbreak. . ²⁰ Compared to traditional media, social media is harder to be monitored, tracked, and analyzed. Public health institutions such as the WHO introduced social media as a crucial part of monitoring while the surveillance of a health crisis. ²⁹ Due to the nature of the textual data and their fast spreading in social media, designing tools that automatically identify rumors and assess their accuracy increased in recent years. ³⁰ Machine learning methods emerged as key players in rumor detection on online social media.

Qazvinian et al in 2011 proposed a supervised machine learning method for judging the relevance of new tweets to the known set of rumors. Their results showed that particularly known rumors can be retrieved with high accuracy after training a machine learning classifier for each rumor. Due to the ever-increasing amount of multimedia information on social media, many efforts have been made to automatically defeat online rumors. Such a process is conducted through mining the rich content provided on the open network with machine learning techniques, like neural network methods. ³¹ Recently, deep neural networks are proposed to automatically learn and fuse multi-modal features for rumor detection. Compared to the works that leverage the traditional methods, such networks can significantly improve the performance of the systems. ³²

The third method is developing an electronic tool such as a mobile and web-based application that improves data collection, situation assessment, and coordination of response measures in outbreaks. SORMAS is a mobile-based and open-source system that uses a cloud service provider, and is accessible via both personal computers and smartphones. This system is applicable for outbreak management and better routine surveillance of all infectious diseases. A section of this application is a rumor table which contains information about all rumors that may be concerned to one or more cases, while detecting rumors reported by someone, an event, or a place. In some countries, the national public health authorities are preparing to deploy SORMAS in response to the COVID-19 outbreak. ³³

In contrast to SORMAS, there are other mobile-based and web-based software that are exclusively used for monitoring the people who had made contact with Ebola cases and do not address other aspects of outbreak response, like case finding and rumor surveillance. ²¹ For example, many software tools are especially developed to monitor the emerging infectious diseases ^34,35 and their associated rumor detection in social media and online sources. TweetCred, ³⁶ Hoaxy, ³⁷ and SUPER ³⁸ are examples of these type of tools.

SORMAS can improve the processes and help health workers in faster reactions compared to the manual or traditional methods. The advantages of this application are their acceptability, ³⁹ low development costs and lead-in times, and the support that it provides for real-time rumor management, contact-tracing, case management, and surveillance. ⁴⁰ In the study of Yavlinsky et al, several mobile-based outbreak management systems were analyzed, but only SORMAS was satisfying and fulfilled all the assessment criteria. ⁴¹

To sum up, the rumors spreading on social media can severely influence people’s daily life. However, to react to the ever-increasing rumors, a majority of the rumor management methods still depend on manual efforts, including human experts or users. In this circumstance, the research on automatic rumor detection is fascinating. Most of the rumors in social media are multimodal which result in difficulties when they are managed using traditional detection methods. Also, data transmission in the traditional methods is asynchronous and on the basis of store-and-forward systems which allow for multiple component interactions to occur. However, the effect of the time gap may need information refreshing during the process. Also, the synchronic model and the real-time methods are increasingly used in the new methods such as machine learning systems to support immediate interaction or at least response within a short time. ⁴² Cost-saving is another advantage of such methods. ⁴³

This paper gives an overview of rumor surveillance methods. Among all the existing methods, some use human-computer methods for rumor surveillance. Compared to the manual methods, these computerized methods prepare better results with higher efficiency, as they reduce labor costs and their associated findings are more accurate. These characteristics of the computerized methods ensure the real-time availability of the most data for everyone involved in the outbreak containment, and allow public health researchers and health practitioners to respond to the rumors with a targeted and rapid reaction. To our knowledge, this was the first study that systematically identified the methods used for rumor detection used in the outbreaks, and assessed their effectiveness.

Due to the nature of the study, our study had several limitations. The number of articles that applied a system for rumor surveillance during outbreaks and included in our study was low. Despite the increasing research on different methods of rumor detection, there were few studies that used this method in outbreaks. So, we had limitations in comparing the effect of different methods on a real condition. In general, the findings of this study indicated that much more efforts should be made to identify the effectiveness of different rumor surveillance methods during outbreaks.

Conclusions

In this study, we reviewed the methods used for rumor detection in the outbreaks. Our findings provided lessons for health care organizations, health care professionals, researchers, and the general public on how to optimize the rumor detection and surveillance methods in recent COVID-19 outbreak. Due to the ever-increasing use of social media, and considering the broad and rapid deployment of information in wide and fast spread infectious disease like COVID-19, developing the new methods such as mobile applications, neural networks, and machine learning systems that can automatically detect and find the sources of misinformation in a timely manner is pivotal.

Acknowledgements

We are grateful to institute for futures studies in health in Kerman University of Medical Sciences, for their support in conducting this research.

Funding

This work was supported by Vice Chancellor of Research and Technology, Kerman University of Medical Sciences, Kerman, Iran.

Competing interests

The authors have no conflicts of interest to declare.

Ethics approval

This study received ethical approval from the vice chancellor for research of the Kerman University of Medical Sciences (IR.KMU.REC.1399.005).

Authors’ contributions

SS contributed in searching, reviewing the literature, summarizing the results, and writing up the manuscript. PJA also contributed to reviewing the literature and summarizing the results. SS and VB contributed to the design and finalize the manuscript.

Appendix

References

1. Schlipköter U, Flahault A. Communicable diseases: achievements and challenges for public health. Public Health Rev 2010; 32:90-119. doi: 10.1007/bf03391594 [Crossref]
2. Thomas SA. Lies, damn lies, and rumors: an analysis of collective efficacy, rumors, and fear in the wake of Katrina. Sociol Spectr 2007; 27(6):679-703. doi: 10.1080/02732170701534200 [Crossref]
3. Liu D, Chen X. Rumor propagation in online social networks like twitter--a simulation study. In: 2011 Third International Conference on Multimedia Information Networking and Security. Shanghai, China: IEEE; 2011. 10.1109/mines.2011.109
4. Huo L, Cheng Y. The impact of media coverage and emergency strategies on the rumor spreading. Discrete Dyn Nat Soc 2018; 2018:4137129. doi: 10.1155/2018/4137129 [Crossref]
5. Ratkiewicz J, Conover M, Meiss M, Gonçalves B, Patil S, Flammini A, et al. Truthy: mapping the spread of astroturf in microblog streams. In: Proceedings of the 20th International Conference Companion on World Wide Web. New York, NY, USA: Association for Computing Machinery (ACM); 2011. p. 249-52. 10.1145/1963192.1963301
6. Towers S, Afzal S, Bernal G, Bliss N, Brown S, Espinoza B. Mass media and the contagion of fear: the case of Ebola in America. PLoS One 2015; 10(6):e0129179. doi: 10.1371/journal.pone.0129179 [Crossref]
7. Gu H, Chen B, Zhu H, Jiang T, Wang X, Chen L. Importance of Internet surveillance in public health emergency control and prevention: evidence from a digital epidemiologic study during avian influenza A H7N9 outbreaks. J Med Internet Res 2014; 16(1):e20. doi: 10.2196/jmir.2911 [Crossref]
8. Chen F, Cao S, Xin J, Luo X. Ten years after SARS: where was the virus from?. J Thorac Dis 2013; 5(Suppl 2):S163-7. doi: 10.3978/j.issn.2072-1439.2013.06.09 [Crossref]
9. Rahu M. Health effects of the Chernobyl accident: fears, rumours and the truth. Eur J Cancer 2003; 39(3):295-9. doi: 10.1016/s0959-8049(02)00764-5 [Crossref]
10. Okware SI, Omaswa FG, Zaramba S, Opio A, Lutwama JJ, Kamugisha J. An outbreak of Ebola in Uganda. Trop Med Int Health 2002; 7(12):1068-75. doi: 10.1046/j.1365-3156.2002.00944.x [Crossref]
11. Metaxas PT, Finn S, Mustafaraj E. Using twittertrails. com to investigate rumor propagation. In: Proceedings of the 18th ACM Conference Companion on Computer Supported Cooperative Work & Social Computing. New York, NY, USA: Association for Computing Machinery (ACM); 2015. p. 69-72. 10.1145/2685553.2702691
12. Shelke S, Attar V. Source detection of rumor in social network–a review. Online Soc Netw Media 2019; 9:30-42. doi: 10.1016/j.osnem.2018.12.001 [Crossref]
13. Sylla AH, Robinson ET, Raney L, Seck K. Qualitative study of health information needs, flow, and use in Senegal. J Health Commun 2012; 17 (Suppl 2):46-63. doi: 10.1080/10810730.2012.666624 [Crossref]
14. Ma J, Gao W, Wong KF. Rumor detection on twitter with tree-structured recursive neural networks. Melbourne, Australia: Association for Computational Linguistics; 2018.
15. Friggeri A, Adamic LA, Eckles D, Cheng J. Rumor cascades. In: Proceedings of the 8th International Conference on Weblogs and Social Media; June 1–4, 2014; Ann Arbor, Michigan, USA. p. 101-10.
16. Liu Y, Xu S. Detecting rumors through modeling information propagation networks in a social media environment. IEEE Trans Comput Soc Syst 2016; 3(2):46-62. doi: 10.1109/tcss.2016.2612980 [Crossref]
17. Hong QN, Pluye P, Fàbregues S, et al. Mixed Methods Appraisal Tool (MMAT), Version 2018. McGill University; 2018. Available from: http://mixedmethodsappraisaltoolpublic.pbworks.com/w/file/fetch/127425851/MMAT_2018_criteria-manual_2018-04-04.pdf. Accessed August 27, 2020.
18. Toyama Y, Ota M, Beyene BB. Event-based surveillance in north-western Ethiopia: experience and lessons learnt in the field. Western Pac Surveill Response J 2015; 6(3):22-7. doi: 10.5365/wpsar.2015.6.2.002 [Crossref]
19. Samaan G, Patel M, Olowokure B, Roces MC, Oshitani H. Rumor surveillance and avian influenza H5N1. Emerg Infect Dis 2005; 11(3):463-6. doi: 10.3201/eid1103.040657 [Crossref]
20. Ghenai A, Mejova Y. Catching Zika fever: application of crowdsourcing and machine learning for tracking health misinformation on Twitter. arXiv:1707.03778v1 [Preprint]. Available from: https://arxiv.org/pdf/1707.03778.pdf. Accessed 20 December 2020.
21. Perscheid C, Benzler J, Hermann C, Janke M, Moyer D, Laedtke T. Ebola outbreak containment: real-time task and resource coordination with SORMAS. Front ICT 2018; 5(7):1-11. doi: 10.3389/fict.2018.00007 [Crossref]
22. Dagina R, Murhekar M, Rosewell A, Pavlin BI. Event-based surveillance in Papua New Guinea: strengthening an International Health Regulations (2005) core capacity. Western Pac Surveill Response J 2013; 4(3):19-25. doi: 10.5365/wpsar.2013.4.2.001 [Crossref]
23. Chandrasekaran N, Gressick K, Singh V, Kwal J, Cap N, Koru-Sengul T. The utility of social media in providing information on Zika virus. Cureus 2017; 9(10):e1792. doi: 10.7759/cureus.1792 [Crossref]
24. Gesser-Edelsburg A, Diamant A, Hijazi R, Mesch GS. Correcting misinformation by health organizations during measles outbreaks: a controlled experiment. PLoS One 2018; 13(12):e0209505. doi: 10.1371/journal.pone.0209505 [Crossref]
25. Banakar M, Kalateh Sadati A, Zarei L, Shahabi S, Heydari ST, Lankarani KB. Public sphere attitudes toward rumor sources on COVID-19 epidemics: evidence from community perceptions in Iran. Res Sq [Preprint]. 2020. Available from: https://www.researchsquare.com/article/rs-30297/v1. Accessed 20 December 2020.
26. Shu K, Sliva A, Wang S, Tang J, Liu H. Fake news detection on social media: a data mining perspective. ACM SIGKDD Explorations Newsletter 2017; 19(1):22-36. doi: 10.1145/3137597.3137600 [Crossref]
27. Barboza P, Vaillant L, Mawudeku A, Nelson NP, Hartley DM, Madoff LC. Evaluation of epidemic intelligence systems integrated in the early alerting and reporting project for the detection of A/H5N1 influenza events. PLoS One 2013; 8(3):e57252. doi: 10.1371/journal.pone.0057252 [Crossref]
28. Ghenai A. Health misinformation in search and social media. In: Proceedings of the 2017 International Conference on Digital Health. New York, NY, USA: Association for Computing Machinery (ACM); 2017. p. 235-6.
29. Jones B. Mixed uptake of social media among public health specialists. Bull World Health Organ 2011; 89(11):784-5. doi: 10.2471/blt.11.031111 [Crossref]
30. Alkhodair SA, Ding SHH, Fung BCM, Liu J. Detecting breaking news rumors of emerging topics in social media. Inf Process Manag 2020; 57(2):102018. doi: 10.1016/j.ipm.2019.02.016 [Crossref]
31. Qazvinian V, Rosengren E, Radev D, Mei Q. Rumor has it: Identifying misinformation in microblogs. In: Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing. Edinburgh, Scotland: Association for Computational Linguistics (ACL); 2011. p. 1589-99.
32. Al-Sarem M, Boulila W, Al-Harby M, Qadir J, Alsaeedi A. Deep learning-based rumor detection on microblogging platforms: a systematic review. IEEE Access 2019; 7:152788-812. doi: 10.1109/access.2019.2947855 [Crossref]
33. SORMAS deployed in multiple countries for COVID-19 in 2020. SORMAS; 2020. Available from: https://sormasorg.helmholtz-hzi.de/. Accessed 20 December 2020.
34. Milinovich GJ, Williams GM, Clements AC, Hu W. Internet-based surveillance systems for monitoring emerging infectious diseases. Lancet Infect Dis 2014; 14(2):160-8. doi: 10.1016/s1473-3099(13)70244-5 [Crossref]
35. Tracey LE, Regan AK, Armstrong PK, Dowse GK, Effler PV. EbolaTracks: an automated SMS system for monitoring persons potentially exposed to Ebola virus disease. Euro Surveill 2015; 20(1):20999. doi: 10.2807/1560-7917.es2015.20.1.20999 [Crossref]
36. Gupta A, Kumaraguru P. Credibility ranking of tweets during high impact events. In: Proceedings of the 1st Workshop on Privacy and Security in Online Social Media. New York, NY, USA: Association for Computing Machinery (ACM); 2012. p. 2-8. 10.1145/2185354.2185356
37. McCreadie R, Macdonald C, Ounis I. Crowdsourced rumour identification during emergencies. In: Proceedings of the 24th International Conference on World Wide Web. New York, NY, USA: Association for Computing Machinery (ACM); 2015. p. 965-70. 10.1145/2740908.2742573
38. Shao C, Ciampaglia G, Flammini A, Menczer F. Hoaxy: a platform for tracking online misinformation. In: Proceedings of the 25th International Conference Companion on World Wide Web. New York, NY, USA: Association for Computing Machinery (ACM); 2016. p. 745-50.
39. Tom-Aba D, Toikkanen SE, Glöckner S, Adeoye O, Mall S, Fähnrich C. User evaluation indicates high quality of the surveillance outbreak response management and analysis system (SORMAS) after field deployment in Nigeria in 2015 and 2018. Stud Health Technol Inform 2018; 253:233-7.
40. Grainger C. A Software for Disease Surveillance and Outbreak Response - Insights from Implementing SORMAS in Nigeria and Ghana. Germany: Federal Ministry for Economic Cooperation and Development (BMZ); 2020.
41. Yavlinsky A, Lule SA, Burns R, Zumla A, McHugh T, Ntoumi F. Mobile-based and open-source case detection and infectious disease outbreak management systems: a review. Wellcome Open Res 2020; 5(37):37. doi: 10.12688/wellcomeopenres.15723.1 [Crossref]
42. Wilson LS, Maeder AJ. Recent directions in telemedicine: review of trends in research and practice. Healthc Inform Res 2015; 21(4):213-22. doi: 10.4258/hir.2015.21.4.213 [Crossref]
43. Nord G, Rising KL, Band RA, Carr BG, Hollander JE. On-demand synchronous audio video telemedicine visits are cost effective. Am J Emerg Med 2019; 37(5):890-4. doi: 10.1016/j.ajem.2018.08.017 [Crossref]