Syndromic Surveillance – monitoring animal health data to detect temporal aberrations in near real-time using free software

 [Proposal for next NOSOVE course in June 2014, please vote in the poll] 

Teacher: Dorea Fernanda

Syndromic surveillance can be characterised as a process involving the continuous analysis of health data to provide immediate feedback.

The increasing amount of health data recording in electronic format presents extra challenges for data analysis, but also new opportunities to extract information from data in real and near-real time. In this course participants will have a chance to learn, through several hands-on exercises, how to use freely available software to set up automated, autonomous routines of data analysis.

The theory and exercises will cover all the basic steps to successfully develop, evaluate and implement a syndromic surveillance system capable of detecting temporal aberrations (which may indicate the occurrence of outbreaks) when monitoring cases load from a given animal health data source, such as laboratory submissions, clinical cases, etc. These steps can be summarized as:

• basic text mining methods for automated classification of records into syndromes;
• retrospective evaluation of data to create baseline profiles following the removal of excessive noise and aberrations, and the identification of temporal effects;
• prospective evaluation of detection algorithms; and finally
• real-time monitoring and implementation.

As all software used are freely available, participants will be able to readily apply the skills learned into their work or research.

Course specifications

Participants are expected to have a basic knowledge of biostatistics. No previous knowledge of the software to be used is expected.

The course exercises will use the statistical programming environment R, and  RapidMiner, the world-leading open-source system for data mining available freely from Rapid-I (http://rapid-i.com/content/view/26/84/). Participants will receive download and installation instructions upon registration, and they should bring their own laptop computer. Datasets will be provided as part of the course, but participants are welcome to bring a dataset of their own to explore some of the techniques learned.

Workshop Content

Day 1 – Introduction to the tools:  Basics of using R and Rapid Miner.

Day2 – Syndromic surveillance, Step 1: Automated classification of records into syndromes. Participants will learn the basics of text-mining and will practice implementing supervised (rule-based) and unsupervised (naïve Bayes, Decision Trees) machine learning methods in order to create automated routines to classify records into syndromes.

Day3 – Syndromic surveillance, Step 2: Retrospective evaluation of data available. Basic concepts of time series analysis will be covered. Participants will practice on a dataset in order to learn how to identify statistical characteristics of the time series which can impact aberration detection, in special temporal effects such as day of week and seasonal patterns. Step 3: Prospective evaluation of data. Participants will practice the use of basic temporal aberration detection algorithms, such as control charts (cumulative sums, Shewhart type control charts and Exponentially Weighted Moving Average). More advanced methods (such as Holt-Winters exponential smoothing and removal of temporal effects using regression models) will then be employed to deal with specific characteristics that can be found when monitoring different data sources.

Day4 – Syndromic surveillance, Step 4: Implementation. Participants will learn how to combine tools (Rapid Miner and R) in order to implement a syndromic surveillance system, from data acquisition to final reports.