🎉 JOGL is soon launching a new version. All the users of the v1 will be migrated to the new version. In the time being, we do not allow the creation of new users on this platform.
Web-based Covid-19 simulation for policy-makers banner
Project
3
Members

Status:
Active/Ongoing
Project maturity:
Implementation
Linked to group(s)/challenge(s):

Web-based Covid-19 simulation for policy-makers

About reviewed project
Mathematical modelling underpins our ability to predict the future of the Covid-19 pandemic. We have generated an individual-based simulation for policy making and we want to make a web-based version that anyone can use.

1 Introduction


1.1 Problem and Background

Covid-19 has influenced how we shop, how we are educated and how we socialise, pushing many of these events online. However, following successive epidemic waves and subsequent development of vaccines we are now seeing a drive to get people back into more public spaces, such as classrooms and Halls of Residences at Universities.  

 

We have generated an individual-based simulation algorithm that predicts the infection spread of Covid-19 within educational spaces and we have used it to provide Welsh Government policy-makers with results on how best to intervene and reduce infection spread. However, in our interactions we have found that policy-makers often need different simulations at different times, which require simple adjustments to our code.  

 

Currently, the code is in the form of MATLAB script, which only trained mathematicians can use. To optimise the interaction between policy-makers and academics, we will construct a simple web-based General User Interface (GUI) that will allow users to simulate their own scenarios and, thus, predict the outcome of interventions under a vast array of conditions. 

 

1.2 Solution summary in simple terms

We have constructed an algorithm (https://github.com/joshwillmoore1/COVID-19-LFD-IBM) that can track individual infections across a variety of different education environments, under several different interventions. The code is comprised of different modules that can be turned on, or off, as appropriate, so that the simulation can match a variety of policy situations. Figure 1 illustrates the ideas behind the algorithm. 

 

Figure 1: Flowchart illustrating the general iterative process controlling the infection dynamics of a discrete population over a typical day. 


We have already tested the code across a variety of scenarios. However, policy-makers always require more simulations as new data is produced and as they think up new scenarios. We will convert our complex simulations into simple user interfaces using “Shiny Applications” that put the power back into the policy-maker's hands. Namely, policy-makers will be able to simulate and intervene in different scenarios as fast as they can think them up. 


1.3 Solution summary in technical terms

Figure 2 illustrates the detailed structure behind our individual-based simulation. The algorithm accepts many different input variables, such as: 

  • probability people follow the rules; 
  • interventions e.g., masks, ventilation, etc.; 
  • ratio of asymptomatic spreaders to symptomatic spreaders; and 
  • probability of false-negative and false-positive results in COVID-19 tests. 

Thus, we can simulate many hypothetical scenarios, such as cases in secondary schools in which compliance is good, but the ratio of asymptomatic individuals is high and the lateral flow devices have a high false-negative probability. 

 

However, with every additional factor that can be changed the number of possible scenarios that can be modelled grows combinatorically. Thus, instead of choosing which results we think policy-makers may be interested in we want to split our expertise. Specifically, we focus on our strengths of producing a usable and robust interface that provides policy-makers with the ability to generate the results they require, whilst exploring different intervention options that will reduce the number of future infections. 

 

Figure 2: Flowchart illustrating infection spread. Colours represent subroutines: grey is base infection, green is testing, red is secondary infection estimation and blue represents infections from outside the population. SI corresponds to secondary infections. 

 

1.4 State of advancement of the project

The research behind this project has been completed and can be found as a preprint at https://medrxiv.org/cgi/content/short/2021.03.08.21253122v1

 

The MATLAB code for the simulations exists (https://github.com/joshwillmoore1/COVID-19-LFD-IBM). 

 

This project focuses on converting the MATLAB code into R code, which can be used to produce a simple web-based GUI, as a Shiny App. The grant will fund a 14-day project which will proceed as proposed below. 

 

  • Days 1-2 Convert basic algorithm loop into R code (see Figure 2). 
  • Days 3-4 Add in interventions. 
  • Days 5-6 Construct interface using https://www.shinyapps.io/
  • Day 7 Test output consistency. 
  • Day 8-10 Iterate usability with test group. 
  • Day 11-13 Documentation writing. 
  • Day 14 Contingency. 

 

2 Project Implementation 


2.1 Solution

We seek to create a solution to the problem of policy-makers wanting to vary simulations to understand prediction sensitivity to parameter and intervention changes. Specifically, whenever we present our research to policy-makers, we are only able to illustrate a small number of solutions that we believe they will be interested in. However, policy-makers are frequently interested in new scenarios, as well as wanting to run simulations with updated data. 

 

By constructing a well-documented, user-friendly GUI, as illustrated in Figure 3, we provide policy-makers with a simple front-end to a complex algorithm (see Figure 2) that can be used to simulate multiple scenarios. Specifically, the blue sections represent inputs. The parameter table on the left represents default parameters, as estimated from current data. These input parameters can either be left at their default values or updated based on a user’s knowledge. 

 

The grid at the top delineated by black lines presents a set of toggle switches that can be turned on, or off, depending on the scenario that is being simulated. For example, we would assume a secondary school population would not mix on the weekend (the toggle would be set to ✗) and we would want to isolate individuals who generate a positive covid-19 test (the toggle would be set to ✓). Moreover, we can set specific days of the week for tests to occur and what signal causes testing to begin. Namely, by default, the testing schedule is assumed to be fixed (as specified by the choice of testing days). However, if “Testing after first symptomatic is found” is ticked then testing does not start until a symptomatic individual is detected. The probability of this event is then highly dependent on the ratio of asymptomatic individuals (parameter in the left blue table). 

 

Figure 3: Schematic diagram of the General User Interface, GUI. 

 

Once the parameters have been set and the scenario switches have been chosen. The calculate button is clicked and the simulation as defined in Figures 1 and 2, is run 100 times. The algorithm is run multiple times due to the stochastic nature of an individual-based simulation. Specifically, by running the simulation 100 times we can provide an estimate for the mean values of several output metrics. Namely, the grey box represents four suggested outputs, i.e. (left to right, respectively) number of non-isolated and infected individuals on a given day, total number of infections, number of isolating individuals and number of recovered individuals. Finally, two summary statistics are quoted: the average total number of infections that have occurred by the end of the simulation and the total number of days spent in isolation, per individual. The goal of the policy-maker is, thus, to find scenarios that minimise these statistics. 

 

For ease of use we will also include the ability to save and load parameter values as text files and output simulation data as “comma separated values” (.csv), which is a standard file type that can be read by all spread sheet software. Thus, a policy-maker can quickly and easily manipulate their scenario, trialling multiple perturbations. Moreover, a user can save their results, allowing them to easily collate, compare and plot their data using external software. 

 

Currently, the Covid-19 simulator has been used to simulate educational environments, such as secondary school classrooms and a University’s Halls of Residence. However, the simulator could be used to predict infection spread in more diverse scenarios. For example, we have been asked to consider situations such as mixing between hospital wards and care homes. These environments could easily be encapsulated into the algorithm with minor alterations.


2.2 Methodology

The agent-based model considers a population of N susceptible individuals split into groups of size Ng. The parameter Ng is the number of local contacts an individual has and represents the number of isolations that would occur should a member of a group become identified as infectious. This parameter can be varied to test a range of isolation policies.  

  

Critically, the effectiveness of isolation is modulated by a ‘compliance’ parameter. Namely, after receiving a positive test result, or becoming symptomatic, an individual isolates themselves with a probability specified by the compliance parameter. 

  

During any day that testing is applied all individuals that are not isolating are tested. The testing phase occurs before the mixing phase (see Figure 1). Thus, any individual triggering a positive result is isolated before they can infect others. 

  

The blue sections of Figure 1 and 2 control external infections, i.e., infections that enter the modelled population due to infected individuals outside of the modelled population. The red sections of Figure 1 and 2 control how secondary infections are generated due to mixing between individuals. During any day where the students can mix, any infectious individual can infect other people. Once a person becomes infected three delay timers are attached to each of them. The three delays, ti, td and tr, represent: 

  • the time between becoming infected to becoming infectious, ti = 3 days; 
  • the time between becoming infected to becoming detectable by a LFD test, td = 5 days; and 
  • the time between becoming infected to recovery, tr = 10 days. 

Any recovered person is assumed to be non-infectious and immune to further infections and that the tests being used are Lateral Flow Devices, LFD, which are assigned to have a false positive probability and a false negative probability. 

 

At the point of infection each agent is either given a symptomatic or an asymptomatic flag based on a prescribed probability. Asymptomatic individuals can infect susceptible individuals and will not isolate unless a positive test is returned. Symptomatic individuals are assumed to follow self-isolation guidance at a rate determined by the compliance parameter at any time they show signs of infection. 

  

To produce the web-based simulator, we need to convert the existing algorithm from the computational language of MATLAB to the language of R. Most MATLAB functions have direct analogues with R functions and as our implementation of the algorithm has minimal dependencies on MATLAB, we expect the conversion to R will result in no loss of functionality, but rather an increase in accessibility and locality optimisation. 

 

2.3 Results

The output of this project will be a GUI, as illustrated in Figure 3. The parameters will be set in the left table. The switches at the top of the GUI will allow a user to create a specific scenario, easily and quickly, with given interventions. Each numerical input parameter will be a reactive variable that will feed directly into the model once the “COMPUTE” button is pressed.  

  

Finally, we will provide the user a button to reset all data to default values and a button to save simulation inputs and outputs to the local machine. The input/output data will be assembled into a single data frame and will be exported to .csv format using Shiny app functionality. 

  

Each component of the model can be active/inactive and, thus, adapted to investigate various infection settings. For example, Figure 4 demonstrates the ability of the model to quickly test the efficacy of two currently proposed isolation policies in classrooms by simply adjusting model parameters. 

 

Figure 4: A comparative plot of two isolation policies currently proposed for primary and secondary schools in the UK. Isolation policies are implemented from changing Ng from N=30 to 1 for ’Firebreak’ and ’Test-to-release’ policies, respectively. Simulations consider 30 agents in a worst-case scenario of high R, high LFD false-negatives and low probability of symptomatic over a four-week period. These simulations consider no additional infections from the wider population and agent compliance is set to 100% with no additional nonpharmacological interventions. Results show the mean of 1000 simulations for each isolation policy. 

  

Specifically, we compare a “Firebreak” strategy (where total individual isolation is imposed when a symptomatic person is found), with a “test-to-release strategy” (a negative results from a Lateral Flow test is used to release isolating individuals from lockdown). By comparing the top-right line graphs a policy-maker could conclude that the total isolation strategy minimises the total number of infections, but leads to large number of healthy individuals isolating (compare bottom-left graphs). Thus, a policy-maker could weigh up the pros and cons of the predicted results. 

 

To demonstrate the versatility of the algorithm, we consider a further scenario in Figure 5. Here, we focus on the two metrics of average total number of infections and average number of days in isolation per student. Figure 5 illustrates how interventions of masks, testing and ventilation can be compared across simulations. Here the simulation was set up to match a secondary school scenario and demonstrates that although testing helps (the points move left with increased testing), simply enforcing mask use is just as good as daily testing (the red points are lower and left when compared to the most transparent black points). Having these simulated results provides a policy-maker with all the predictive power they need to generate an informed decision. 

 

Figure 5: Secondary school simulation comparisons across different R values and different test false negative rates. The colour and transparency of each symbol represents a different intervention, see legend for details. 

  

3 Safety, quality assurance and regulation 


There is no intrinsic risk associated with operating a web-based GUI. However, there are two possible risks stemming from the GUI’s results, which we consider in the next two sections. 

 

3.1 Incorrect code 


If the GUI is incorrectly coded the results will be wrong. At best, the resulting decisions made a policy-maker would simply be quantitatively incorrect, whilst be qualitatively correct. At worst, the resulting decisions could be dangerous if the simulations incorrectly promote interventions that lead to increases in infection rates. 

 

Such risks can be alleviated by first comparing the app’s results with MATLAB’s results. Consistent results will demonstrate that the code has been correctly translated one language to another. Further, the creation of the R code will follow a “test-driven” development. Namely, simple test cases (which can be directly checked) will be used to track the software development by repeatedly testing the software against all test cases. Further, due to the modular form of the algorithm (see Figures 1 and 2) each part of the code can be tested and coded separately. Not only will this speed up the debugging process, but it ensures code accuracy as we can question the workflow in terms of the meta structure of Figures 1 and 2, rather than the exact implementation. 

 

3.2 Incorrect input 

Even if the code is completely correct the GUI could be incorrectly used by a policy-maker. Specifically, mistakes in the input will generate mistakes in the output, resulting in the policy-maker having an incorrect understanding of the situation. As in the previous section, these mistakes may be inconsequential, at best, or dangerous at worse if the policy-maker suggests a solution that raises the infection rate. 

 

We provide two solutions to alleviate these potential issues. Firstly, we will provide a clearly documented manual page for the GUI. Further, if time permits, we would like to record tutorial videos demonstrating the use of the GUI, which will support the written documentation. 

 

Secondly, we state that the GUI’s results should not be the only source of evidence that policy-makers use when deriving their conclusions. Having worked with the Welsh Government we know that this is the case, but we will make this explicit in the GUI’s usage by having a “splash screen” disclaimer appear at the beginning of the GUI’s usage that specifies that the use of the simulated data should only be undertaken by someone who (i) has read the documentation and (ii) is willing to compare their results with other sources. 

 

 

4 Impact, issues and risks


4.1 What impact do you feel your project could have?

This app will help policy-makers make rapid decisions based on evolving data, without the need of background knowledge of numerical computation.  

 

Our work has already influenced Welsh Government policies regarding education workplaces and residences. The results have been considered by: the Welsh Government’s Technical Advisory Group; the Higher Education Task Force; and the Environmental Science Policy Committee. The work has been further communicated to UK-wide colleagues and has been presented to the English Government’s “Scientific Pandemic Influenza Group on Modelling”. 

 

The connections with policy-makers already exist. We want to put the power of predictive simulation into their hands. 

  

4.2 What do you think would make your project a success?

The key to the success of the project is in providing at GUI that is simple to use and interpret. This is considered during days 8-10 (see Section 1.4), where (before the GUI is released widely) we will provide select policy-makers with a “beta-build” and iterate the GUI based upon the responses regarding ease of use. Further, during the manual writing stage, we will acquire feedback from the policy-makers regarding the clarity of our instructions. 

 

4.3 Please list the known issues, potential risks, grey-areas, etc in your project 

We have included many important features that we believe are key for understanding the spread of Covid-19. However, there are many more features that are not currently included. 

 

For example, rather than fixing parameters we could use a Bayesian approach to sample from realistic parameter distributions, which can be generated from data. Equally, current research is focused on generating estimates of real time simulations of airborne particle spread. This could be encompassed into our simulation, but we would need to increase the time resolution from days to hours. Equally, the two-compartment model is only a crude representation of the larger social interaction networks that humans have and, thus, more realistically, we should include a network structure of contacts between the individuals. 

 

All these additional features could be included within the model. However, more time would be required and our aim is to get this tool into the right hands as quickly as possible, whilst updating the code through future iterations. 

 

Beyond additional features, there are limitation placed on the algorithm due to the speed of simulation. Namely, the time to simulate a scenario grows in proportion to the number of individuals being considered. As we are limited by the processing power of shinyapps.io, there is little that can be done to mitigate for this issue, apart from highlighting in the manual what scenarios are appropriate to be run on the simulator. Namely, the code runs rapidly for approximately 100 people, thus, we are focusing on small scale simulations. 

 

5 Originality


5.1 What other projects on JOGL are like yours? 

There are several projects that are like ours. However, none have the same technical background and aim of working with policy-makers. 

  1. EPI-CENTER, https://app.jogl.io/project/169/epicenter, aims to create an open-source epidemiological modelling toolkit. We focus on the useability of our algorithm rather than producing a complete description of the disease. Thus, although our aims are different there is definitely a synergy between our projects of using the EPI-CENTER toolkit to inform our code, which we then package as a GUI for policy-makers. 
  2. Risk Calculator VS COVID-19, https://app.jogl.io/project/170/Savedoctors, has a similar aim of packaging an algorithm for a non-mathematical audience. However, there focus is on calculating risks for doctors, rather than infection tracking. However, there may be a possibility to collaborate on the useability aspects of the GUI. Namely, what has the team found works best in terms of producing a user-friendly calculator? 
  3. Open Epidemiologic Sanitary Smart Resources (Madrid), https://app.jogl.io/project/190/OESSR, focuses on creating an evolving map of virus spread. This is a similar idea to ours in that their work allows a lay person to track infection spread. However, our scale of solution is much different, namely we are on the scale of a small community, whereas this is on the scale of a city. 
  4. Project Lockdown, https://app.jogl.io/project/311/ProjectLockdown, although different in approach as they are looking to generate a platform the provide information on Covid-19 related human rights, there is a synergy with our project as we are looking to aid policy-makers in forming their decisions. Thus, it would be useful to link with this team to understand what interventions are occurring around the world, which would direct the next addition of features in our algorithm. 
  5. CoronaNet Research Project, https://app.jogl.io/project/456/CoronaNetResearchProject, this project focuses on extracting real-time data. This could then be used to update the default parameters in our GUI, or offer range of prebuilt parameter groups from around the world. 


5.2 Is this an innovative project? 

As seen in the previous section, JOGL hosts several projects that focus on providing predictive tools to the non-scientific audiences. However, we seek to serve a specific niche of this community, policy-makers. Critically, policy-makers have direct impact on the spread of infection, as well as the happiness and freedoms of their constituents. Thus, it is imperative that they are well informed and adaptable to the rapid changes that we see in the data. Further, our work will seek to build on some number of the JOGL projects as we investigate how to extract timely parameter values, or current intervention practices. 

 

This work also innovates on the research that we have already produced. Namely, we have all the resources ready to be used, we simply require funding to action the transformation from MATLAB code to shiny app. 

 

5.3 Is there already an open source version of this project? 

All of the MATLAB code that the project will use can be found through the GitHub page 

https://github.com/joshwillmoore1/COVID-19-LFD-IBM 

This page will be updated with iterations of the MATLAB code and the R code that is used to form the shiny app. 

 

6 Team experience 


Joshua W. Moore (JWM), achieved a first-class mathematics Batchelors degree in 2019 and is now a PhD candidate in applied mathematics. His project focuses on developing multiscale models of mammary organoids to optimise their industrial production. This project is an interdisciplinary industrial collaboration with Cellesce Ltd

 

He won a grant from Cardiff Undergraduate Research Opportunities Programme (CUROP), which funded a summer placement, under the supervision of TEW, creating open-source, test-driven software that could be used by clinicians to calculate the appropriate radiation dosage for spinal tumours. A preprint can be found at https://www.medrxiv.org/content/10.1101/2020.12.20.20248585v1

 

Alongside his doctoral work he has worked closely with TEW to develop web-based solutions for a variety of Covid-19 related problems. He was a member of team that successfully won a NERC Hackathon challenge. The team considered the impact of social distancing on the ability of public transport to be more environmentally efficient than private transport. This work was transformed into an online app, demonstrating a local optimum of seating arrangements on public transport (https://bit.ly/App-interface). 

 

Moreover, JWM’s rapid coding skills were used to develop an online calculator, which provided the Welsh Government with estimates and error bounds. This work has led to his first publication [1] and converted into a simple GUI (http://bit.ly/Secondary_infections_app). 

 

JWM’s theoretical work on organoid structure has been shortlisted for the Lighthill-Thwaites prize, which is awarded by the Institute of Mathematics and its Applications. He is due to give an invited lecture in April 2021, resulting in this research being published in the IMA Journal of Applied Mathematics. 

 

JWM is a supporter of open-source software development and publishes all his code through his github page https://github.com/joshwillmoore1

 

Thomas E. Woolley (TEW), www.thomaswoolley.co.uk, is a Senior Lecturer in Applied Mathematics at Cardiff University. He is a former Junior Research Fellow of the University of Oxford, St John's College (2013-2017) and Fellow of Modern Mathematics of the London Science Museum (2014-2016). 

 

TEW's work focuses on using his mathematical skills in stochastic simulation, deterministic modelling and spatial patterning to understand biological complexity. As such he has worked on a wide variety of problems, from animal pigmentation patterning to neurogenesis (see https://www.cardiff.ac.uk/people/view/783107-woolley-thomas for a recent list of publications). 

 

His most recent work on Covid-19, in collaboration with JWM, covers several applications. He has published work, used by the Welsh Government, on the impact of students returning home from University during the Winter holiday [1]. This work was turned into a simple GUI (http://bit.ly/Secondary_infections_app). 

 

He was also the senior advisor to the NERC sponsored Hackathon project that JWM’s team won. The team considered the impact of social distancing on the ability of public transport to be more environmentally efficient than private transport. This work was also transformed into an online app, demonstrating a local optimum of seating arrangements on public transport (https://bit.ly/App-interface). 

 

He has published over 50 peer reviewed papers in a variety of internationally leading journals and books, including Journal of the Royal Society Interface (impact factor 3.9), Cerebral Cortex (impact factor 8.3) and Nature Communications (impact factor 11.88). He has over 1000 citations, with over 170 on the highest cited paper (see http://bit.ly/TEW_scholar for data on publishing and citations).  

 

TEW has received over £100,000 in academic funding for workshops and research projects. Most recently he won funds for two KESS2 Phd studentships that link with industrial partners. Specifically, he is working with the biomedical company Cellesce Ltd on a project involving modelling organoid formation for understanding breast cancer and the fertilisation clinic London’s Women’s Clinic on a project involving optimising embryo choice for IVF success.  

 

TEW is currently supervising 4 doctoral students and 3 Master’s students. Previously he has supervised 10 successful Master’s dissertations and two doctoral candidates. 

 

[1] Paul R. Harper, Joshua W. Moore & Thomas E. Woolley (2021) Covid-19 transmission modelling of students returning home from university, Health Systems, 10:1, 31-40, DOI: 10.1080/20476965.2020.1857214  

 

7 Funding and Costs 


7.1 Costing 


2000 Euro comes to £1,778 using Cardiff University’s approved exchange rate. The University specifies a 5% contingency, which covers fluctuations in exchange rates, so JWM’s wage will come from 

0.95x1778=£1689.10. 

JWM’s hourly rate is 

£15.00 (basic rate) + 12.07% (holiday pay) = £16.81 per hour. 

The University pays 

JWM’s wage + £0.90 (admin cost) = £17.71. 

Thus, JWM would be covered for 

1689.10/17.71=95.4 hours. 

Which is 13.6 days of work. 

 

7.2 Funding so far 


Currently, this work has been done as a side project alongside the doctoral work of JWM. Having funds would allow us to focus solely on the project, allowing us to deliver the GUI in a timely manner. TEW’s position is funded pro bono by Cardiff University whilst he is working with the Welsh Government. 

 

7.3 Funding request 


We request funding of 2000Euro, which will pay for JWM to work on the project for 95 hours, or approximately 14 days of work. The student will be employed through Cardiff University’s “Jobshop” and paid an appropriate hourly rate. This comes to £1,778 which is 2000Euros using the University approved exchange rate (see Section 7.1). 

Additional information
  • Short Name: #SimulationForPolicyMakers
  • Created on: March 10, 2021
  • Last update: July 12, 2021
  • Grant information: Received €1,800.00€ from the OpenCOVID19 Grant Round 5 on 03/24/2021
Keywords
Simulation modeling
coding
Mathematical software
Mathematic
3Good Health and Well-being
4Quality Education