Concern about when we will know if a vaccine is successful.

What do Vladimir Putin and 15 Nobel Laureates have in common?

Yesterday Vladimir Putin announced that Russia was poised to commence using a vaccine based on early positive results.  This suggestion has been widely derided in the media given the inadequacy of the testing on that vaccine.  Yet two weeks ago 150 scientists and ethicists internationally, including 15 Nobel prize winners, signed an open letter arguing that the current approach to vaccine testing was wrong and demanded a speedier process. Is the planned approach to testing by Oxford and other vaccine research groups too timid? 

What do we know so far?

As has been widely reported, and described in this blog:

• Vaccines suitable for human use have been produced
• In younger volunteers they are safe (at least in the short term)
• They produce a substantial immune response which hopefully should be sufficient to stop infection

What we don’t know yet:

• Whether in practice infection will be stopped
• How well the vaccine works in older people
• How long the vaccine will continue to work after it’s been given
• (None of these have been tested in the Russian vaccine)

The ‘Phase 3’ study 

• To assess in practice whether any vaccine ‘works’, a study needs to be undertaken- the  so called ‘randomised clinical trial’ – which compares the rate of developing an infection between 2 groups: half given the vaccine and the other half an inactive vaccine.
• The comparison is needed to calculate the size of the reduction in infection rate in those given the active vaccine  from the ‘background rate’ provided by the inactive vaccine group
• In such a trial, the volunteer does not know which group they belong to, ensuring that the two groups behave similarly in their day to day behaviours and reporting of any symptoms 

What’s the problem?

• Basically, even if the vaccine ‘works’ by preventing infection it might take more than 6 months from now to prove this.  My justification for this conclusion can be found at the bottom of this page

What was the suggestion from the open letter signed by the Nobel Laureates?

• Given the concern that we would need 6 months, or even longer,  to show success, that would mean we would be well into next year before any vaccine programme could be started 
• This delay, the letter argues, leads to avoidable numbers of deaths, severe illnesses and more damage to economies globally
• Their proposed solution is a push for ‘challenge testing ‘
• I’ve mentioned this previously, but in summary with this a relatively small number of volunteers are given the vaccine, tested for their immunity, and then exposed to the virus.  Hopefully most would not get an infection, hence proving the vaccine’s success
• Such a study would be much smaller and is not dependent on the background rate
• If the volunteers are young and healthy, even if they contacted the infection, they would probably not develop any complication of Covid-19. 
• 30000 volunteers globally have signed up and are ready to go

Is this ethical?

• The letter has been signed by many distinguished ethicists 
• We can’t predict now even in this very selected group, whether there is still a risk of dying from a catastrophic immune response

And also

• We would not know if the vaccine was successful in elderly people or those who had other health problems 

My own view? 

An interesting dilemma.  This would be a major departure from our normal approach to testing vaccines for diseases where there is not a treatment.  There is nothing normal about this pandemic though but also the speed of the development of the vaccines is breath-taking. The cautious view is that societies should continue with all social distancing and other measures for a few more months and just be patient about the vaccine.  If this first wave does not settle down though, then maybe the pressure for a more rapid approach to getting a vaccine out will grow  

How I calculated that 6 months might not be long enough

• As the Oxford and other research groups openly admit, if the background risk of infection is very low it may be impossible to prove the new vaccine is successful in preventing infection
• This  trial is studying around 12000 patients: 6000 in each group
• The volunteers  were recruited in May and June and are being followed to  determine how much lower the infection rate is  in the active vaccine group
• Obviously, the more months of follow up , the greater the number of new infections and hence the greater the chance the trial will have enough numbers to prove vaccine success.
• I have attempted to see how many months will this take.  I have looked into  how likely is it that they will get a positive outcome in (i) 3 months and (ii) 6 months 
• In doing this calculation I have used the survey data that suggest at the moment the background rate of infection  in England is 2/10,000 population per week.
• This is an estimate but could be as high as 5/10,000 or as low as 1/10,000. 
• I have taken as a positive outcome that the vaccine would need to reduce the rate of infection by at least 70%.  (This is the average for influenza vaccine but may be pessimistic.) 
• I have used standard statistical methods to assess the chances of the trial showing the vaccine at 3 and at 6 months and  for each of these 3 estimates of background rate.

Background infection rate/week	Lowest 1/10,000	Middle 2/10,000	Highest 5/10,000
3 month trial	✘	✘	✓
6 month trial	✘	✓	✓

• What this table shows that at the lowest estimate of background risk the trial would not show a benefit even at 6 months.  Benefit will only be shown at 3 months if the background risk is at an (unrealistically high I believe) level of 5/10000/week
• In fairness if the vaccine success was as high as 90%, then the trial would be successful at 3 months for the middle (but not the lowest) background estimate.