Literature search

We searched MEDLINE, Embase, the Database of Abstracts of Reviews of Effects, the Cochrane Central Register of Controlled Trials, the Cochrane Database of Systematic Reviews, Health Technology Assessment, the Collective Index of Nursing and Allied Health Literature, PsycINFO and Scopus for pertinent English-language studies published from Jan. 1, 1990, to Dec. 9, 2014. (The search strategies are available in Appendix 1, Tables S1–S9.) The search start date marks 4 years before N95 respirators became a part of standard respiratory protective equipment among health care workers in the United States.

We also conducted searches of the grey literature to obtain unpublished data. These searches were limited to the past 5 years (see Appendix 1, Table S10, for search details).

Study selection

Randomized controlled trials, prospective and retrospective cohort studies, and case–control studies were eligible for inclusion in the meta-analysis. Participants in clinical studies were health care workers in a health care setting. We defined health care worker as any worker in a health care setting who might be exposed to a patient with an acute respiratory infection. We excluded studies that solely involved protection of patients or community populations.

Surrogate exposure studies (i.e., experiments involving manikins or volunteers exposed to artificially produced aerosols) were not eligible for inclusion in the meta-analysis but were summarized to provide an overview of the laboratory-based experimental evidence for use of N95 respirators to protect against acute respiratory infections. Aerosols are defined as a suspension of very small (0.01–100 μm in diameter) particles or droplets in the air.19 Studies with manikins or adult volunteers exposed to an aerosol simulating what might occur in a health care setting were considered.

Study designs assessed the use of National Institute for Occupational Safety and Health certified N95 respirators compared with surgical masks. Certification must have been under public health regulations (42 CFR part 84). Respirators certified under the former regulations (at 30 CFR part 11) were ineligible because they are no longer in use.20 We also included data on European standard filtering facepiece (FFP2) respirators (standards EN149:2001 and EN149:2001+A1:2009) as data on N95 filtering facepiece respirators. We did not include data on elastomeric facepiece respirators because they are not in widespread use in health care settings. The term “surgical mask” was considered equivalent to medical masks, procedural masks, isolation masks, laser masks, fluid-resistant masks and face masks that meet bacterial and particle filtration efficiency standards required by the US Food and Drug Administration (ASTM standard F2100–11) but are not certifiable as N95 respirators. Other types of respirators and surgical masks not explicitly described here were excluded.

Data extraction and quality assessment

The primary outcome of interest from RCTs, cohort studies and case–control studies was laboratory-confirmed respiratory infection, including respiratory infections diagnosed by means of polymerase chain reaction, serology, respiratory virus culture and Bordetella pertussis bacterial culture. Secondary outcomes were influenza-like illness, and workplace absenteeism due to hospital-acquired respiratory infections. The outcomes extracted from surrogate exposure studies were filter penetration, face-seal leakage and total inward leakage.

Two reviewers (J.D.S. and C.C.M.) independently screened abstracts, titles and full texts as described in the selection of studies. Data extraction was conducted using an electronic spreadsheet template (completed independently by J.D.S. and C.C.M.) and compared for discrepancies. Data from surrogate exposure studies were transformed, when appropriate, from fit-factors, protection factors or filter efficiencies to penetration percentages. When necessary, one of us (J.D.S.) contacted authors for additional information (Appendix 1, Table S11).

Randomized controlled trials were explicitly assessed for bias according to the Cochrane risk-of-bias tool.21 Cohort and case–control studies were assessed for risk of design-specific bias using the relevant Newcastle–Ottawa Scale.22

Outcome-specific quality of the body of evidence was assessed in duplicate by the same 2 reviewers using the Grading of Recommendations Assessment, Development and Evaluation (GRADE) framework.23^,24 Disagreements were resolved through consultation with a third reviewer (J.J.). The quality of evidence can be graded as high, moderate, low or very low.

Data synthesis

Where data could be combined for meta-analyses, these data were reported as odds ratios (ORs). We combined similar study designs only for the meta-analysis. Data were measured on dichotomous outcomes (laboratory-confirmed respiratory infection, influenza-like illness and workplace absenteeism). A random-effects analysis model and inverse variance statistical method were used for meta-analysis using Review Manager (RevMan).25

Cluster RCTs were adjusted for the meta-analysis with individual RCTs. We used the intraclass correlation coefficient to determine the design effect.26 Design effect was used to determine the effective sample size.26 When the effective sample size was not a whole number, it was rounded to the nearest whole number.

For meta-analyses involving rare events, zero cell counts were adjusted by including a correction (the reciprocal of the size of the contrasting study arm).27

We assessed evidence of heterogeneity using the χ² test and I² statistic; a χ² value less than 0.10 or an I² value greater than 50% indicated significant heterogeneity.28^,29 Subgroup analysis was planned if there were more than 5 pooled studies and when significant heterogeneity was present.

All statistical analyses were performed with the use of RevMan (version 5.2; The Nordic Cochrane Centre, The Cochrane Collaboration, 2012).

Search results and study characteristics

We screened 8962 titles, excluded 8855 and retrieved 107 articles for full-text assessment. We selected 31 eligible articles involving 29 studies; 6 were clinical studies that we included in the meta-analysis, and 23 were surrogate exposure studies (Figure 1). No unpublished abstracts of RCTs, cohort studies or case–control studies were found.

We included 3 RCTs, 1 cohort study and 2 case–control studies in the meta-analysis.11^–17 The main characteristics of these studies are found in Table 1. All 6 studies reported laboratory-confirmed respiratory infection. Definitions of laboratory-confirmed respiratory infection differed. None of the RCTs used B. pertussis bacterial culture or viral culture. Neither of the RCTs by MacIntyre and colleagues12^–14 used serology. The SARS cases in the cohort study15 and one of the case–control studies were confirmed only by serology.16 Zhang and colleagues17 confirmed influenza only by polymerase chain reaction. All of the RCTs reported on influenza-like illness. One RCT also reported workplace absenteeism; however, the outcome could not be confirmed to result from nosocomial respiratory infections.11

Effect on outcomes

No significant difference in risk of laboratory-confirmed respiratory infection was detected between health care workers using N95 respirators and those using surgical masks in the meta-analysis of the RCTs (OR 0.89, 95% confidence interval [CI] 0.64–1.24; I² = 0%), the cohort study (OR 0.43, 95% CI 0.03–6.41) or the case–control studies (OR 0.91, 95% CI 0.25–3.36; I² = 0%) (Figure 2). Similar results were found in 2 post-hoc meta-analyses: in one, we combined data from the 3 observational studies (OR 0.79, 95% CI 0.24–2.56; I² = 0%); in the other, although not advised, we pooled data from all of the studies as an intellectual exercise to try to ascertain whether more precision could theoretically be obtained (OR 0.88, 95% CI 0.64–1.21; I² = 0%).

We found no significant difference in risk of influenza-like illness between N95 respirators and surgical masks in the meta-analysis of the 3 RCTs (OR 0.51, 95% CI 0.19–1.41; I² = 18%) (Figure 2). We also found no significant difference in risk of workplace absenteeism between N95 respirators and surgical masks in the 1 RCT that measured this outcome11 (OR 0.92, 95% CI 0.57–1.50) (Figure 2).

Risk of bias

The risk of bias for the RCTs is summarized in Figure S1 of Appendix 1. In brief, risk-of-bias ratings were identical across each domain of the Cochrane risk-of-bias tool for all included RCTs (low risk of bias for random sequence generation, incomplete outcome data, selective reporting and “other” bias; unclear risk of bias for allocation concealment; and high risk of bias for blinding of participants) except for blinding of outcome assessment, which was rated as unclear risk of bias for the RCT by Loeb and colleagues11 but as high risk of bias for the other 2 RCTs.12^–14

Risk of bias for the cohort and case–control studies is summarized in Table S12 of Appendix 1. In brief, the cohort study15 received a rating of 6 stars, one of the case–control studies received 3 stars,16 and the other case–control study received 6 stars.17

Outcome-specific quality of evidence

The ratings of importance and outcome-specific quality of evidence that we assessed using the GRADE approach are summarized in Table S13 of Appendix 1. In brief, laboratory-confirmed respiratory infection was deemed a critically important outcome for decision-making with low-quality evidence from RCTs, and an important outcome for decision-making with very-low-quality evidence from observational studies. Influenza-like illness was rated as an important outcome for decision-making with very-low-quality evidence from RCTs. Work-related absenteeism was considered not an important outcome for decision-making with very-low-quality evidence from 1 RCT.

We did not conduct subgroup analyses because no significant heterogeneity was detected. No meaningful sensitivity analyses could be performed because too few studies were included.

Summary of surrogate exposure studies

Twenty-three surrogate exposure studies were included.30^–53 Their outcomes and general methods (e.g., participants, particles used for exposure, number and type of respirator or surgical mask used, flow rates and breathing rates of manikins, size of challenge particles and range of particle size measured) are summarized in Appendix 2 (available at www.cmaj.ca/lookup/suppl/doi:10.1503/cmaj.150835/-/DC1). In general, compared with surgical masks, N95 respirators showed less filter penetration, less face-seal leakage and less total inward leakage under the laboratory experimental conditions described.

Limitations

Despite our study’s many strengths, including a comprehensive search strategy for published data and grey literature, and a thorough review and assessment for risk of bias and quality of evidence using validated tools, limitations of this review should be acknowledged.

None of the studies included in the meta-analysis, except the RCT by Loeb and colleagues,11 independently audited compliance with the intervention. Potential confounding due to concurrent interventions (e.g., gloves, gowns and hand hygiene practices) as part of routine and additional precautions for droplet transmission were not accounted for by our meta-analysis.

We did not assess the impact of harms associated with mask and respirator use that could negatively affect the efficacy of the assigned intervention because it was out of the scope of our review.55

Acute respiratory infections may have been acquired during the study from community exposures rather than nosocomial exposure. In one of the RCTs,12^,13 transmission may have occurred via contamination of provided respiratory protective equipment during storage and reuse of masks and respirators throughout the workday.

Only 2 respiratory virus seasons were assessed by the 3 RCTs; in one trial,14 the peak period of one of these influenza seasons was missed, and in another trial,11 the H1N1 outbreak in 2009 halted the study during the other respiratory season. Year-to-year strain variation of influenza necessitates additional data from other seasons during peak periods.

The weighting of the meta-analysis was influenced by the laboratory-confirmed respiratory infection outcome of serology used in one of the RCTs.11 However, health care workers who received influenza vaccination were appropriately excluded from analysis based only on serology.

Bias due to lack of blinding in all studies was a key factor in the relatively low GRADE quality assessment, and it is impossible to overcome because the health care workers would know which mask they were wearing.

Finally, these results are not generalizable to infections transmitted primarily through airborne routes (i.e., tuberculosis, measles and varicella) or to protection from acute respiratory infections during aerosol-generating medical procedures.3

Conclusion

Although N95 respirators appeared to have a protective advantage over surgical masks in laboratory settings, our meta-analysis showed that there were insufficient data to determine definitively whether N95 respirators are superior to surgical masks in protecting health care workers against transmissible acute respiratory infections in clinical settings. Additional, large RCTs are needed to detect a potentially clinically important difference owing to small event rates. Initial guidelines on preventing acute respiratory infection relied on surrogate exposure data and data extrapolated from the protection of health care workers against tuberculosis because clinical evidence did not exist at that time.58^,59 Randomized controlled trials conducted in clinical settings represent the most valid information to evaluate the effectiveness of N95 respirators. They are more relevant to real clinical situations and report actual outcomes in health care workers, and therefore they are the best evidence on effectiveness to inform policy-making.