Firearm injuries go down during NRA conventions

On March 1 this letter to the editor by Anupam Jena and Andrew R. Olenski, with data, appeared in the New England Journal of Medicine (click on screenshot to go to the letter):

The reference is at the bottom of this page, and the 19-page supplementary appendix (to a two-page letter!) is here.

The authors used American insurance data on gun injury rates during 9 years of National Rifle Association (NRA) conventions between 2007 and 2015. The uninsured population was not sampled, a caveat that the authors mention in their Appendix. They looked at firearm injuries on all days for three weeks before and after the dates of the conventions themselves, as well as during the conventions. They explain the controls in the Appendix:

For example, for the 2015 NRA annual convention held Friday, April 10 to Sunday, April 12 in Nashville, TN, the treatment group consisted of individuals who received outpatient (including emergency department) or inpatient care during those dates and the control group consisted of all individuals who received care Friday through Sunday in the 3 weeks before and after the convention.

The data are plotted as the injury rates during the convention versus the same days before and after the convention. Their hypothesis, based on claims of gun owners, was that gun injuries would be higher during NRA conventions, as inexperienced people without proper training would be using the guns, while NRA members, having that training and experience, wouldn’t be using their guns during conventions.

In fact, as these data show, the rate of gun injuries was significantly lower during NRA conventions, and during the periods before and after them. The differences were statistically significant at the p = 0.004 level, which is considered highly significant (anything lower than p = 0.05 is seen as statistically significant).

Among 75,567,650 beneficiary-period observations in the claims analysis, 14.3% occurred on NRA convention dates. The unadjusted rate of firearm injuries was lower during convention dates than during control dates (129 beneficiaries with a firearm injury among 10,883,304 persons [1.19 per 100,000] vs. 963 beneficiaries with a firearm injury among 64,683,254 persons [1.49 per 100,000]; P=0.004; relative difference, 20.1%; 95% confidence interval, 6.7 to 34.0). The findings were unaffected by adjustment for covariates (Figure 1).

They note that gun-related injuries, which include deaths, drop by 20% throughout the US during NRA conventions, and decrease by 63% in the state hosting the convention. They conclude, dryly, that this is not consistent with the “inexperienced gun users cause injuries” hypothesis, but is consistent with the notion that “experienced gun owners” (read: NRA members) cause injuries, and the decrease is related to NRA members holstering their weapons during conventions:

These findings are consistent with reductions in firearm injuries occurring as a result of lower rates of firearm use during the brief period when many firearm owners and owners of places where firearms are used may be attending an NRA convention. Our results suggest that firearm-safety concerns and risks of injury are relevant even among experienced gun owners.

The data:

 

Figure 1. Firearm Injuries among Commercially Insured Persons That Occurred on Dates of National Rifle Association (NRA) Annual Conventions and Control Dates, 2007–2015. Shown are the adjusted rates of firearm injuries during convention dates versus control dates in a beneficiary-level multivariable linear regression of firearm injury as a function of indicator variables for convention and control dates, age, sex, indicators for calendar week and year (to adjust for seasonal trends), and state fixed effects. In Panel A, the adjusted estimates are from a model in which the key explanatory variable was a binary variable for NRA convention dates versus all control dates combined. In Panel B, the adjusted estimates are from a model in which the key explanatory variables were indicator variables for week relative to the NRA annual convention. In both panels, 𝙸 bars indicate 95% confidence intervals.

Now this is a big drop, and it surprised me. As Ars Technica notes, the NRA has the same objection that struck me, but the authors have a potential counterclaim:

In a statement to CNN, NRA’s director of public affairs, Jennifer Baker, called the study “absurd.” She continued: “This study is another example of when data and numbers fly in the face of logic and common sense.”

Baker went on to note that only a small fraction of the country’s gun owners—a group that totals about a third of Americans—attend the NRA’s annual conventions. She questioned how such a relatively small number of gun owners could explain such large decreases in injuries.

In a response to CNN, co-author Jena emphasized that the study was not designed to explain the cause of the drops. But he speculated that gun owners who attend NRA conventions may be those who tend to use their guns more frequently than non-attending owners.

Moreover, he and Olenski noted a potential domino effect from the convention disrupting other group events or trips involving firearms and venues, such as shooting ranges and hunting grounds, where owners may temporarily close up to attend the convention. Last, the researchers noted that many NRA convention goers travel long distances to attend, potentially helping to explain the nationwide declines. For instance, 60 percent of the 81,000 NRA members attending the 2017 convention ventured more than 200 miles to get there.

Good Lord: 81,000 people go to an NRA convention? That is one big meeting! But Baker’s objection does need to be considered.

Remember that this is from nine years of data, though. Perhaps there is some other correlate that explains the significant reduction in gun violence; and, as I noted, this is the insured population only, while many gun users are uninsured. In the meantime, take this as an intriguing result that might be right, but deserves extra careful scrutiny since it conforms to what we gun opponents want to believe.

h/t: Michael

____________________

Jena, A. B. and A. R. Olenski, 2018. Reduction in firearm injuries during NRA annual conventions. New England J. Med.  378:866-867 DOI: 10.1056/NEJMc1712773

43 Comments

  1. Posted March 3, 2018 at 12:08 pm | Permalink

    We should get the CDC to look into this further. Oh damn, they are prevented from doing so by law.

    • Heather Hastie
      Posted March 3, 2018 at 12:58 pm | Permalink

      +1.

    • Posted March 3, 2018 at 10:17 pm | Permalink

      This is an independent study then?

      • Posted March 4, 2018 at 9:56 am | Permalink

        The authors are from Harvard Med School and Columbia and their article was in the New England Journal of Medicine. After a quick scan, they didn’t say anything about funding.

  2. Darrin Carter
    Posted March 3, 2018 at 12:09 pm | Permalink

    Sob

    • Darrin Carter
      Posted March 3, 2018 at 12:10 pm | Permalink

      Sub again

  3. Michael Fisher
    Posted March 3, 2018 at 12:11 pm | Permalink

    Sub [Lurking for the weapon ‘genomics’ posters – they’re entertaining]

  4. glen1davidson
    Posted March 3, 2018 at 12:14 pm | Permalink

    You’re not shooting while you’re stocking up.

    Glen Davidson

  5. Posted March 3, 2018 at 12:26 pm | Permalink

    Very difficult to understand. I’ll accept their numbers, but the graphs are not all that impressive.

    “This study is another example of when data and numbers fly in the face of logic and common sense.” Does she mean like quantum mechanics or relativity?

    Too bad no nut case runs amok with a gun in one of these conventions. I suspect, tho, that security is very tight.

    • Ken Phelps
      Posted March 3, 2018 at 12:33 pm | Permalink

      That kind of caught my attention too. Darn all that reality stuff, getting in the way of what she thinks. Clearly she has “another way of knowing”.

      • Mark Joseph
        Posted March 3, 2018 at 3:29 pm | Permalink

        Science: ruining things since 1542.

    • Jessy Smith
      Posted March 3, 2018 at 1:31 pm | Permalink

      The NRA isn’t so quick to dismiss odd sounding statistics when they are in firearm users favor, like the study that states 1 to 1.5 million firearm users use a firearm defensively every year.

      30 percent of Americans own firearms, or 100 million people, meaning 1 to 1.5 percent of these people use their firearm to prevent a crime every single year.

      This is right about the national average for crime in the USA. The implications on known crime statistics make the numbers absurd.

      • Posted March 3, 2018 at 2:19 pm | Permalink

        Perhaps “using a firearm defensively” includes cases where the gun owner hears some unexplained sound in the night and gets their firearm out, ready to defend their family.

        • alexander
          Posted March 4, 2018 at 2:51 am | Permalink

          Or policemen on the way to arrest someone…

      • Mark Joseph
        Posted March 3, 2018 at 3:31 pm | Permalink

        I have always felt that that number, bruited ad nauseam by gun nuts, was absurd. Thank you for the “back of the envelope” calculation.

        • Posted March 3, 2018 at 6:10 pm | Permalink

          Add to that, a great deal of crimes are orchestrated in a way such that the criminal doesn’t want to be seen (i.e. rather than mugging someone, they break into a store, a car or house while it is closed). So seeing anyone, with or without a gun, may deter a lot of criminal acts. But I have read the hypothesis elsewhere than this type of data is highly subjective and error prone; and, yes, it may include many cases like a possum knocking over the garbage can and the homeowner grabs his gun…

          More telling, other civilized countries without our level of gun ownership don’t have crime rates (especially violent crime and murder) at the rate that would be expected if these guns were really stopping all these crimes.

          • Michael Fisher
            Posted March 3, 2018 at 6:23 pm | Permalink

            Agree with all of that. That’s why UK coppers don’t want to carry firearms – it ends up as an arms race where everyone loses. I was reading a NYT report today [linked in my comment above] about a US L.A. policewoman who carries an AR-15 as standard – boggles my Brit mind.

            • Michael Fisher
              Posted March 3, 2018 at 6:26 pm | Permalink

              And not a great tool for the job anyway given the superior semi-automatic choices available against an imagined assailant with equal armament, vest etc.

    • Posted March 3, 2018 at 10:22 pm | Permalink

      If it flies in the face of logic and common sense, that means her premises are wrong, or thinks using common sense is just as rigorous as an empirical study.

  6. mikeyc
    Posted March 3, 2018 at 12:26 pm | Permalink

    I commented on this the other day – if you dig into the data (in the supplement) this decrease in reported gun related injuries during NRA conventions reached significance ONLY for men and only in the South and East. There was no decrease in injuries among women or people from the East or Midwest. As Dr PCC(e) says this comprts with expectations of those of us who advocate gun control so I agree that we need to be careful with the data.

    We need more data but our dear leaders have made that difficult, impossible in some ways, to obtain.

    • mikeyc
      Posted March 3, 2018 at 12:28 pm | Permalink

      South and WEST.

      Stoopid fat fingers

  7. Sven
    Posted March 3, 2018 at 12:32 pm | Permalink

    It also puzzles me why gun conventions have such strickt gun rules. Any gun carried needs to be sealed and without amo.
    I don’t understand why this is nessicary, according to their own words it should be filled with people who know how to handle guns safely without rules.

    • Mark Joseph
      Posted March 3, 2018 at 3:32 pm | Permalink

      This argument also works against Drumpf’s “arm the teachers” lunacy. OK, let’s see the senate and the house of reps have armed guards, too. Or better yet, let them all carry. They don’t allow that, hmm? Wonder why not?

  8. Michael Fisher
    Posted March 3, 2018 at 12:36 pm | Permalink

    Lengthy, interesting NY Times article today: ‘Once Banned, Now Loved and Loathed: How the AR-15 Became ‘America’s Rifle’ by ALI WATKINS, JOHN ISMAY & THOMAS GIBBONS-NEFFMARCH HERE

  9. Posted March 3, 2018 at 1:00 pm | Permalink

    It is pretty surprising that the findings would be this significant, but on the surface some reduction in injuries makes sense given the overwhelming amounts of peer reviewed data demonstrating that increased gun ownership is correlated with increased injuries, suicides, and homicides.

    Given insured owners are likely to exercise gun safety and be more responsible than uninsured owners, it would seem plausible that the reduction amongst the rest of the gun-owning population would be even more significant.

  10. Randall Schenck
    Posted March 3, 2018 at 1:02 pm | Permalink

    These stats do not surprise me at all. I would expect to see the injury numbers go down during their conventions. I would also very much suspect that injuries go up during the big hunting seasons, especially deer and bird hunting times.

    It is pretty simple…the more you drive, the more wrecks you have. Having some training in gun safety does not have much affect on how the individual is going to act and operate in the great outdoors (outside the house). I have been surprised at how sloppy a person operates or handles himself when using a gun. It can be very surprising and alarming when you think you know the person. Long ago when I still did these things our practice was to warn a guy once. After that he was gone. You cannot allow politeness or anything else to let these things go. Your life depends on it.

    • Posted March 3, 2018 at 1:14 pm | Permalink

      It’s simple human behavior. No matter what the task or the amount of practice, we will screw some percentage of the time. Practice and diligence simply serve to lower the rate of screwing up. It’s an acceptable risk when driving a car. Handling firearms, on the other hand, is an unacceptable risk for the person doing it and those around them.

  11. Heather Hastie
    Posted March 3, 2018 at 1:13 pm | Permalink

    It would be good if it were possible to interview those with injuries. I’d like to know if the big increase the week before the convention is NRA members going out shooting because they’re excited about the upcoming convention and/or they want a recent anecdote for conversation at the convention.

    The uptick following the convention might also have something to do with members getting geed up at the convention. That would need more weeks to be looked at to see if injuries fall away again.

    It might be that injury levels are higher in the few weeks before and after the convention, and drop away the rest of the year because of pre- and post-convention excitement. If that’s the case, it may help confirm a hypothesis around the convention and gun injuries.

    • Randall Schenck
      Posted March 3, 2018 at 1:25 pm | Permalink

      I really think it is simply a cutting down on the numbers. Many gun users are out of action, as we say, during the convention and just prior and after. The big show is in Dallas so lots of them come from around the country. Make a vacation of it maybe. So, less guns in action, less accidents. You will find nearly all of these folks have hand guns. Home safety is their biggest excuse and these hand guns travel well. However, hand guns are by far, the most dangerous of all guns. More accidents by far and anyone who might try to say they are not more dangerous, does not know guns.

      • Heather Hastie
        Posted March 3, 2018 at 1:42 pm | Permalink

        I think that probably is it too. It’s just a major problem more data aren’t collected. Of course, it’s the NRA stopping that so I suspect they know what the results show. They have made a lot of claims, and you can bet they investigated the issues themselves. When they found the data didn’t back up their claims, they moved to stop investigation and any related funding.

        • Mark Joseph
          Posted March 3, 2018 at 3:36 pm | Permalink

          +1 (well, that’s the standard comment. Actually, more like +1,000).

  12. Posted March 3, 2018 at 1:41 pm | Permalink

    You’ll find no bigger advocate for gun control (for gun elimination, in fact) than me, but I find the Jena and Olenski study to be very unconvincing. Two main points about their analysis:

    (1) A p-value of 0.004 is very modest given the sample sizes that they were working with. In a laboratory, natural sciences setting, our sample sizes generally range from maybe 30 to 300. In this context, a p < 0.05 can often be considered evidence of a nontrivial effect.

    But the Jena and Olenski data contain more than 70 million observations. That's 5 orders of magnitude higher, meaning they have far more ability to detect a nontrivial effect. Yet this only produces a p-value one order of magnitude lower than the common cutoff.

    (2) This is unsurprising because the number of events (firearm injuries) is so small: between 1.2 and 1.5 out of 100,000. That means out of their 70 million observations, only a little more than 1000 are events of interest. With such sparse data, the traditional analytical route Jena and Olenski take is doomed from the start. They should have realized something was seriously wrong when they performed their adjusted analysis (adjusting for age, sex, etc.): the appropriate models didn't converge (because of this sparsity!). Instead, they just used a clearly incorrect model for the covariate adjustment.

    In short, I find the study quantitatively unconvincing.

    • mikeyc
      Posted March 3, 2018 at 1:52 pm | Permalink

      Yes. The effect size is so small that it will be sensitive to small differences in reported injuries that can’t be captured in their covariate analysis. I’m skeptical of these results too.

    • Jay
      Posted March 3, 2018 at 6:54 pm | Permalink

      Some of the covariates they attempted to control for — age and sex — don’t even make sense. A covariate cannot confound a relationship unless it is correlated with both the outcome and the predictor of interest. Their predictor, week of the year, does not correlate with age or sex. Had they omitted these subject-level covariates, they could have conducted their main using an approach appropriate for count data, and thereby avoid the convergence problem. Instead, they shot themselves in the foot by including useless subject-level covariates, which forced them to analyze the data at the subject level.

      • ThyroidPlanet
        Posted March 3, 2018 at 7:11 pm | Permalink

        This is interesting

        But can you explain that at a bit lower level?

        Asking for a friend.

        • Posted March 3, 2018 at 8:39 pm | Permalink

          Jay makes an excellent point. The authors are basically trying to model the *probability* of a firearm injury for an individual as a function of week of the year (i.e. does this change on weeks of an NRA convention?). So they add variables like age and sex into the model because they have these data. I suppose it’s conceivable that age or sex could interact with week of the year if, say, the NRA conventions are attended mostly by men under the age of 25.

          But trying to model this probability of firearm injury for an individual is what forces the data to be so sparse (1.2 events out of 100,000), and this destroys the credibility of their models. Instead, as Jay suggests, they could have simply modelled the *number* of firearm injuries as a function of week of the year. This would avoid the sparsity issue since the response variable would now consist of counts (e.g. 10 events per week), instead of tiny probabilities indistinguishable from zero (like 1.2 out of 100,000). Moreover, this isn’t an individual-level variable, so no need to control for sex or age, etc.

          • ThyroidPlanet
            Posted March 4, 2018 at 9:45 am | Permalink

            Thanks … says my friend!

  13. ThyroidPlanet
    Posted March 3, 2018 at 1:48 pm | Permalink

    1. The convention dates error bars are too wide (for me, a nobody) as 95% CI – I’d want to use standard error of the mean, which will shrink it, but I thought physicists only accept … 5 sigma data?… suggestive but I’d say more work is needed.

    • ThyroidPlanet
      Posted March 3, 2018 at 1:49 pm | Permalink

      Oh, I had a 2. and 3. but you know. DELETE.

  14. Jay
    Posted March 3, 2018 at 4:01 pm | Permalink

    Here’s something I never thought I’d say: the NRA spokesman is correct. The supplementary materials reveal the p-hacking. When the authors go out 1 more week in either direction (ie, ±4 weeks), the p-value becomes 0.02, which is unimpressive, especially in light of the enormous sample size. The authors have reported the more-impressive ±3 week data, despite it being less reliable, as it is based on fewer observations.

    • Posted March 3, 2018 at 4:07 pm | Permalink

      I would expect gun nuts’ accidents to increase just before and after their NRA show. Before the show they are thinking more and more about their guns, deciding what to wear, who they are going with, etc. After, they come home with new guns, excitement, etc. so they’re more likely to spend time playing with their toys.

      • Jay
        Posted March 3, 2018 at 4:31 pm | Permalink

        Had the authors precisely defined how long “just before after” is, committed to their definition and method of analysis in a publically registered protocol before obtaining their data, I might actually believe that.

  15. gravelinspector-Aidan
    Posted March 4, 2018 at 2:58 am | Permalink

    The uninsured population

    One of the most bizarre phrases that can be truthfully used about any modern country.


%d bloggers like this: