Bad Dogs or Bad Data?
Data pitfalls we use when we talk about pit bulls
A bit about bullies
I recently ended up in a Twitter thread in which someone compared pit bulls to assault weapons, and implied that saying "no bad dogs, just bad owners", is the same as saying "guns don't kill people, people kill people". I should have known from the start that I was walking into an illogical conversation, but here I am still thinking about it days later.
They cited bite statistics and common tropes about pit bulls - ones I've heard many times being a pit bull owner myself. There is an entire subreddit devoted to the idea that pit bulls should be removed from society entirely. My issue isn't (entirely) the opinion - people are entitled to feel however they want about dogs, breeds, and the like. Don't like pit bulls? That's fine with me. But there is only one thing I hate more than bad opinions, and that's bad data to back them up.
If you Google "dog bites by breed", you'll find harrowing numbers -- "Pit bulls account for 65% of dog bites in the US". While the number may vary year over year, the sentiment is the same: Pit bulls are dangerous, and the data backs it up!
There's just one problem. It doesn't.
Bad data and the bias that it comes from is the cause of many of our societal disagreements, our conspiracy theories, and the opinions that we defend. More important than being able to cite a number, however, is having the ability to understand where it came from and whether or not it means anything. This article isn't about whether pit bulls are good or bad - it's about the data that people use to make laws about breeds. Being able to understand data pitfalls is bigger than this issue alone.
So let's take a look at data pitfalls, how they relate to pit bulls, and what you can do to ask better questions about data more broadly to understand the full picture.
A caveat. I own a pit bull. With that, I too will have some amount of bias in this conversation. I will do my best to remain as objective as possible. I will be the first to tell you that my dog came with a complicated past and her own set of issues. Pit bulls, like any breed, have pros and cons, and each individual should weigh all of the potential issues before adopting a pet.
What is a Pit Bull?
If you answered "a dog breed", believe it or not, you're not entirely correct. "Pit bull" is a broad term to describe at least four pedigreed breeds of dogs: the American Pit Bull Terrier, the American Staffordshire Terrier, the Staffordshire Bull Terrier, and the American Bully.
In many cases, however, an even more broad definition is applied that goes beyond genealogy all together. In the legal case State of Ohio v. Anderson, legal precedent was established that "any dog owner of ordinary intelligence" could visually identify a pit bull, meaning that an individuals visual assessment of a breed would hold legal weight in court. These precedents have meant that, in the eyes of the law and henceforth in the data collected, any dog resembling the physical characteristics of a pit bull (namely a stocky build and square head) can be characterized as a pit bull regardless of true genealogy. This, despite the fact that at least 25 other breeds share these general characteristics.
In a practical sense, this means that the definition of a pit bull can be liberally applied in sampling whereas the same generalizations may not extend to other breeds. In a case in which a mixed breed dog consisting of possibly a dozen or more breeds has bitten someone, should the dog bear any physical characteristics of a pit bull (which is likely, given that these traits are shared by some 25+ breeds), it can be labeled as a "pit bull" bite.
For example, in the city of Denver, their laws state that "Pit bull-type dogs are defined as any dog displaying a majority of physical traits of any one or more of the specific breeds mentioned above, or any dog exhibiting those distinguishing (physical) characteristics".
This is an example of what is likely to lead to sampling bias, in which some members of a population are systematically more likely to be selected in a sample than others. This leads to a sample that may be unrepresentative of the true population. Because the label of "pit bull" can be legally applied so liberally, it is more difficult to assess the true bite risk and statistics of, say, a pure bred American Pit Bull Terrier than it may be to assess the relative bite risk of a more physically distinguishable breed like a Labrador.
A brief bully history
It's impossible to talk about the data surrounding pit bulls without discussing the dark history of breed specific legislation in the United States.
As of 2018, there were roughly 1000 communities in the US that had some form of breed-specific legislation (BSL) or breed bans in place. These laws prevent the ownership of "dangerous" breeds, typically honing in on ownership of pit bulls specifically.
While the enforcement of these laws is typically entirely subjective — the previous enforcement of the Denver pit bull ban relied on law enforcement officers, not veterinarians, to decide if a dog had "majority pit bull traits" — the history of these laws is even worse.
In her recent Animal Law article, "The Black Man's Dog: The Social Context of Breed Specific Legislation", Harvard fellow Ann Linder takes a deeper look at the origins of BSL in the United States. In her in-depth analysis, she finds that the driving (and often blatant) motivation for enacting BSL is not the inherent threat imposed by pit bulls, but rather the "types of people" tending to own them.
The following are quotes taken directly from policy makers across the US in the implementation of breed bans:
"We don't want those people here" - Aurora CO city council member Bob Fitzgerald
"I want gangs out of Lancester. I want to make it uncomfortable for them to be here. Anything they like, I want to take it away from them. I want to deliberately harass them. If they have a Pit Bull, they may as well put a sign on their head saying 'come get me'" - CA Mayor Rex Parris
"Oh, you bet if I could, I would ban the owners too. By banning the dogs, we lose the violent behavior that comes with the owners. It's not about a fact, it's about a feeling." - Aurora CO city councilmember Molly Market
The Ohio Supreme Court stated that pit bulls are "dangerous" in part because they are "found largely in urban settings".
Pit bulls, more than any other dog breed, are owned by Black and Latino Americans. "Those people" and their choice of dog ownership find themselves on the frontlines of a brand of redlining in the US that began in the 1980s and continues still today. The sentiment is clear - ban the dogs, ban the people who come with them.
The racial motivations become even clearer as Linder takes a deeper look at the demographic and bite statistic data, breaking down a study in which participants were told a breed of dog and asked to describe the type of owner that they pictured. Despite having relatively similar per capita bite statistics, German Shepherds very rarely find themselves included in the same breed bans that pit bulls do. Linder states "[German Shepherds (GSD)] were widely considered one of the most dangerous breeds of dog [...] yet they are not commonly included in breed-specific bans today." She continues "The study showed that 82% of participants considered a white person to be the most likely owner of a GSD, while for pit bulls that number was just 34%. These results, though not conclusive, may provide insight as to why German shepherds, which were cited for the majority of dog bites for decades, are generally not included in bans against dangerous breeds today."
Confirmation bias describes the type of statistical bias in which we only accept data which aligns with our prior view of the world. As our lawmakers associate pit bulls with gangs, drugs, and violence, it stands to reason that confirmation bias will play a role in any data collected regarding their temperament, as well as the drafting and enforcement of the laws written around them. This bias has historically exacerbated public perception of the dogs, the subsequent data collected on them, and the policy decisions regarding their permissiveness in society.
But are they dangerous?
"Pit bulls account for 65% of dog bites" is a hell of a statistic. As stated earlier, however, it doesn't paint the whole picture.
In order to understand if a "pit bull" is inherently a dangerous dog, we have to look not at the raw rate of bite statistics by breed, but rather a more informative figure of the estimated bite rate per 100,000 dogs. Let's step through some basic arithmetic to get an idea of this true figure.
How many pitties?
To know the per-capita bite rate of dog breeds, we have to first estimate the population of breeds in the US. As we went into earlier, it's actually impossible to know how many "pit bulls" there are in the US, because a "pit bull" isn't a breed. It can be any of four main breeds (American Pit Bull Terrier, American Staffordshire Terrier, Staffordshire Bull Terrier, and American Bully) as well as any dog with "pit bull" traits, shared by over 25 known breeds.
As of 2018, the World Animal Foundation estimated that there were roughly 4.5 million pit bulls in the US, in addition to another 4.5 million pit bull mixes that could be legally described as pit bulls. This number is a lower bound on the total population, as there are an uncountable number of pit bulls subject to backyard breeding. At the time of the 9 million pit bull estimate, they put the total number of dogs in American households at 76.8 million, making pit bulls roughly 11.7% of the dog population in the US.
For the sake of illustration, we will compare this to Rottweilers, Huskies, and Mastiffs, making up 1.8%, 1.3%, and 1% of the US dog population respectively.
The breed vs. bite breakdown
More important than the overall bite rate, in determining the actual risk of being bitten by a particular dog, is the per capita bites rate. We can estimate this by looking at the number of fatal dog bites in the US with respect to the overall population estimate by breed.
This paints a very different picture when we adjust for population. Yes, pit bulls account for more bites than any other breed, but after adjusting for the mixes and dogs that can be classified as pit bulls we see that they make up a much larger population than the other dogs here. This places the bite count into perspective - when squared up with the population, a random pit bull is more likely than a husky or mastiff to be responsible for a fatal bite (by a factor of 2, not 20 as the raw data might suggest), but less likely than a Rottweiler.
What makes a dog bite?
If the goal is to understand how likely you are personally to be bitten by a random dog based on their breed, the per-capita bite rate still doesn't paint the full picture. Taking that rate at face value would imply that all dogs are being assessed under the same conditions. This is an example of selection bias, in which proper randomization isn't achieved, meaning that the sample isn't necessarily representative of the population or isn’t measuring samples under the same conditions.
The issue is that all dogs are not treated equally, and this ends up playing a huge role in the factors that lead to a dog bite.
A ten-year study published by the American Veterinary Medical Association found that breed-specific legislation had no impact on the prevalence of dog bites.
So what did make a dog more likely bite?
Isolation from positive human interaction
Abuse and neglect
Being left unsupervised with children or vulnerable adults
Abandoned or trapped in an enclosed environment
As the study states, "these conditions potentially predispose dogs to enhanced territorial, protective, and defensive behaviors toward stimuli that occur commonly in everyday life. The most preventable incidents involved very young children left alone with dogs to whom they were unfamiliar or toddlers allowed to wander off and encounter unfamiliar dogs." This study found that 87% of fatal dog bites had one or more "preventable" factor.
What does this have to do with pit bulls? In part because of their historic reputation, pit bulls spend longer than any other breed in shelters, accounting for 20-30% of shelter dogs in the US. While many shelters in the US work hard to ensure animals get the best possible care, it isn't hard to see the connection between shelter life and the above listed causes of bite tendencies - isolation, neglect, and left in enclosed environments - all leading any dog to be more likely to bite in the future.
It's a self-fulfilling prophecy. Because they are believed to possess dangerous tendencies, pit bulls are left in shelters, leading to heightened isolation and increasing their risk to develop bad temperament.
Who’s a good dog?
The American Temperament Test Society is a "national not-for-profit organization for the promotion of uniform temperament evaluation of purebred and spayed/neutered mixed breed dogs". They conduct a temperament test across thousands of randomly sampled dogs of all breeds in order to accurately assess a breed's inherent temperament under a controlled environment.
The twelve minute test consists of several sections, during all of which the dog must be without any intervention from their owner. The subtests include situations such as a neutral stranger, a friendly stranger, hidden noises including gunshots, and strange objects like umbrellas. A dog is considered as passing as long as there are no instances of "unprovoked aggression, panic without recovery, or strong avoidance".
Of the 36,333 dogs tested, a total of 84% passed the test. Of the 960 American Pit Bull Terriers tested, 87.4% passed.
That is a better result than 73% of the other breeds (with at least 20 dogs in that breed in the test results), including German Shepherds, Rottweilers, Boxers, Dobermans, and America's Sweetheart — the Golden Retriever.
Okay, so what?
As I said earlier, the goal of this isn't to convince you to go out and buy a pit bull. Any dog will have different predispositions, triggers, manners, and tendencies. Rescue dogs - account for the vast majority of pits - will often come with complicated pasts that can predispose them to difficulties later in life. Every dog owner should consider both the dog and the breed they are adopting and whether or not they can responsibly care for it.
The goal of this was to show a single example in which the data doesn't paint the full picture. Data is not neutral - it is as flawed as the humans who collect and synthesize it. It will contain the same biases possessed by both those who generate it and those who consume it. Understanding the types of statistical bias present when looking at data enables us all to have better informed conversations, whether it be about dogs, vaccines, guns, or whether or not you should eat gluten.
Dog tax if you made it this far.
Awesome article!!!