Hand guns are so, so much more common in crime, rifles are barely a blip on the map. Also, handguns have almost no use other than killing humans/sport. (You can argue that they can offer some sort of protection from wild animals when you're hiking, by scaring them away with noise... I can't really think of much else)
Semi automatic rifles cover the gamut of utility. They're not JUST for killing people and/or sport. Every reason you could legitimately need a gun for, the broad category "semi auto rifle" covers, so banning them has a disproportionate impact to people who use them legally and as tools vs banning handguns.
If people seriously want to make a dent in gun crime/accidental deaths/suicide we need to look at handguns, but they're not scary looking enough so there's no clout. Instead we get stupid laws that try to ban scary looking black guns or limit magazine sizes. Pisses off gun owners that know it's useless and doesn't actually get at anything that can make a difference. It's all theater.
Read the federalist papers if you want to understand the 2nd amendment better. You're just as wrong as the people who like to say that the 2nd amendment was just to protect having a militia.