my friend had one in his backpack this summer, on his way to the store to try and find air cartridges. He saw a guy he knew on the street, and mentioned what he was up to, the guy said he had some old cartridges at his house. So my friend opened his pack to show the gun (without taking it out) to make sure they were talking about the same thing (same kind of co2 cartridges). Then they kept walking. Like 2 minutes later, cop cars drive up on the curb, cops jump up with their guns out, they make my friend lie down, cuff him, etc. Apparently, when he opened his pack, some nosy motherfucker saw the gun, thought it was real and called the cops. So my friend got busted downtown in front of hundreds of people, and the cops were pissed when they found out it was a pellet gun, and gave him a ticket anyway, for "having played with an air gun in a public place".
So I guess it's technically not illegal, but it can still be a hassle because guns freak people out.