The same argument was put up for Airsoft. Cars kill people? Why can't we ban cars? Simple - it's a part of living. Airsoft isnt' really a 'majority sport' and it also resembles a real fire arm that shoots plastic bbs at high velocity. That already gives the public the hibijibis. Should the public feel safe that we're shooting nerf guns with facemasks? Not really because they're not wearing any face protection. What if it damages my face? They will think.
Servulus said to do a survey, but the survey can go pear shape if one person goes and reports it to the news that 'kids have to wear masks' to play with nerf guns. Remember, it only takes ONE person to screw everything over for everybody. So just keep it as safety glasses only.
If you ever studied FTV and new media or have a glympse of what they do, the news can twist just about ANYTHING. You say something positive, and they'll cut a sentence from your little 'interview' and BAM make it sound negative. For the media, it's SO easy to do.
You can wear facemasks if it's a private indoor war, go crazy, no ones going to really bother you about it.