Some BMX'ers file a little off the tube spacer that is in the middle of Profile-style cranks. This means when you tighten the cranks, there is more pressure on the bearings and they don't spin as freely.
Obviously, the whole point of the tube spacer is to not allow too much pressure on the bearings, so I suppose you risk damaging the bearings.
The little balls themselves are quite hard but the races are not so much. The little spacer (in a perfect world) will match the space between the bearing races in the frame. When you install the bearings with the spacer there should be NO space and that little tube should fit snugly between the bearings which means that when you tighten up the crank arms that spacer will not allow the inner races to move out of alignment with the outer races and the cranks should (in a perfect world) spin freely with no side loading on the bearings. This of course is not a perfect world and wile the cranks spin decent, usually there is some amount of binding.
Filing that little tube will allow a load on the bearings that they are NOT designed to take and deal with poorly. You WILL damage the bearings. Now this is a group of people (generally) who do not expect parts to last forever we generally do damage our bikes and parts, sometimes on purpose. If you remove a little material then you may cause just a little load and the bearings may last quite some time before they detonate and you may be happy with that. I pulled a bearing out for maintenance a while ago and the outer race was cracked through but there was no symptoms.
Long and short, it is bad for bearings but is bearing life something you can trade off?
Keep in mind, if you take off too much of that little tube you will bust the bearings in no time.