Hi Kevin,
DC motors are sometimes just that simple.
One thing to look for while you have the brushes out, look for play in the shaft at the bearings. [up and down play]
If there is any play, this may be the cause of your brushes wearing out and at that point you should look for a replacement
or send your motor to a rebuilder.
Good Luck
Nick-