It could be from the washing, a little too aggressive, or just the wood. Sometimes the way the grain is cut can produce small flakes.
Go ahead and sand these out. The best grit depends on your sander, palm sander or something more heavy duty. Start with 80 grit and see how it goes. You can always change the grit, courser or finer, depending on the results. You don't want deep groves but a fairly fine smooth surface.