If it is stuck now there is no need to strip off this primer and paint. To test the painted areas use a strip of masking tape; press the masking tape to the surface, rub the tape for full contact, then quickly remove or rip the tape from the surface. If excessive amounts of paint is removed, down to the raw plaster, then there could be a problem in the future.
Most likely you won't have any problems. Make sure to use a good primer on the plaster walls before painting. Also, a quick sanding on the over-spray areas is a good idea.