You need to prime the remaining wallpaper glue. The primer will seal the wallpaper glue and stop any problems before they have a chance to start. I would use a quick drying oil based primer for this job. Kilz Oil Base or Zinsser Cover Stain will work great.
It is best to plan on priming all of your walls. Then do any repairs that are necessary. Now you can apply a texture, prime the new texture then paint as usual.