Good work takes time!
I run into this often when I am bidding a job and I mention it will take 3-5 days to complete an exterior of a smallish "cracker-box" type home. The owner's eye bulge and they say "That long? I had this painted three years ago and it only took 2 days!"
My response is this…
"If I painted it three years ago, you wouldn't need a paint job now." Quality materials and proper preparation is the difference between a good paint job and a job that will not last."
After powerwashing the house needs to dry well before it can be scraped and/or sanded where necessary. Then caulked properly then primed. Another day to dry and then paint. If two coats are necessary, add yet another dry day.
You paid for the job to be done, you shouldn't complain that they are giving it the attention it deserves. This will pay off in dividends if you get 10-20 years before needing another repaint!
If you are unhappy with the quality of the end result, address that with the handyman and then you can bring up the time factor.
I hope this helps keep perspective.