How do I make my new parakeet feel at home?

I just bought a parakeet from a pet store and he is terrified. I have his cage all set up very nicely with everything he needs. Should I just let him be for a few days and then let him get used to my hand in the cage?

I stuck my hand in there once to see if he would step on my finger and he spazed out, flew out of the cage and it tok me forever before I caught him. Now I think he hates me.