Try to actively think about engaging your core muscles , everytime you see someone doing a handstand its all about balance and core strength , you will have to get used to get away from the wall too , as long as you're not on a hard surface you'll be fine