My 4-year-old step son said he didnt want me to go on vacation?
Of course. A 4 yr old has little concept of time and has no idea how long next year even is. He may, though, want time alone with his bio Dad as he gets older. That is normal and you should not take it as an insult. Let his relationship with you grow naturally and do not stew over it. It sounds like you are doing a good job so far!