I'm sorry you felt that you had to shorten your plans, if the neighbors can handle one week they could handle two weeks just as easily but the decision is yours to make of course.
You must be flying out on my birthday since it is the only Friday the 13th of the whole year, don't let it bother you, it is just another day of the year, nothing bad is going to happen.
He will be fine for a week, heck he will be fine for two weeks too, I bet it is you that will be missing him more than him missing you, am I right or what!!!