What would you do in this situation?

Even if you don't forget about him-act like you have. He has a bit of arrogance, and the thing with his friends sucks. If you Absolutely must be involved with him, make him work. Make him do the pursuing and at the slightest hint of that sh*t with his friends, drop him.