My 3 and a half year old wont go to sleep without me its going to break my husband and i up.?

My son is 3 and a half years old his my second child and I'm pregnant with my 3rd child. He cry's and cry's if I don't sit with him beside his bed or let him fall asleep next to me on the lounge or in my bed. He carries on when my husband tries to get him to go to sleep. He just wants me. Once he is asleep my husband or I will put him in his own bed in his own room (across from ours) but EVERY night he gets into our bed when we realise we put him back but he always comes back over and over sometimes I'm so tired I don't even know his climbed in with us until the morning. My husband gets angry about all of it and we fight always about our sons sleeping and not sleeping situation I don't know what to do. I don't want a new born baby up every 4 hours as well as my 3 and a half year old son whom will be 4 years old when the baby is born. ANY ADVICE I don't want him to be crying and upset every night but can't keep being the only parent that deals with him.