Is it time to stop swaddling my son?

yes stop swaddling him. If his arms are swaddled he cant get them loose im guessing his face could be in the mattress, id be nervous too. I stopped swaddling around 4 1/2 months but i had the swaddles with the velcro on them, so i would put my babys legs in the swaddle and then wrap the rest under his arms so they were free. I hope that makes sense... I did that till about 5months then just stopped. He was never a good sleeper so like you he woke every two hours. Good luck