My boyfriendof 2 years and finance/babyfather of 1 thinks he will do better without me and the kids(2)?

he's says we are holding him back and he doesn't want to hold me back. but i dont have anyone but him and the kids. NOw he wants to leave me because he was incarcerated and thinks i cheated on him. But to be honest with you all i really didn't. I don't know y he's being so insecure but i don't know what to do. Olease help!!!!