Getting disconnected prevents reconnection to the game servers?

If I get disconnected midmatch, there's an issue where even if I sign out and sign back in using the Xbox button on the controller, I still can't log back onto VF5 xbox live because it keeps telling me that I need to be connected to XBL, which I am. The only fix I have for it so far is to just close the game and restart it. Is there some other way?

 

Turn the console off and then on again