I’m sort of half-and-half about this one.
While it’s true that some Arduboy games use the serial connection to do screen mirroring, debugging and to act as a crude keyboard or controller,
most actual games don’t need the serial functionality to work so not having the USB would only be a hindrance in a few special cases (at a guess, somewhere between 1-3% of games).
Perhaps it would be best to leave it off for now and work on adding it in a version 2.0 or something.
To be honest the only one of the three I would miss is debugging,
but the question is: how many people would actually be trying to develop Arduboy games using it?
If the point is for the hat to just act as a way to play Arduboy games on the Pokitto,
then it doesn’t really need the USB port to fulfill that purpose.