Since the "fan" started the issue by attempting a kick at the keeper, I am not sure why the referee thought a red card was in order.
Since play was at the other end of the field I'm thinking the sideline as well as the main referee was not watching the fan come onto the field of play.