Training mode will allow all alerts and create an allow rule for anything run. You will need to change the firewall to custom policy. This will alert you for any Internet attempt if a rule is not present.
Before you switch modes go through your rules and delete the allow rules that training mode created.