So I created a new ruleset for applications that are communicating locally by TCP/IP, but should not connect to the internet.
The ruleset, if manually assigned, works just fine, but it’s not showing up as a choice in the firewall alert dialog, whenever a new app tries to make a connection.
EDIT: I don’t really have a clue why, but now it’s showing up. I even tried restarting CIS without success, but now, a little while later, I’m getting dialogs with my new ruleset.
So I suppose this topic can be ignored/deleted.