Always Ask for Inbound Connections to Specific Port

Is there a way to configure the firewall to always prompt the user to allow or block incoming connections on a specific port? I set up an application rule to ask for connections made to the port, but the problem is that when I allow one connection, it allows all subsequent connections even if they’re from a different IP. Is there any way to have it only remember my choice for that specific IP (or better, session) without making a rule for each IP (since IPs change over time)?

Try setting the alert frequency to Very High.

Tried this out and it seems to be doing what I want, haven’t noticed any problems. Much more simple solution than I expected. Appreciated!

Just to note, very high is for each specific IP address and port, while setting the alert frequency to high is for each specific port number.