Ok, original author of this thread here...
I just installed COMODO v5 (before that I had v4), and tried doing the speedtest.net test again. Pings were back to normal.
Whatever it was in v4 that was causing the problem seems to have been solved with v5!
Version 5 looks awesome btw, so I'm pretty happy to be using it!

IMPORTANT UPDATE: I found out that it's a combination of COMODO Firewall and ESET NOD32 Antivirus that's causing the problem.
Install either one of them separately, and everything is fine! But once you install both of them concurrently you'll get this issue with very high 500+ ms pings!
I figured this out completely by accident after trying ESET Smart Security 4 (Firewall + Antivirus bundle) because I wanted a good firewall to replace COMODO. Using it I had no issues with pings, everything was running smoothly.
Today I decided to try out COMODO Firewall v5 to see if this issue was resolved with the new version (since preciously I was using v4), and also because I really like COMODO's Firewall and wanted to use it again.
Anyway long story short I had to do a full uninstall of ESET Smart Security 4 (uninstalling both antivirus & firewall because you can't uninstall them separately with ESET Smart Security 4). And then I installed COMODO Firewall v5 and did the speedtest.net test without having any anti-virus software installed. Pings were perfect (very low), and so I proceeded to write the above message saying the issue has been resolved with v5. But as soon as I installed NOD32 antivirus again, and did the speedtest.net test again, I was once more getting pings of 500+ ms.
I uninstalled ESET NOD32, and the pings went back to normal! I then tried re-installing it, confirmed that the pings were once again very high, and then uninstalled COMODO Firewall v5. After that again the pings went back to normal.
So the conclusion is that there is some kind of unwanted interaction between these two programs where if you have both of them installed at the same time you will get very high pings! Uninstall either of them while leaving the other and the problem is resolved.
I'm really not an expert either of these two programs. All I know is what I described above. Any knowledgeable people on here have any ideas as to what may be causing this issue when these two programs are installed side by side?