There are perfectly legitimate reasons for explorer.exe wanting to access the internet, there are, however, also exploits that can make use of this process.
Please understand that explorer.exe is actually a lot more than that simple file manager you interface with. Explorer.exe, for all intents and purposes, is the Windows shell. It contains the code for a number of the widgets on your desktop.
Also, consider some or the purposes to which this application may be put. Accessing FTP sites, Network shares, WEBDAV and many more.
The situation regarding exploits and explorer.exe are more complicated. One simple check you can do, however, is to check the path to the executable and also check the files details. The file itself should be in your %system%\windows folder. Typically this will be:
If it's any where else I would be very suspect.
The buffer overflow problem, once again, may or may not be a security issue. Clearly, the first thing to do is make sure your system is fully patched and check for any nasties.
Another cause of buffer overflow has been attributed to corrupt codecs. If you have installed a codec pack, such as K-lite, it may be worth while uninstalling it and seeing if the problem goes away.