How the Comodo Sandbox works - technical discussion

[Guest]

[Endymion]

Would 3xist, Egemen, or Ronny care to comment as to why CIS Beta fails CLT so badly when using the Sandbox? It is making me a little paranoid. I'm thinking of using CIS with Sandboxing disabled. With it disabled, CIS passes CLT a lot better. I can't seem to get an answer on this subject. I hope one of you will relieve me of my worries.

http://forums.comodo.com/cis4-beta-testing/leak-tests-understanding-results-t50776.0.html;msg362832#msg362832

Many tests are fooled by the sandbox and reported with "vulnerable" result whereas they  should have been reported as "Protected".
Leaktests fooled by File virtualization

5. Invasion: Runner   Vulnerable
Tries to replace default browser executable with a different one to exploit an existing firewall policy.
Actual Result: it drops the actual/true browser executable into sandbox subfolder
8. Invasion: FileDrop   Vulnerable
Tries to drop test.exe into %windir%\system and %windir%\system32.
Actual result: It drop test executables into sandbox subfolder

Leaktests fooled by Registry virtualization

27. Hijacking: WinlogonNotify   Vulnerable
Actual result: Redirected to registry sandbox. Fail to write real HKEY_LOCAL_MACHINE\Software\Microsoft\Windows NT\CurrentVersion\Winlogon\Notify\crypt32chain\DllName

28. Hijacking: Userinit   Vulnerable
Actual result: Redirected to registry sandbox. Fail to write real HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Winlogon\Userinit

29. Hijacking: UIHost   Vulnerable
Actual result: Redirected to registry sandbox. Fail to write real HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Winlogon\UIHost

30. Hijacking: SupersedeServiceDll   Vulnerable
Actual result: Redirected to registry sandbox. Fail to write real HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\services\sens\parameters\servicedll

31. Hijacking: StartupPrograms   Vulnerable
Actual result: Redirected to registry sandbox. Fail to write real HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Terminal Server\Wds\rdpwd\StartupPrograms

32. Hijacking: ChangeDebuggerPath   Vulnerable
Actual result: Redirected to registry sandbox. Fail to write real HKEY_LOCAL_MACHINE\Software\Microsoft\Windows NT\CurrentVersion\AeDebug

[AyeAyeCaptain]

A very informative thread thanks to everyones input. 

[L.A.R. Grizzly]

Many tests are fooled by the sandbox and reported with "vulnerable" result whereas they  should have been reported as "Protected".

Thanks for helping me understand the CLT failures. Maybe Comodo needs to update Leak Tests to properly evaluate sandboxing!

[Endymion]

Thanks for helping me understand the CLT failures. Maybe Comodo needs to update Leak Tests to properly evaluate sandboxing!

You're welcome.

Indeed checking the results it is awkward so an updated test-suite would be welcome.

There are some tests which might fail. the ones I mentioned are those I get predictable results and are incorrectly reported as Vulnerable.

eg: Coat sometimes fail here as well but it looks that specifically testing Coat after a reboot get a protected result.

Appinit and ActiveDesktop might get reported as Vulnerable or Protected when CLT is sandboxed. It looks like that when they reported as Vulnerable they are trapped by the sandbox.

Usually running Appinit twice will cause the 1st run to be protected and the 2nd to be vulnerable (but virtualized in the sandbox).
ActiveDesktop might trigger a COM alert for {75048700-EF1F-11D0-9888-006097DEACF9} GUID.

Blocking that alert will have the test to abort without attempting to write to the registry. The test result will be protected
Allowing that COM alert would get a "vulnerable" result though the real registry will be untouched and the system is Protected.

[andrewuaic]

maybe it could be possible to use the sandbox to annalyze in real time what sandboxed applications are accessing and could warn if some application has malware behaviour

[mouse1]

Thanks, that's pretty much what I had concluded after playing with it for a bit. I too find that files get bounced out of 'My Safe files', and agree re notification. It seems that (at least some) files that are trusted or windows system in the computer security policy are not trusted as far as the sandbox goes. I've done a bug report.

Best wishes

Mouse

Those entries list  executables which have been sandboxed at least once.
There are some executables which are removed from "My safe files" generally after a reboot.
If this is what happens there is no workaround AFAIK.

If an executable cannot be added to "My Safe files" the user ought to be notified that is already safe/whitelisted .

It looks like that even trusted application will run in the sandbox if lauched by a sandboxed app.

Usually installer applications will trigger a D+ elevation alert. Allowing such alerts will have the installer run outside the sandbox.
If an installer does not trigger such alert in most cases it is usually safelisted.
If "Automatically detect installer/updaters and run them outside the sandbox" is unticked all installer will run in the sandbox.

[mouse1]

Have prepared the summary below for myself, wiith some help, thanks, from Ronny, Endymion and posts in a moderators topic. This is obviously just the opinion of a volunteer moderator on how it works, not an official Comodo description. Thought it might help others, though as you can see some areas are I'm just guessing.

All updates and corrections gratefully received - they will be reflected in edits to this post.

Declared purpose
To allow software whose security status cannot be immediately identified to be automatically run safely without alerts while it is investigated (eg via CIMA) by Comodo

Aspects of this purpose intentionally not yet implemented
- installers do not work if sandboxed (?may never be implemented - too difficult in 64 bit?)
- automatic investigation by Comodo is not enabled
- automatic sandboxing does not include virtualisation of program data

Components
The sandbox consists of:

• running software as a Windows ‘job’ with limited OS user account privileges and OS job limits. Referred to below as restricted privileges. (See appended descriptions of privileges).
• file system virtualisation. In C:\Sandbox
• Registry virtualisation. In Hkey_local_machine\system\sandbox\[app name]\[app created keys]
• A special set of D+ restrictions. Technically, automatically sandboxed software can write to the disk but it cannot cannot a) write to (ie infect) existing protected files or registry keys b) key log or screen grab, set windows hooks, access protected COM interfaces or access non-sandboxed applications in memory
  

Rules
Known safe software, including know safe installations software, is not sandboxed. Definition of known safe = (signed or identified by hash) AND (not on local/cloud AV list) AND (on Comodo white list or in local my safe files)

Known software believed to be unsafe (on cloud av list) is put in sandbox with restricted privileges, then caught by CIS version 3 facilities when local virus database updated

Unknown software is automatically sandboxed, unless CIS thinks it’s an installation routine and "Automatically detect installer/updaters and run them outside the sandbox" is ticked. If CIS thinks its an unknown installation routine, or it asks for Windows admin privileges,  you will be asked (once only) whether you want to run it with elevated privileges, and if you say yes, the VENDOR and the file becomes fully trusted you receive no further warnings of any form for files from that vendor. (However the alerts allows you to make a system restore point). If you say no the installer runs in the sandbox, and the installed software is unlikely to work. I suppose CIS probably guesses whether it is an installer from the code signer (or maybe more likely from the file asking for admin privs?).

Any processes directly invoked by sandboxed software are sandboxed.

Any software can be manually sandboxed, including installation routines, but the latter must be run with ‘unrestricted’ (ie only slightly restricted) privileges, which can include virtualisation.

Processes
Automatic sandboxing.
This currently consists of, silently:

• running it with restricted privileges. Nothing is virtualised.
• Enforcing a special set of D+ restrictions
• Making a background check on it using CIMA (next version)
• If it looks OK taking it out of the sandbox

Manual sandboxing.
You can:

• Run a program, including an installation progam, once in sandbox by using ‘run a program in the sandbox’ (run-once sandboxing). No virtualisation is applied, but various levels of restricted privileges, with associated D+ restrictions, can be applied
• Run software, including an installation program, in the sandbox continuously by adding it in the programs in the Sandbox view (continuous sandboxing). Various levels of restricted privileges and virtualisation can optionally be applied. (D+ file/registry access restrictions are not applied if fully virtualised?).
• not sure what will happen re a CIMA check (in next version). Will it do one?

Unfortunately after running an installation program virtualised, the installed program will not be able to access its files. (This holds , whether the installed program is run within or outside the sandbox and whether it is located inside or outside the sandbox). Also, if a program is run sandboxed and virtualised (after normal installation) it may forget settings made while sandboxed, and may function abnormally in other ways because it cannot access some parts of its data. This appears to arise because virtualisation makes assumptions regarding where (under what directory root) program settings and user data is stored. This happens even with the lowest level of restriction - 'unrestricted' applied.

Un-sandboxing
You can take an app out of the sandbox:

• and treat it as trusted by putting it in My Safe Files then re-starting it before rebooting. Making it 'Trusted' or 'Windows System' in D+ does not seem to work yet, occasionally 'My Safe Files' may not work either, as some files seem to be autmatically removed from "My Safe Files" on reboot
• and treat it as a trusted installer, by defining it as an installer in the D+ Computer Security Policy
  
• and treat it as trusted automatically via the CIMA check (applies to manually sandboxed items?)
• to get rid of it by?

No sure what happens to virtualised program data when a program is unsandboxed - is it copied to the relevant Application Data, or My Docs directory?

GUI & logging (updated to cover release version 10/03)
Automatically sandboxed items are added to My Pending Files, along with other files submitted to Comodo for analysis. However my pending files provides no way of distinguishing between sandboxed pending files and unsandboxed pending files.

Re manual sandboxing, continuously sandboxed items appear in the ‘Programs in the sandbox’. Run-once sandboxed items are not displayed in 'Programs in the sandbox' or in 'My pending files'.

In both manual and automatic sandboxing, a log record is made when software is put in the sandbox, but not when it is taken out. Requests denied by MS job monitoring are not logged even in the Windows event file - on XP anyway - unless they result in an application crash or some such event. Requests denied by D+ due to sandboxing restrictions are logged.

Automatic sandboxing and manual run-once sandboxing can be seen as brown highlighted entries ('jobs') with restricted privileges (process properties: security tab and job tags) in process explorer. Not all brown entries are sandboxed - Chrome (hence CD) and IE tab processes already use MS jobs - you need to check the job and security restrictions to be sure - see appended screen shots for what to expect from automatic sandboxing. Not all sandboxed entries are marked brown with normal process explorer settings - you need to untick the other process highlighting rules to ensure that all sandboxed entries will be coloured brown.

All programs which have ever been sandboxed in any way, including those currently sandboxed, have keys under: Hkey_local_machine\system\sandbox\

[LaserWraith]

Have prepared the summary below for myself, wiith some help, thanks, from Ronny, Endymion and posts in a moderators topic. This is obviously just the opinion of a volunteer moderator on how it works, not an official Comodo description. Thought it might help others, though as you can see some areas are I'm just guessing.

[...]

Thank you mouse1 for writing that detailed explanation/hypothesis. 

I have a somewhat unrelated question though - I see, in your screenshots, some extra buttons.  What are those from? 


Curiosity killed the wraith, I know, but they look interesting. 


Anyway, you helped me understand the sandbox a bit better.  I'll soon update my CIS 4 review. 

[mouse1]

My pleasure. I'll take a look at your review later, sounds interesting.

If you are referring to the blue Dialog boxes, these are from Sysinternals (now microsoft) process explorer. They are just showing you the job limits and security provs for a default, autjomatically sandboxed process. The exact privs will be different depending on the sandboxing restriction level, I think.

Mouse

[ailef]

"Invasion: Runner" is not vulnerable on my system using the sandbox, i use windows 7 64 bit.

the final score was 260/340.
vulnerable to invasion : filedrop (1)
                   injection : setWinEventHook / setWindowsHookEX / services (3)
                   hijacking : userinit / supersedeservicedll / startup programs / appinitDlls (3)

[BigMike]

Hm, I know, that the sandbox is still heavy work in progress and I don't really know how to test, if it's doing what I expect it to do, but I've read the help file which states:

Sandboxed applications will not produce any Antivirus, Firewall or Defense+ Alerts.

Sorry, I don't know what's the catch - if I'm running a program in sandbox, because I'm not really sure if it's completely trustworthy - how should I come to a decision if I get no alerts??
If a program is sandboxed by CIS, because it's unknown and it doesn't work how I expect it afterwards, how could I realize, that it maybe doesn't work, because CIS denied something important if there's no alert??

With all due respect to the developers for making CIS more silent for the sake of unexperienced users - I don't think this is the right place to not show alerts!

[mouse1]

Hm, I know, that the sandbox is still heavy work in progress and I don't really know how to test, if it's doing what I expect it to do, but I've read the help file which states:Sorry, I don't know what's the catch - if I'm running a program in sandbox, because I'm not really sure if it's completely trustworthy - how should I come to a decision if I get no alerts??

Comodo will test the file in the background far more deeply than you can. If it passes these tests it will be unsandboxed.

If a program is sandboxed by CIS, because it's unknown and it doesn't work how I expect it afterwards, how could I realize, that it maybe doesn't work, because CIS denied something important if there's no alert??

You can't know for certain, unfortunately. You need to wait until its out of the sandbox to be sure, but the sandbox will get better at this I think until only badly coded programs and malware have significant requests are denied. [Edit: While its in the sandbox, some denied requests will be logged, possibly all, which helps a bit]. Many (like me) feel therefore that an explanatory alert on going into the sandbox is necessary, and logging (but not alerting) denied requests needs to be comprehensive.

See post by me (unofficial opinion) above in this topic for more info on how the sandbox is intended to work. (Bit techie but it may help). It is here.

[BigMike]

Comodo will test the file in the background far more deeply than you can. If it passes these tests it will be unsandboxed.

I've read, that Comodo will analyze this file - but if I put it manually in the sandbox it won't be automatically unsandboxed (at least I hope so). This part was about manually sandboxed programs.
I came across this idea because of another passage in the help file:

Comodo Internet Security allows you to run programs which are not previously added to the Sandbox inside the Sandbox on a one-off basis. This is helpful to test the behavior of new executables that you would have downloaded or the ad-hoc applications that you do not trust and not going to use often.

But I can't test the application if I get no feedback from CIS. So thanks for the hint with the logs. But I think I'll have to wait for the final to see if I'm happy with this method.

You need to wait until its out of the sandbox to be sure, but the sandbox will get better at this I think until only badly coded programs and malware have significant requests are denied.

I don't share your optimism on this point. Just think of the copy protection of some games. It's deeply rooted in the system, to make it harder to circumvent it. Such a game will probably never run sandboxed.
But also more and more other application use such techniques. I'm using Defense+ in paranoid mode and am sometimes really astonished of the alerts some applications provoke...

[mouse1]

But I can't test the application if I get no feedback from CIS. So thanks for the hint with the logs. But I think I'll have to wait for the final to see if I'm happy with this method.

You could try running it in the sandbox unrestricted. Not sure what D+ restrictions are applied, and which are logged. Certainly some things are logged for progs in the sandbox,  memory accesses for example.]

I don't share your optimism on this point. Just think of the copy protection of some games. It's deeply rooted in the system, to make it harder to circumvent it. Such a game will probably never run sandboxed.
But also more and more other application use such techniques. I'm using Defense+ in paranoid mode and am sometimes really astonished of the alerts some applications provoke...

I agree, this may be difficult. Lets see how they manage. I was thinking of business applications.

[Chiron]

There should still be an option to view in real time all files that have been automatically sandboxed. That way if a program isn't working correctly that file can be specifically added to the safelist and the problem can be solved.

Right now I have it disabled. I realized that if I have it enabled some of my games don't work.

[Tags:]