Blog

AV-Comparatives Chose Best Antivirus to Defend Against APT Attacks

Now Reading

Austrian lab AV-Comparatives conducted extensive dynamic antivirus testing for Windows 10 in September-November 2020 and selected the best antivirus.

Comprehensive antiviruses Kaspersky, Bitdefender and ESET showed the maximum result of Advanced + in protection against APT attacks.

In the Enhanced Real-World Test, AV-Comparatives researchers use hacking tools and hacking techniques that allow attackers to gain access to internal computer systems.

General Information

The term Advanced Persistent Threat (APT) is commonly used to describe targeted cyberattacks aimed at breaking into an information system and employing sophisticated cyber attack techniques and methods.

The ultimate targets of such attacks can be to steal, distort or damage confidential information, or create opportunities for sabotage, which in turn can lead to financial and reputational losses to targeted organizations. These attacks are highly targeted and use specialized tools such as obfuscated malicious code, malicious use of legitimate system tools, or fileless malicious code.

AV-Comparatives’ benchmarks use a variety of techniques and resources to simulate the behaviour of real malware. Lab researchers use system applications to bypass signature detection. Various scripting languages are used for this (JavaScript, batch files, PowerShell, Visual Basic scripts, etc.). The tests use both single-stage and multi-stage malicious samples that perform obfuscation during deployment or encrypt malicious code before execution (using Base64, AES standards).

Fully patched 64-bit Windows 10 systems were used as target systems. Each system had its own antivirus installed.

Once the victim launches the payload, a Command and Control Channel (C2) is established. To do this, a listener must be running on the attacker’s machine. With the help of the C2 channel, the attacker gains unlimited access to the compromised system. The functionality and stability of the remote control was assessed individually in each test case.

The tests used 15 different attacks. In future tests, public reports are expected to provide more details on test difficulty and test coverage. This testing focused on security, not detection.

Vendors taking part in the AV-Comparatives main test series (AV Main-Test-Series) were given the opportunity to opt out of this test, before the launch of open testing, so not all development companies were taken into account in this test.

Test area

Advanced Dynamic Testing shows how well tested antivirus solutions cope with highly specialized targeted attack methods. The test does not take into account the overall level of security or the effectiveness of protection against malware downloaded from the Internet or distributed via portal.

This test shows whether the antivirus protects against specific attack and exploitation methods used for the ATP. The number of participants in the Enhanced Real-World Test is expected to increase next year.

Test procedure

Scripts containing malicious code such as VBS, JS, or MS Office macros could execute and install a fileless backdoor on the victim’s computer and establish a connection to the attacker’s command centre (C2 channel), which usually used a different physical location and could be located in a different country.

In addition to these common scenarios, attacker can deliver fileless malicious loads using exploits, remote calls (PSexec, wmic), task scheduler, registry keys, Arduino single board computer (USB RubberDucky), or WMI calls. For doing this, they can apply built-in Windows tools such as PowerShell.

These methods allow placing malicious programs from the Internet directly into the memory of the target system and develop an attack on the local network using the built-in tools of the operating system. In addition, an attacker can make threats resilient on the victim’s machine.

Attack methods

Extended dynamic testing used several command line stacks, CMD / PS commands that allow malware to be loaded from the network directly into RAM, and base64-encrypted calls.
These methods allow attackers completely avoid access to the file system, which is usually well protected by antivirus solutions.

Sometimes applied simple methods of hiding, or the method of calling the stager load is changed, i.e. the load is not broken part. When the malware has downloaded the second part, a connection is established to the attacker via HTTP or HTTPS. This internal mechanism allows establishing a C2 channel to the attacker, bypassing the protective measures of most firewalls and NAT.

After a successful tunnel deployment, an attacker can use all available control mechanisms of C2 products (Meterpreter, PowerShell Empire, etc.). It gets the ability to upload and download files, capture screenshots, register keystrokes, control the Windows shell, and capture images from a webcam. All used tools are available free of charge. Their source code is open source for research purposes. However, this toolkit is maliciously used by attackers for criminal purposes.

False positive test

An antivirus that blocks 100% of malicious attacks, but also blocks completely safe (non-malicious) actions, can be very destructive. AV-Comparatives is testing false positives as part of its Advanced Dynamic Testing to see if antivirus can tell the difference between malicious and safe behaviour. Otherwise, the antivirus can completely block 100% of malicious attacks that use email attachments, scripts or macros, simply by blocking these functions. In this case, most users will not be able to perform daily tasks.

Therefore, the results of the test for false positives were taken into account when calculating the final scores of antiviruses.

It is important to note that continually warning the user about opening harmless email attachments can lead to a “boy who cried wolf” scenario. Users who encounter a number of redundant warnings will sooner or later assume that all warnings are false and ignore the genuine warning when it appears.

Tested antiviruses

The following vendors participated in the Enhanced Real-World Test. These vendors’ antiviruses have proven themselves well in internal preliminary testing. The companies-developers were sufficiently confident in the capabilities of their antiviruses in resisting fileless attacks and decided to participate in open testing. All other antivirus developers participating in the main test series refused to test.

Avast Free Antivirus 20.8
AVG Free Antivirus 20.8
Bitdefender Internet Security 25.0
ESET NOD32 Internet Security 13.2 – 14.0
F-Secure SAFE 17.8
Kaspersky Internet Security 21.1
VIPRE Advanced Security 11.1

All consumer antiviruses were tested with standard settings (default settings).

Test Results

AV-Comparatives Chose Best Antivirus

Designations:

AV-Comparatives Chose Best Antivirus

Researchers at AV-Comparatives believe that the primary goal of a defence system (AV, EPP or EDR) is to detect and prevent the development of persistent threats and other types of malware as quickly as possible. In other words, if APT is recognized before or immediately after launch, therefore connectivity to the command centre is prevented, then there is no need to prevent post-exploitation activities. A good burglar alarm should start when criminals break into home, not when they start stealing owner’s belongings.

Antivirus that blocks certain features, such as email attachments or scripts, will be classified as “Tested”. However, none of the tested antivirus has demonstrated this behaviour in scenarios with false positives / blocking functionality.

If a user-specific warning were shown, the corresponding antivirus would receive half a point, but this test did not register such cases.