Malware Forensics
Introduction to Malware
Malware is harmful software that corrupts or disables computer systems and gives limited or complete access of the systems to the malware creator for malicious activities such as fraud. When an unauthorized individual wishes to get unauthorized access to a secured or locked device, malware can be helpful. Malware includes viruses, worms, Trojans, rootkits, backdoors, botnets, ransomware, spyware, adware, scareware, crapware, rough ware, crypters, keyloggers, etc. These may delete files, slow down computers, steal personal information, send spam, or commit fraud. Malware can perform various malicious activities ranging from simple email advertising to complex identity theft and password stealing.

Different Ways for Malware to Enter a System
The most common ways an attacker can send malware into a system are as follows:
Instant Messenger Applications
Instant messaging apps like Facebook, WhatsApp, LinkedIn, or Google can spread infection. Using instant messengers to receive files puts users at serious risk. There is always a chance of getting infected by a Trojan pr viruses, regardless of who sends the file or where it comes from. It is impossible for the user to know exactly who is on the other end of the connection at any given time. For example, if you receive a file through an instant messenger application from a known person such as Ram, you will try to open and view the file. This could be a trick whereby an attacker who hacked Ram’s messenger ID and password wants to spread Trojans across Ram’s contacts list to trap more victims.
Portable Hardware Media
Portable hardware media such as pen drives, and external hard drives can also inject malware into a system. A simple way of injecting malware into the target system is through physical access. For example, if Ram can access Varun’s system in her absence, then he can install a Trojan or create a backdoor by copying the Trojan software from his flash drive onto her hard drive.
Another means of portable media malware infection is through the Autorun function. Autorun, also referred to as AutoStart, is a Windows feature that, if enabled, runs an executable program when a user connects a USB device. Attackers can exploit this feature to run malware along with genuine programs. They place an Autorun.inf file with the malware in a USB device and trick people into plugging it into their systems. Because many people are not aware of the risks involved, their machines are vulnerable to Autorun malware.
Browser and Email Software Bugs
Vulnerabilities in outdated web browsers can seriously risk the user’s machine. Without installing or running any software, a visit to a malicious website from one of these browsers can instantly infect the computer. A similar situation arises while using Outlook or any program with known issues to check email. Once more, the user’s system could become infected without ever downloading an attachment. Always utilize the most recent versions of your email program and browser to lower these risks.
Insecure Patch Management
Unpatched software is quite dangerous. Users and IT administrators do not update their application software as often as they should, and many attackers take advantage of this well-known fact. Attackers can exploit insecure patch management by injecting malware into the software, which can damage the data stored on the company’s systems. This process can lead to extensive security breaches, such as the theft of confidential files and company credentials. Applying patches and updating software packages on a regular basis are essential, and patch management must be successful in reducing threats.
Untrusted Websites and Web Applications
A website could be suspicious if it is hosted on a free website provider or offers programs for illegal activities.
- It is highly risky to download programs or applications located on “underground” sites, because they can serve as a conduit for a Trojan attack on target computers. Users must assess the high risk of visiting such sites before browsing them.
- Many malicious websites have a professional look, massive archives, feedback forums, and links to other popular sites. Users should scan the files using antivirus software before downloading them. Just because a website looks professional does not mean that it is safe.
- Always download popular software from its official site, and not from third-party sites with links to the same software.
Downloading Files from the Internet
Trojans or malware enter a system where users download Internet-driven applications such as music, free resources, movies and games from malicious websites, thinking that they are legitimate. Microsoft Word and Excel macros are also used effectively to transfer malware, and downloaded malicious MS Word or Excel files can infect systems. Malware can also be embedded in images, audio and video files as well as in video subtitle files.
Email Attachments
The most popular way to spread malware is through an attachment in an email. The attacker employs creative ideas to fool the victim into clicking and downloading the attachment, which can take any shape. Documents, audio files, video files, brochures, invoices, lottery offer letters, job offer letters, loan approval letters, admittance forms, contract approvals, and more might all be included in the attachment.
The following are the typical methods used by attackers to spread malware via the Internet:
Black hat Search Engine Optimization (SEO): Also known as unethical SEO, black hat SEO employs aggressive SEO strategies like page swapping, gateway page insertion, keywords stuffing, and adding irrelevant keywords to malware pages to raise their search engine ranks.
Social Engineered Clickjacking: To fool people into clicking on websites that look authentic, attackers insert malware inside them. Without the user’s Knowledge or agreement, the virus that is included in the link starts to run when it is clicked.
Spear-phishing Sites: This technique is used to mimic legitimate organizations, such as banks, to steal passwords, credit cards and bank account data, and other sensitive information.
Malvertising: This technique involves embedding malware-laden advertisements in legitimate online advertising channels to spread malware on systems of unsuspecting users.
Compromised Legitimate Websites: Attackers frequently utilize compromised websites to get malware into systems. The malware executes destructive actions after an unwary person visits the hacked website and unintentionally installs it on their computer.
Components of Malware
Attackers and malware developers use components that can assist them in achieving their objectives when creating malware. Malware can be used to change system settings, grant access, erase data, steal information, or just grow and take up space. Malware can spread and operate secretly.
Some essential components of most malware programs are as follows:
Crypter: IA program that can hide the presence of malware is called a crypter. This malware is used by attackers to avoid detection by antivirus software. It shields malware from analysis or reverse engineering, making it more challenging for security systems to identify.
Downloader: A downloader is a kind of Trojan that infects a computer or other device by downloading harmful files and other software from the Internet. When an attacker initially gains access to a system, they typically install a downloader.
Dropper: It is a covert carrier of malware. Attackers can carry out the installation operation secretly by embedding well-known malware files inside droppers. To use the dropper, attackers must first install malicious software or code on the machine. Without being picked up by antivirus software, the dropper can carry malware code and run malware on a target system.
Exploit: It is part of malware that contains code, or a series of instructions designed to exploit a flaw or weakness in a digital system or gadget. Attackers use such code to breach the system’s security through software vulnerabilities to spy on information or to install malware. Based on the type of vulnerabilities abused, exploits are categorized into local exploits and remote exploits.
Injector: This program injects exploits or malicious code available in the malware into other vulnerable running processes and changes the method of execution to hide or prevent its removal.
Obfuscator: It is a program that hides the malicious code of malware via various techniques, thus making it difficult for security measures to detect or remove it.
Packer: This software compresses the malware file to convert the code and data of the malware into an unreadable format. It packs the malware using compression techniques.
Payload: It is the part of the malware that performs the desired activity when activated. It may be used for deleting or modifying files, degrading the system performance, opening ports, changing settings, etc. to compromise system security.
Malicious code: This is a piece of code that defines the basic functionality of malware such as stealing data and creating backdoors.
Understand Malware Forensics
Digital forensic investigators are usually called upon to confirm and investigate a computer with a suspected malware infection. These investigators are tasked with identifying the malware, understanding its behavior, tracing its source, and assessing its potential effects on both the individual system and the broader network. To safely analyze the malware, they carry out their investigations within a secure environment, employing various forensic tools and techniques to thoroughly examine the malicious software.

Introduction to Malware Forensics
The process of identifying, analyzing, and investigating various malware properties to identify the culprits and reason behind the attack is known as malware forensics. The process also includes tasks such as finding the malicious code and determining its entry, method of propagation, impact on the system, ports it tries to use, etc. To conduct forensic analysis in a malware sample, forensic investigators employ various tools and techniques.
Why Analyze Malware?
The process of reverse engineering a specific piece of malware to determine its origin, functionality, and potential impact is known as malware analysis. Performing malware analysis allows for the extraction of detailed information about the malware, making it an integral part of any penetration testing process.
The primary objectives of analyzing a malicious program are as follows:
- Determine what exactly happened and the malicious intent of the malware.
- Identify indicators of compromise and exploit vulnerability.
- Identify the extent of damage caused by the attackers.
- Identify the offender responsible for installing the malware.
- Find signatures for host and network-based intrusion detection systems.
- Find the system vulnerability that the malware has exploited
- Determine whether an insider or the gate crasher is responsible for the malware entry.
Malware Analysis Challenges
The following are the various challenges that forensic investigators might face while performing malware analysis:
- Accuracy of the analysis process.
- Amount of data to be analyzed
- Detection of malware pieces and traits.
- Changing technologies and dynamics of malware creation and propagation.
- Anti-analysis procedures such as encryption, code obfuscation, and deletion of records
- Ability of malware to detect instances of being analyzed in virtual environments and change behavior accordingly.
- Rapid evolution of many malware variants that demand diverse analysis approaches.
- Highly obfuscated or complex malware code requires detailed analysis, which may be time-consuming.
- Investigators might lack high-level expertise and experience in discovering and analyzing advanced malware strategies.
- Require advanced reverse engineering skills for analyzing the functionality of complex malware samples.
- Legal compliance issues regarding the handling and storage of malware.
Malware Forensic Artifacts
Finding and examining malware artifacts is essential to malware forensic investigations to understand the scope, impact and nature of a malware attack. These artifacts are pieces of data that suggest or demonstrate malicious activity.
The following table shows the common malware forensic artifacts in the system:
- File system artifacts: It includes executables stored in unknown locations, temporary directories or unallocated disk space, and files with double extensions or unexpected changes in file sizes.
- Registry artifacts: It includes AutoStart registry keys having unauthorized changes and newly added or modified registry keys with unusual characteristics.
- Network traffic indicators: It includes massive network traffic to known malicious IP addresses, traffic patterns indicating command and control communications, and large or irregular DNS requests to non-standard domains.
- Log file anomalies: It includes unsuccessful login attempts, modifications to security policy, alerts from intrusion detection systems, unusual application errors, and unexpected starting or stopping of system services.
- Process and memory artifacts: It includes processes that do not usually run, familiar processes running from unusual locations, signs of code injection in the legitimate processes’ memory, and unusual CPU and memory usage.
- Browser artifacts: It includes unauthorized plugins or extensions, and unidentified modifications made to the browser settings.
- Email artifacts: It includes malicious email attachments and large volumes of outgoing spam emails.
- Cryptographic artifacts: It includes encrypted files accompanied by ransom notes and encrypted communications that use self-signed or untrusted SSL/TLS certificates.
- Authentication records: It includes multiple account lockouts and logins from unexpected locations or at odd times.
- System and software vulnerabilities: It contains proof that known vulnerabilities have been exploited for malware injections.
Indicators of Malware
Malware attacks create an immediate and significant impact on business assets as they can spread rapidly throughout the organization’s network within a short span. To detect a malware incident, investigators should keep an eye for the following indicators of malware attacks:
- Unusual network traffic flows.
- Unexplained bounced emails.
- Irrelevant alerts, ads, and popups.
- Logs demonstrating malicious attempts at port scanning, unauthorized access, etc.
- File modification, deletion, or relocation.
- Blue screen of death (BSOD).
- Unexpected system freezes, shutdown, and crash.
- Inability to install updates.
- Automatically disabled security programs.
- Changes to web browser settings.
- Suspicious processes running at system startup.
- Unauthorized program installation, launch, and closure.
- Alerts of spam messages from the system or email
- Consumption of storage space
- Numerous unwanted emails and messages on social posts.
- Unknown running processes and registry entries.
- Strange dialog boxes appear and request permission to run any program.
- Host and unknown remote systems network connections.
- Unusually open ports and off-hour usages.
- Increased outgoing web traffic even though users are not working.
- Unknown file share access
- Presence of unauthorized sessions and unknown malicious software.
- Detection of multiple failed logins.
- Download of any unfamiliar browser extensions or plugins.
- Websites are getting redirected to another web server.
- Presence of unusual executables in active processes
- Disabled control panel
- Programs are activated or deactivated without the user’s consent
- Suspicious activities in administrator or privileged accounts
Setting Up a Controlled Malware Analysis Lab

Malware Analysis Lab
A laboratory system is necessary for this to ensure the safety of the production environment. The most effective way to set up such a lab involves the use of virtualization software, which enables investigators to host multiple virtual systems running different operating systems on a single computer.
Some commonly used software to simulate real-time systems in a virtual environment include:
- VMware vSphere Hypervisor
- Microsoft Hyper-V Server
- VirtualBox
Malware establishes connections with other systems and networks to replicate itself, obtain orders from the attacker, or steal data. Researchers can use multiple interconnected virtual machines on a single physical computer to analyze malware behavior on connected systems and learn about its propagation techniques and other characteristics.
Investigators must take precautions, such as isolating the malware analysis lab from the production network using a firewall to prevent malware from spreading. One can use removable media to install tools and malware. Investigators can also use a write-protected USB key. Using a malware analysis lab also allows investigators to capture screenshots during analysis. Additionally, it allows them to take snapshots of the laboratory system, which can be used to easily restore it to a previous system state.
Preparing Testbed for Malware Analysis
To create a separate network for malware analysis, the testbed or malware analysis lab must have sandbox environments. Verify that the sandboxes have sufficient storage and processing power to set up and run different virtual computers for malware testing. The lab should be equipped with the tools needed to accurately model databases and networks. To prevent interfering with business operations while malware investigation is underway, the sandbox should not be linked to the operational network.
Requirements for building a tested:
- An isolated test network to host the test bed and isolated network services such as DNS.
- Victim machines with different operating systems configuration states installed.
- Virtualization snapshots and reimaging tools to quickly wipe and rebuild the victim’s machine.
- Certain tools are necessary for testing; the important ones are listed below:
- Imaging tool: To obtain a clean image for forensic and prosecution purposes.
- Data analysis: To perform static analysis of potential malware files.
- Registry tools: Malware infects the Windows registry and other configuration variables. These tools help identify the latest saved settings.
- Sandbox: To manually perform dynamic analysis.
- Log analyzers: The devices under attack record malware activities and generate log files. These tools are used to extract log files.
- Network capture: To understand how the malware leverages the network.
Malware Analysis Tools
The investigation team requires sophisticated tools and environments for analyzing and forensically investigating malware attacks because the malware can impact other systems in the organization if not handled carefully. Therefore, organizations must provide the investigation team with the following tools and malware toolkits.
A laptop with software tools, devices for storing data backups, hardware needed to connect to the affected machines, and standard networking supplies and connections must all be part of the malware toolkit. To gather and move questionable malware files from infiltrated systems to the system being analyzed, investigators should always use detachable devices like USBs.
Hardware Tools
The hardware tools required to extract, store, and move malware must be included in the malware toolkit. As soon as malware is detected, it should be combined with specialist technologies to lock or encrypt it.
The following hardware tools must be part of the malware toolset that the investigator team builds:
- A system with a virtual client installed to run the sandbox.
- A jump kit that is ready to use and includes various connector types for acquiring and backing up data from the compromised system.
- Storage media for storing the acquired and backed-up data.
- A write-protected device to prevent data modification during acquisition and backup.
Software Tools
Some software tools required for malware forensic analysis are listed below:
- Virtualization software such as VirtualBox, VMware vSphere Hypervisor, VMware Horizon, and Microsoft Hyper-V Server.
- Malware sandboxes for automated analysis include Joe Sandbox, CrowdStrike Falcon Sandbox, and ANY.RUN.
- Extraction from forensic images data collection tools include Forensic Toolkit, X-Ways Forensics, Autopsy, and The Sleuth Kit (TSK).
- PE analysis tools such as PEView, PeStudio, and FileAlyzer.
- Tools for taking snapshots of hosts such as Regshot.
- Memory dumping tools such as Belkasoft Live RAM Capturer and FTK Imager.
- Network sniffing tools such as Wireshark and tcpdump.
- Network simulation software such as iNetSim.
- Process exploring and monitoring tools such as Process Monitor and Process Explorer.
- Hex viewing tools such as Hex Editor Neo and Hexinator.
- Debugging tools such as OllyDbg, Ghidra, and IDA Pro.
- Tools for searching malicious strings including BinText, FLOSS, and Hex Workshop.
Other Supporting Tools for Malware Analysis
The following are some supporting tools required to perform malware analysis:
Screen-Capture and Recording Tools
- Snagit (https://www.techsmith.com)\
- TechSmith Capture (https://www.techsmith.com)
- Camtasia (https://www.techsmith.com)
- Ezvid (https://www.ezvid.com)
Network and Internet Simulation Tools
- NetSim Professional (https://tetcos.com)
- ns-3 (https://www.nsnam.org)
- Riverbed Modeler (https://www.riverbed.com)
- GNS3 (https://www.gns3.com)
OS Backup and Imaging Tools
- FTK Imager
- R-Drive Image (https://www.drive-image.com)
- O&O DiskImage 18 Server (https://www.oo-software.com)
Documentation Before Analysis
Documentation involves the process of recording detailed information on malware analysis. Investigators should be quick in taking note of the steps they follow, properties of the executable file they are analyzing, study results, and supporting material such as screenshots, etc. Investigators can also take note of the system status, platform, operating system, and tools used for the process.
The following are some of the documentations that an investigator should prepare before analyzing any executable file:
- Full path and location of the file
- MAC-timestamp
- The system information where file was stored, including the OS version, file system, user accounts, and IP address
- References to the file within the file system or registry
- Who found the file and when
- Details of forensic investigation tools
Types of Malware Analysis
Malware analysis can be categorized into two types: Static analysis or Dynamic analysis. Both methods show how the suspicious malware under investigation works, but they differ in the equipment, time, and expertise needed to complete the analysis.

Static Malware Analysis
The practice of examining an executable file without running or installing it is known as static analysis. Because the investigator doesn’t install or run the questionable file, static analysis is safe. Some malwares, on the other hand, can carry out harmful tasks without installation. Therefore, investigators should perform static analysis in a controlled environment.
Static analysis involves accessing the source code or binary code to find data structures, function calls, call graphs, etc. that can represent malicious behavior. Investigators can use various tools to analyze binary code to understand the file architecture and impact on the system. Data is lost when a system’s source code is compiled into a binary executable, which complicates code analysis. Information regarding the malware’s functioning, network signatures, exploit packaging method, dependencies, etc., can be found by analyzing the binary code.
The process of looking at a binary without running it is primarily done manually. It requires the extraction of vital data, such as data structures, utilized functions, and call graphs, from the malicious file. This data cannot be examined by the investigator after program compilation.
Some static malware analysis techniques are listed below:
- Identifying file dependencies
- Analyzing ELF executable Files
- Analyzing Mach-O executable files
- Analyzing Malicious MS Office Documents
- File fingerprinting
- Local and online malware scanning
- Performing strings search
- Identifying packing/obfuscation methods
- Finding the portable executables (PE) information
- Analyzing suspicious PDF Documents
Dynamic Malware Analysis
The method of examining malware behavior by executing it in a monitored environment is known as dynamic malware analysis. This type of analysis requires a safe environment, such as virtual machines and sandboxes, to prevent the malware from spreading. Environmental design should include tools that can capture every movement of the malware in detail and provide relevant feedback. Usually, virtual systems serve as a foundation for these kinds of experiments.
Dynamic analysis is performed to gather valuable information about malware activity, including files and folders created, ports and URLs accessed, functions and libraries called, applications and tools accessed, information transferred, settings modified, processes and services started by the malware, etc.
It is your responsibility as an investigator to create and configure the dynamic analysis environment such that the malware cannot spread to the production network. If something goes wrong during the test, the testing system should also be able to recover from a predetermined timeframe. The investigator must do the following actions to accomplish this:
System Baselining
Baselining is the process of capturing a snapshot of the system at the start of the malware analysis process. This snapshot can be compared with the system’s state after executing the malware file to understand the modifications done by the malware. The system baseline is a part of recording details of the file system, registry, open ports, network activity, etc.
Host Integrity Monitoring
Examining the alterations that have taken place within a system or computer following a sequence of events or incidents is known as Host integrity monitoring. This process helps analyze the impact on the system and its properties by comparing snapshots of the system before and after the incident or action.
In malware analysis, host integrity monitoring helps understand the runtime behavior of a malware file as well as its activities, propagation techniques, URLs accessed, downloads initiated, etc.
System Baselining
Investigators must acquire a picture of the forensic workstation’s baseline state before malware is executed to perform system baseline.
Following the creation of the baseline, which was previously created for the Windows workstation as part of pre-execution planning, investigators need to take the following actions:
- Run the malware on the Windows workstation for a certain period and take a second snapshot of the workstation.
- Compare the second snapshot with the baseline to detect changes made by the malware to system properties such as file systems and registry keys
Investigators can record and compare system conditions prior to and during malware execution using tools like WhatChanged Portable. This tool scans modified files and registry entries and lists them in a text file format. It should run in the background while the malware is running on the workstation to record changes in the file system and registry.
WhatChanged Portable
WhatChanged is a system utility that scans modified files and registry entries. It can be used to check program installations. WhatChanged Portable can run from a cloud folder, external drive, or local folder without installing it into Windows. It uses the “brute force method” to check files and the registry.
To use WhatChanged Portable, follow these two steps:
- Take a picture to see the computer’s status.
- Run it again to check the changes since the last snapshot.
Host Integrity Monitoring
Hots integrity monitoring of a malware sample involves running the virus on a forensic workstation and tracking its actions in real time to understand its function and goal.
Investigators can watch in real time as the malware unpacks itself, what malicious actions it takes on the registry, system files, and kernel resources, and whether it attempts to communicate with the outside world, like the network, by running the malware on the forensic workstation.
This makes it possible for investigators to identify and comprehend the behavioral traits of the malware that is being studied. Enforcing preventive measures against dangerous threats can be greatly aided by their ability to record and collect real-time information on the dynamic behavior of various malware sample kinds.
Investigators can monitor the runtime behavior of the malware in two ways:

System Behavior Analysis
It involves monitoring the changes in operating system resources when malware is executed. System behavior analysis involves monitoring the changes in the following system components after the execution of the malware: Registry artifact, Processes, Services and Startup programs, Event logs, API calls, Device drivers, Installation, System calls, Scheduled tasks, Files and folders.
Network Behavior Analysis
It involves tracking the malware’s network-level activities. Network behavior analysis includes monitoring the following network properties: Monitoring IP addresses, looking for connected ports, Examining DNS entries, Monitoring browser activity.
Perform Static Malware
The practice of examining an executable file without running or installing it is known as static analysis. It is safe to conduct static analysis because the investigator does not install or execute the suspect file. However, some malware does not need installation to perform malicious activities; therefore, it is better for investigators to perform static analysis in a controlled environment.

Static Malware Analysis: File Fingerprinting
Finding the hash value for a certain binary code to identify and monitor data over a network is known as File Finderprinting. This process includes the calculation of cryptographic hashes of the binary code to identify its function and compare it with other binary codes and programs from earlier scenarios. Unique hash values or fingerprints are developed using various cryptographic algorithms that utilize data such as strings, metadata, size, and other information. The computed hash value can be used to uniquely identify the malware or periodically verify if any changes are made to the binary code during analysis. The calculated hash value can also be compared to the hash value of malware that has been recognized and saved in databases, such as the online database VirusTotal.
Similar programs can be tracked and identified from a database using these fingerprints. Certain record kinds, such as encrypted or password-secured files, pictures, audio, and video, are incompatible with fingerprinting because their contents change from the predefined fingerprint.
Message-Digest 5 (MD5) algorithm and Secure Hash Algorithm (SHA) are the most used hash functions for malware analysis. Various tools such as HashMyFiles and HashCalc can be used to create a fingerprint of the suspicious file.
Static Malware Analysis: Local and Online Malware Scanning
You can use reputable and current antivirus software to locally scan the binary code. Numerous antivirus software providers may have already found and documented the code in question if it is a part of well-known malware. To have the code examined by a few different search engines, you can also upload it to services like VirusTotal.
VirusTotal calculates the hash values of a suspicious file and compares them with online and offline malware databases to determine the existence of the recognized malicious code. By providing more in depth understanding of the code, its operation, and other crucial elements, this procedure streamlines additional research.
Static Malware Analysis: Performing Strings Search
Software programs include some strings that are commands for performing specific functions such as printing output. Strings communicate information from the program to its user. A program’s malicious purpose can be represented by a variety of existing strings, such as reading internal memory or cookie data encoded in the binary code that has been created.
Searching through the strings can provide information about the basic functionality of any program. Look for the malicious string during analysis to find out what bad things a software can do. For example, that URL string will be kept in the application whenever it accesses a URL. To identify suspicious files, it is best to be vigilant when searching for strings and for encrypted and embedded strings.
To extract embedded strings from executable files, use programs like BinText. Make that the tool is capable of scanning and displaying Unicode and ASCII strings. All the strings can be extracted and copied into a text or document file using certain programs. To make the process of looking for dangerous strings easier, use such tools to transfer the strings to a text file.
Static Malware Analysis: Identifying Packing/Obfuscation Methods
To evade detection, attackers compress, encrypt, or alter a malware executable file using packing and obfuscation. Obfuscation also hides the execution of the programs. When the user executes a packed program, it also runs a small wrapper program to decompress the packed file and then run the unpacked file. This makes it more difficult for reverse engineers to use static analysis to determine the true program logic and other metadata.
As an investigator, you should determine if the file includes packed elements and identify the tool or techniques used for packing it. Use tools such as PEiD, which can detect the most used packers, cryptors, and compilers for PE executable files. The process of choosing a tool to unpack the code will be made easier by locating the packer. Additionally, by comparing the memory artifacts caused by malware with those of on-disk apps, you can try to reconstruct malicious-code executable content from memory dumps.
Static Malware Analysis: Finding Portable Executables (PE) Information
Windows OS uses the PE format, which is an executable file format that holds the data needed by the windows system to control the executable code. It stores metadata about the program, which helps in finding additional details of the file. For instance, the Windows binary is in PE format and consists of information such as time of creation and modification, import and export functions, compilation time, DLLs, linked files, strings, menus, and symbols. An OS’s code mapping and file information are stored in the PE format’s header and other parts.
The PE of a file contains the following sections:
- Program codes and instructions that the CPU can execute are contained in the .text file
- Import and export information and other read-only data used by the program are contained in the .rdata file
- The program’s global data is contained in the .data file, which the system can access from any location
- The executable’s resources, including menus, icons, images, and strings, are contained in the .rsrc file, which provides multilingual support.
The header data can be used to learn more about a program or file, including its features. The above-mentioned data can be extracted using programs like PE Explorer and PEView.
Static Malware Analysis: Identifying File Dependencies
Any software application depends on a variety of operating system libraries that are built in to help carry out specific tasks within a system. Programs need to work with internal system files to function correctly. They store the import and export functions in kernel32.dll file. File dependencies include details on the program’s location on the computer, the registration procedure, and the internal system files it requires for correct operation.
Investigators need to locate the libraries and file dependencies, as they contain information about the runtime requirements of an application. Investigators must then confirm that they are able to locate and examine these files, since they may include information regarding malware. File dependencies include linked libraries, functions, and function calls. Examine the dynamically linked list in the malware executable file. Investigators may be able to deduce the capabilities of the malware software by learning about all the library functions. Investigators need to understand the different DLLs that are utilized to load and execute programs.
Some standard DLLs are listed on the table below:
- dll – Offers access to sophisticated core windows components including Service Manager and Registry.
- dll — Core functionality, including access and manipulation of memory, files, and hardware.
- dll – Components for the user interface, including buttons, scrollbars, and elements for managing and reacting to user inputs and actions.
- dll — Functions for displaying and manipulating graphics.
- dll — Interface to the Windows kernel.
- dll and Ws2_32.dll — Networking DLLs that aid in network connection or network-related tasks.
- dll — Supports higher-level networking functions.
Static Malware Analysis: Analyzing Mach-O Executable Files
Mach object (Mach-O) is an executable file format, like Linux’s ELF and Windows’ PE formats. It is linked to binaries found in iOS and macOS. The process by which memory reads the data and code contained in a binary file is established by this file format, which is also used to distribute code. Because the sequence of code in a binary file affects memory utilization and paging activities, Mach-O malware directly affects a program’s performance. With the use of this malware, attackers can create two arrays that overlap in memory and specify where in memory to run a Mach-O executable. This feature can be used by attackers to escalate privileges and take advantage of next-stage vulnerabilities that need root access.
Analyze Suspicious Documents
Attackers commonly exploit PDFs and Microsoft Office documents, including Word, PowerPoint, Excel, and others, to disseminate malware. Investigators must first comprehend the document’s structure to assess whether a PDF or MS Word document is harmful. They must be aware of all common packing and obfuscation techniques and comprehend how attackers can insert malicious code or programs into the document. By applying forensic techniques and comparing the document structure with a standard structure, investigators should also be able to identify malicious papers.
Perform System Behavior Analysis
Malware may interact with the registry, files, folders, Windows processes and services, and device drivers, among other system components, while it is running. To accomplish its goals, it might establish malicious Windows services or processes or alter or remove registry keys.
Using forensic tools and comparing the results with a baseline image, investigators can examine changes in the registry, processes, or services after the malware has been executed. They can also look at the API calls the malware makes and keep an eye on event logs to see how the infection alters system properties.
System Behavior Analysis: Monitoring Registry Artifacts
The Windows registry stores OS and program configuration details, such as settings and options. For programming malware, the registry stores its functionality. Malware manipulates the registry to ensure automatic execution upon computer or device bootup or user login.
Forensic investigators can execute the malware on a Windows forensic workstation and observe its interactions with the system registry files, particularly the registry keys and values that it creates, modifies, or deletes.
Investigators can investigate specific registry locations while performing a runtime analysis of the malware to learn more about its functionality. Monitoring AutoStart registry keys can be quite useful, as those are the most common locations targeted by malware.
Windows AutoStart Registry Keys
The AutoStart keys within the Windows registry, which allow programs to be executed automatically upon system reboot or user login, are the most common locations targeted by malware to achieve persistence on any compromised machine.
Some of the Windows AutoStart registry keys targeted by malicious programs are discussed below:
Run/RunOnce Keys
Malware often modifies the below-mentioned registry keys to continue running on the system whenever the user logs in:
- HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Run
- HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\RunOnce
- HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Policies\E xplorer\Run
- HKEY_CURRENT_USER\SOFTWARE \Microsoft\Windows\CurrentVersion\Run
- HKEY_CURRENT_USER\SOFTWARE \Microsoft\Windows\CurrentVersion\RunOnce A malicious program can also modify the following system-related keys:
Startup Keys
Malware authors also try to place their malicious executable file within the startup directory of the compromised system and create a shortcut entry on the location indicated by the Startup subkey, which is configured to execute the service automatically on each login and reboot. These startup locations are found both at the user level and system level:
- HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Explorer\S hell Folders, Common Startup
- HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Explorer\ User Shell Folders, Common Startup
- HKEY_CURRENT_USER\SOFTWARE\Microsoft\Windows\CurrentVersion\Explorer\Sh ell Folders, Startup
- HKEY_CURRENT_USER\SOFTWARE\Microsoft\Windows\CurrentVersion\Explorer\Us er Shell Folders, Startup
System Behavior Analysis: Monitoring Processes
Malware enters the system through images, music files, videos, etc., downloaded from the Internet, camouflaging themselves as genuine Windows services and hiding their processes to avoid detection. Some malware uses PEs to inject themselves into various processes, such as explorer.exe or web browsers. Malicious processes are visible but appear legitimate; hence, they can bypass desktop firewalls. Attackers use specific rootkit methods to hide malware in the system, making it difficult for antivirus software to detect.

Understanding the processes that malware starts and then takes control of after execution is made easier with the use of process monitoring. To determine the complete nature of a file or program, collect data about the processes that were operating prior to the malware’s execution, and contrast them with the processes that were operating following it, it is also essential to observe the child processes, associated handles, loaded libraries, functions, and execution flow of boot time processes. This technique helps quickly identify every process that the virus launches and cuts down on the amount of time needed to evaluate the processes. To find suspicious processes, malicious parent/child processes, malicious DLLs, and connections, use process monitoring tools like Process Monitor.
- Process Monitor
- Process Explorer
System Behavior Analysis: Monitoring Windows Services
Malware and other malicious programming are created by attackers to install and operate on a computer as services. Since most services serve processes and apps by operating in the background, malicious services can operate without user input or intervention and remain undetectable even when they are causing harm to the system. Windows services are spawned by malware, giving attackers remote access over the victim’s computer and the ability to send destructive commands. To conceal its services and processes, it might also use rootkit techniques to change the following registry keys.
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services
These malicious services are more harmful than executable code and typical malware because they operate as SYSTEM accounts or other privileged accounts, which grant more access than user accounts. To evade detection, attackers also attempt to hide their activities by renaming malicious services to seem like legitimate Windows services.
Using Windows service monitoring tools like Windows Service Manager (SrvMan), which can identify service modifications and search for questionable Windows services, investigators can track malicious services started by suspicious files during dynamic analysis.
- Windows Service Manager (SrvMan)
- Advanced Windows Service Manager
System Behavior Analysis: Monitoring Startup Programs
Malware can run harmful operations every time the machine boots up by changing system settings and inserting itself into the startup menu. To detect malware, it is necessary to manually scan for suspicious startup applications or use startup program monitoring solutions like Autoruns for Windows.
The steps to manually detect hidden malware are discussed below.
Step 1: Check Startup Program Entries in the Registry
When users log into a Windows operating system (like Windows 11), startup items including applications, shortcuts, folders, and drivers are configured to launch automatically at startup. The installed programs or drivers may add startup items, or the user may do it manually. Registry entries for Windows Explorer and Microsoft Edge startup settings may contain programs that launch at Windows 11 startup.
Windows Startup Settings
- HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Run
- HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Run
- HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\RunOnce
- HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\RunOnce
Explorer Startup Settings
- HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Explore r\Shell Folders, Common Startup
- HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Explore r\User Shell Folders, Common Startup
- HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Explorer\Shell Folders, Startup
- HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Explorer\User Shell Folders, Startup
Startup Program Monitoring Tool: Autoruns for Windows
Every startup monitor’s autostart locations are known by the Microsoft Autoruns application. It shows which programs are set up to execute when the system boots up, when you log in, and when you launch Windows built-in apps. Users can set Autoruns to display additional destinations, like toolbars, browser assistance objects, Winlogon notifications, explorer shell extensions, and autostart services, after this software is added to the Startup folder, Run, RunOnce, and other registry values.
The Hide Signed Microsoft Entries option in Autoruns allows the user to examine the autostart images set up for other accounts on the system and zoom in on third-party autostart images that are uploaded to the user’s system.
- HiBit Startup Manager (https://www.hibitsoft.ir)
- Autorun Organizer (https://www.chemtable.com)
- Quick Startup (https://www.glarysoft.com)
System Behavior Analysis: Monitoring Windows Event Logs
The process of log analysis yields information about events or actions that may reveal potential Trojan or worm attacks on the system. It assists in locating security flaws and acts as a main source of information. When logs are examined for various components, this procedure aids in the detection of zero-day backdoor Trojans and any potential attacks (failed authentication/login attempts). Web servers, authentication servers, firewall systems, and IDS/IPS are examples of components that can have their logs monitored. File types, ports, timestamps, and registry entries are also included in the logs. In Windows, the “Windows Logs” part of Event Viewer allows for the analysis of system, application, access, audit, and security logs.
The logs are located on the following paths:
- System logs: Start → Windows Administrative Tools → Event Viewer → Windows Logs → System
- System Security logs: Start → Windows Administrative Tools → Event Viewer → Windows Logs → Security
- Application Logs
- Start → Windows Administrative Tools → Event Viewer → Applications and Services Logs
Key Event IDs to Monitor
While performing runtime analysis, investigators need to look at specific event IDs and corelate the event descriptions to any anomalous activities to understand the behavior of the malware. The following are a few of the significant event IDs:
Event ID 4688
Malicious programs often include an .exe file into the filesystem to compromise a machine. Event ID 4688, which is generated whenever a new process is initiated, can help forensic investigators look for suspicious process names or process paths upon malware execution. Malicious process names are often misspelled, such as “scvhost.exe” instead of “svchost.exe,” or “iexplorer.exe” instead of “explorer.exe.” Any Windows process running from an unusual path should also be investigated, such as C:\Windows\svchost.exe instead of C:\Windows\System32\svchost.exe
Event ID 5156
When Windows Filtering Platform permits a program to connect to another process on the same computer or any other distant computer via UDP or TCP ports, this event is created. This event ID may be crucial for identifying the attack’s source during dynamic malware investigation. Investigators can use the event description to identify the following details:
- Application name: This is the complete path or name of the malicious software that connects to an internal or external IP address.
- Direction: Indicates if an outgoing or inward connection is permitted.
- Destination address: It is the IP address the connection was received from.
- Destination port: This is the port number that any distant computer can use to establish a connection.
Event ID 7045 and 4657
Any new installation is often a planned event. By developing a persistent malicious service that runs even after the system is rebooted, an attacker may attempt to keep control of the compromised system. By altering specific registry keys, the attacker can also produce a persistent malware mechanism that allows them to add the malicious payload to the list of programs that are run when the system reboots, including Windows starter programs. As a result, any unexpected service installation or unusual registry key changes found using event IDs 4697 and 4657 oughts to be taken seriously and investigated further.
Event ID 4660 and 4663
Malware may try to access, alter, or remove any files and folders from the compromised system after it has been executed. Thus, event ID 4660, which is produced upon the deletion of any object—whether it is a kernel, file system, or registry object—should be watched by investigators. Investigators must follow event 4663, which verifies whether access permissions were used along with the name and type of the object, account name, and process name that accessed the item, as this event ID lacks the name of the deleted object. Other access request data, such ReadAttributes, WriteAttributes, READ_CONTROL, etc., can also be tracked by tracking event IDs 4663.
Event IDs 7036 and 7040
To stay persistent on the target machine, any malicious program may also turn off essential Windows protection services like Windows Defender, Windows Firewall, or antivirus software. An investigator would be able to search for any such suspicious activity by keeping an eye on events 7036 and 7040.
System Behavior Analysis: Monitoring Scheduled Tasks
Using Windows Task Scheduler, attackers create malware that will remain dormant and activate on a specified date or occasion. To identify malware, such as logic bombs that can execute various triggers, investigators must look for scheduled tasks.
To view a list of all the system scheduled tasks, investigators can utilize command-line applications like Windows Task Scheduler and Schtasks.
System Behavior Analysis: Monitoring Files and Folders
Malware can alter system files and folders to store data there. You should be able to locate the files and folders that the virus generates as an investigator and examine them to gather any pertinent data that may be stored. These files and folders might also have dangerous strings or secret computer code that the infection would plan to run on a predetermined timetable.
Using the “openfiles” command, which shows the list of open files, investigators must also look for any files that were opened remotely by the malware or an attacker.
To find malware, investigators should also look at prefetch files, clipboard information, etc. Prefetch files are created by Windows when a program is first launched. This file assists in locating running programs, related files, DLLs, processes, and services. To find out whether the attacker used apps like CCleaner to remove activity history, investigators should examine prefetch files. Additionally, prefetch files offer a timeline that aids investigators in determining the precise moment the malicious programs were run. To view prefetch files, use programs like WinPrefetchView.
Use programs like PA File Sight, Tripwire File Integrity and Change Manager, and Netwrix Auditor to search for questionable files and directories to find any Trojan horses that may have been installed and system file changes.
Perform Network Behavior Analysis
Monitoring network communications is an important part of dynamic malware analysis as it can exhibit the malware’s operations over network properties. The secret to stopping malware’s propagation and reducing hazards may lie in understanding how it enters the network. Investigators should thus keep an eye on the network environment while the virus is operating to observe how it interacts with the network properties.
Network Behavior Analysis: Monitoring Network Activities
The technique of recording network data and closely examining it to find malware activities is known as network analysis or monitoring. It assists in identifying the kind of data or network packets or traffic that are sent over the network.
Malware uses the network for several purposes, including spreading, downloading harmful content, sending private files and data, giving attackers remote control, and more. In addition to manipulating the victim computer’s network configuration to call out a certain URL, IP address, or domain name, some malware groups—such as Trojan horses, worms, and bots—wait for additional instructions from the attacker. Therefore, methods that can identify malware artifacts across networks should be used by investigators.
Network traffic monitoring, statistical analysis, and manual traffic review help investigators in detecting any privilege malware escalation activities over the networked computers.
For network activity monitoring, investigators can execute the malware on the forensic workstation and monitor the following aspects:
- IP addresses going from and connecting to the workstation
- Ports being opened on the workstation
- List of DNS entries recorded on the workstation
Network monitoring tools such as Capsa Network Analyzer, and Wireshark, can be used to monitor and capture live network traffic to and from the victim’s system during the execution of the suspicious program. This helps in understanding the malware’s network artifacts, signatures, functions, and other elements.
Monitoring IP Addresses
Investigators must take the following actions to ascertain whether a file that is suspected of being malware is attempting to call out any remote or malicious IP addresses:
- On the Windows forensic workstation, they must launch the Wireshark application, which will show all network activity.
- They must run the file suspected of being malware on the workstation while Wireshark is running in the background.
- They should then keep an eye on the network’s real-time traffic to look for any unusual activity.
- It should be noted as unusual if they discover any distant or unknown IP addresses that the workstation is attempting to connect to.
- Lastly, they should check the IP address that was acquired using internet virus detection programs to see if it is malicious.
Network Behavior Analysis: Monitoring Port
Malware programs corrupt the system and open system input/output ports to establish connections with remote systems, networks, or servers to accomplish various malicious tasks. These open ports can also create backdoors for other types of harmful malware and programs. Open ports act as communication channels for malware. They open unused ports on the victim’s machine to connect back to the malware handlers. Finding such viruses will help in identifying the suspicious ports.
As an investigator, you can also determine whether malware is trying to access a particular port during dynamic analysis by installing port monitoring tools such as TCPView and Windows command-line utility tools such as netstat. These port monitoring tools provide details such as the protocol used, local address, remote address, and state of the connection. Additional features may include process name, process ID, remote connection protocol, etc.
Network Behavior Analysis: Monitoring DNS
DNS is used by malicious applications to connect to the C2 server, which is configured by the criminal. To evade detection by reverse engineers, malware employs Domain Generation Algorithms, or DGAs, and quickly makes several DNS queries to several domains to establish a connection with its C2.
By altering the DNS server settings on the system, malicious software like DNSChanger gives attackers command over the DNS server that is utilized on the victim’s computer. The attackers can then take control of the websites that the victim attempts to access via the Internet, force him or her to visit a phony website, or obstruct their online browsing.
To determine whether the malware is attempting to reach a particular domain name, investigators should examine the DNS entries stored on the workstation (sometimes referred to as the DNS cache) during runtime analysis. By entering ipconfig /flushdns at the command prompt and hitting Enter, they must remove DNS cache entries from the workstation. They can then detect the malicious domain name by running the virus.
Examining DNS Entries
Investigators must take the following actions to determine whether the virus is attempting to contact any domain name:
- They must first use the Windows forensic workstation
- To run the suspect file. They must then type the command ipconfig /displaydns into the workstation’s command prompt. All the entries kept in the DNS cache, including all current and attempted visits to websites, FTP servers, etc., will be seen by doing this.
- Next, they can identify any suspicious or unrecognized domain name from the DNS cache list.
- After execution, they can use hybrid analysis to further investigate any questionable or unknown domain names to determine whether they are malicious.
Network Behavior Analysis: Monitoring Browser Activity
Malware can download harmful files via connecting to C&C servers, rogue websites, and other DNS servers through browsers. To find malicious traffic and the location of the system, investigators must examine suspicious browsing activity.
Investigators can look for any surfing activity that may have taken place over ports 80, 443, or 8080, which are used by browsers to connect to the C&C servers. Additionally, investigators can monitor web traffic at firewalls, look at web caches, and filter web traffic by URL and harmful strings in web logs. To keep an eye on user browsing behavior, employ network monitoring tools like Colasoft Portable Network Analyzer and Wireshark.
Perform Ransomware Analysis
An essential component of forensic inquiry is ransomware analysis, which aims to comprehend the workings of a specific ransomware, including its encryption methods, infection pathways, and any potential flaws that could be used for decryption or mitigation. Both static and dynamic analytic techniques are frequently used in this kind of analysis. Use tools for ransomware analysis, such BlackCat (ALPHV).