Automated State Machines Applied in Client Honeypots - Case Study Example

Automated Machines Applied In Client Honeypots Machines In simple terms a machine would imply a machine indicating different states of a task being performed. If we can plan out strategy in advance, in such a manner that each state gets defined in terms of time and resources, then the probability of completing the task in time increases i.e. the workflows become much smooth. Allen (2006) defined a workflow as “a series of steps, decisions, and rules needed to complete a specific task”. There are essentially two types of workflows: Sequential Workflow, and State Machine Workflow We will use the state machine having the ability to effectively monitor the attacks from malicious sources. The main advantage is that the state machine is not predictable while moving from one state to another, and it is driven by events coming from outside. A state machine is divided into a series of states and transfers from one to another through transitions, which are generated by events. An example of event is a user turning on the system: the system transfers from the “off” state to the “on” state. A state machine cannot be in two states at one time; this makes it easy to query the state of the workflow. The basic idea of this project is to generate the state machine, automatically according to the system state, to build up a map of the system states from the first interaction of the honeypot with the server until the end of the session. The benefits of using an automated state machine with a high-interaction client honeypot are: 1. The state machine is generated automatically according to the system state while it is working. 2. For each client honeypot scan, the state machine is generated afresh. 3. Each state machine contains all the system states while interacting, which helps to review the system states. 4. The state machine generated includes each action done by any malware and saves the action along with its parameters. 5. The map of the state machine for each interaction helps to analyze the malware and identify its process in order to develop a patch for it or learn its tricks. The state machine used here is a finite state machine where: The Σ alphabet input includes the finite number of events Σ = {e1, e2, e3 …, e18}. S is the none-empty finite states as S = {s0, s1, s2, s3, s4, s5}. S0 is the initial state. S1 is the File system state. S2 is the Processes state. S3 is the Registry state. S4 is the Ports state. S5 is the final state. δ is the transition function to states as δ: S × Σ S. This finite state machine can be also illustrated with the 5-tuple as A ={S, Σ, δ, S0, S5}. The state machine diagram shows in Fig. 2. Figure 2: The state machine diagram. (The boxes indicate states and the arrows represent transitions.) The state machine includes four main states that will be monitored with 18 possible transitions between the states. The finite state machine can be described as the state table which shows the possibility and direction of each transition to/ from state, the table is shown in Table 1. Table 1: The state transition table of the finite state machine on Fig 2 As mentioned earlier, there are several issues to discuss about the finite state machine as well as the client honeypot in order to get a better understanding. Some of the issues such as the system states to be monitored, the transitions from one state to the next, the information saved with each state, and the tools for monitoring changes in each system state are worth mentioning here. These issues are discussed in the next section. 2. A Client Honeypot with a State Machine The previous section described the state machine and its benefits for client honeypots. In order to incorporate an automated state machine into a client honeypot, the issues listed in section-1 are addressed below. The first question is “what are the monitored system states?” To answer this question we have to understand what are the system states during infection by a malware, and other benign states. The system state can be changed by adding a registry value, running a new process, or inserting a new cookie. On the other hand, we have to look at the tools that can be used to monitor each change. The system states that will be monitored while interacting with a target server are: 1. The File system will be monitored for any changes such as adding, deleting or modifying files or folders. 2. The Registry will be monitor for any added, deleted, or modified registry values. 3. Processes will be monitored for new processes run or processes started by the default process killed. 4. TCP and UDP ports will be monitored to detect if any ports are opened or closed; these ports could be used a backdoor by the attacker. The second question is “how does each state transfers to the next state?” Each of the four states listed above will have different transitions to another state. Each transition is responsible for transferring from the current to the next state or even returning to the same state. For example, a malware can add a new file, which will transfer the system state to the file system state. If the malware then adds another file, the state will transfer to the same state but with new information which adds the new file along with its action to build the state machine map. The transitions are described and mentioned above in Table 1. The third question is “what information will be saved for each state?” In each state there are several pieces of information that should be saved with the state to identify and show the malware activity. The information items saved with each state are: Type: This field saves the action which can be different for each state. The Table-2 shows each state with its types. Action: This field saves the action parameters such as the target file created or the registry value inserted. Finally, the last question is “what are the tools for monitoring system state changes?” the answer is fully described in the section-3 which shows the monitoring tools. Our main aim here is to build a framework that combines some free or open source tools with our tools that could help us to monitor as well as allow us to customize them to our needs. Table 2. The state types State Type File system Add, modify or delete Registry Add, modify or delete Processes New or kill ports Open or close 3. Automated State Machine Example The previous section shows how the drive-by download attack can be detected by the security tools available in the Windows environment. The result of this attack scenario can also be shown in our state machine diagram to get a clear example of how the automated state machine could be applied in the high-interaction client honeypot. To summarize the previous attack for understanding its steps, so that we can apply it to the state machine, the steps are shown below in time order: 1. The exploit transfer via the source page to the user’s vulnerable browser. 2. The attack exploit by starting a new process called rundll23.exe which was not run from the default process. 3. The TCP port 28876 is opened from the new process running. As shown above the steps are easy to understand and applied to the state machine. Fig. 3 shows the automated state machine of the CVE-2005-0553 exploit. Figure 3: The automated state machine of the CVE-2005-0553 exploit. The Fig 3 above shows our automated state machine applied on the CVE-2005-0553 exploit; there are four states which summarize the exploit and the machine states. The states’ steps are illustrated below: 1. S0: The first state is the initial state as shows in the figure 3 above, the initial state started with (e 1) transition which is responsible to transfer the machine state from the first state (default state) to new state by adding new process. Therefore, the state gets transferred to state S2. 2. S2: The processes state according to the automated state machine in figure 2, the state described the new state of the machine and shows the new action which runs the ‘rundll32.dll’ process. The state then used the transition (e 7) to transfer the machine state to the next state which is S4. 3. S4 is the Ports state as described in figure 2. This state is transferred by the transition (e 7) and indicates that there is a new port open i.e. port 28876 TCP. Then, the state used the transition (e 18) to transfer to S5 state. 4. S5 state is the final state which ends the exploit activity and effects to the system. As mentioned above, by using the automated state machine we can save the whole changes map of an exploit and malicious software within the client honeypot. The automated state machine saves each exploit with its state machine map for further analysis and it can be used to understand some exploit effect such as most of the malicious activity that started by adding new process and then edit a particular file it will be finish by opening a particular port. 4. Future work The Process Follower: If a malicious website is trying to infect the user with different malware, it could happen that the user will be affected by different malware at the same time. How could a state monitor know if the file system changes were caused by malware A or B? There has to be a way to distinguish each malware and then follow its changes so that there is no confusion about the activities of malware and we get to know the correct results for each malware along with their changes and activities. Improve the Client honeypot: The client honeypot tools available such as Capture-HPC has a good way to detect the malicious website. However, many more malicious websites are trying to avoid being detected by honeypot. For example, the malicious website could require clicking on the website to lunch the malicious website because all the current client honeypot are just visiting the website and then scanning and monitoring the system for any changes. To improve that, the client honeypot can customize the Internet Explorer which then comes up with a default browser that interacts with the malicious website and adds new toolbar. This could in turn come up with a reason to click on the page when visiting it or even auto filling and submitting any forms on the page to get a better chance for catching hold of any malicious technique that remains hidden from the existing honeypot. References: 1. Allen, K. Scott (2006). ‘Programming Windows Workflow Foundation: Practical Wf Techniques and Examples Using Xaml and C#’. Published by Packt Publishing Ltd. ISBN 1904811213, 9781904811213. Read More