Serious network security requires not just threat detection but threat hunting. The latter requires more human expertise and intuition, and it’s harder to perform with a high level of quality. An organization that puts a strong emphasis on security needs to make cyber threat hunting a regular part of its network protection plan.
Like other buzzwords, “cyber threat hunting” isn’t always used consistently, and some businesses apply it to what should properly be called threat detection and monitoring. Others make a sincere effort at threat hunting but don’t get it right. It’s an art that requires technical experience, knowledge of threat intelligence, and imagination. A good threat hunter is a valuable asset.
There’s an idea in popular culture that cyber threat hunting is “hackers hunting other hackers back.” This isn’t totally wrong, but it suggests a dramatic image that is quite different from the day-to-day work. Let’s look at what’s really involved.
What is cyber threat hunting?
Threat hunting isn’t the same as intrusion detection. Detection is a heavily automated process that relies on threat intelligence, signature identification, behavioral analysis, machine intelligence, and other techniques to identify penetration attempts that have succeeded or are likely to. Attackers constantly look for ways to get around the capabilities of intrusion detection systems. Threat hunters look for threats that can slip through the gaps.
Cyber Threat hunting, then, is the art of discovering ways of targeting a network that evade existing defensive mechanisms. It covers potential attacks, ones in progress, and ones that have already exploited weaknesses in the network.
Unlike threat detection, threat hunting doesn’t work from known signs of malicious activity. It goes searching for signs. It relies heavily on tools and information repositories, but it can’t be reduced to algorithms and mechanical techniques. It’s not a replacement for automated security but a way to make sure it doesn’t miss anything.
How does threat hunting work?
The cyber threat hunter faces a huge challenge. A network, even one belonging to a small business, has devices of many kinds. A huge amount of data passes through them. The quantity of information is the reason that computer security systems rely so heavily on automation. A SIEM system can look at millions of data items while a human analyst examines one.
Then why is it even worth bringing direct human investigation into the picture? There are several answers to this question.
- Criminals are constantly developing new techniques to evade detection. The next big threat will use techniques that current defenses aren’t good at catching. Spearphishing took everyone by surprise when it was first used against executives. Supply chain compromises that infect trusted software, such as the one that affected multiple federal agencies in 2020, are hard to spot.
- Security systems have notification thresholds. Intrusion detection and SIEM systems don’t report every indication of trouble; if they did, administrators would be swamped with false alarms and would ignore real threats. Attackers take advantage of this by staying under the radar. They attempt lateral movement or communicate with command and control servers only intermittently, hoping to avoid the attention of the security systems.
- Threats can masquerade as legitimate activity. If an insider participates in an attack or a user password is stolen, it’s possible to make data theft look like legitimate usage. Analysis of user activity can detect abnormal patterns, but if the intruder avoids being too greedy, the deviation may not be enough to trigger an alert.
- Some systems get neglected when setting up security. They might be older machines that people have forgotten are still active. They might be deemed too unimportant to worry about. They can still be gateways to valuable information, easy to attack.
- Every network is unique. It has valuable assets and ones that are less important. It has systems that may not look important but give access to valuable information. Security systems should reflect these facts, but they aren’t always well-tuned enough.
Threat hunters need to be deeply familiar with their network, so they can spot anything that looks wrong. They need to review the latest threat intelligence to find out how criminals are evading security measures. They have to consider the network’s purpose and its key assets, so they can focus on protecting what is most important.
A necessary skill, one where humans still beat computers, is the ability to anticipate an adversary’s thinking. Knowing how their systems work and what an intruder would want from them, they can think of likely attack plans. They can ask themselves what would happen if someone attempted those plans. What signs would it leave? Then they can look for those signs.
The use of threat intelligence is important. We may think of cyber threat intelligence as something security software uses to identify patterns in logs and indications of compromise, but it’s equally valuable for keeping threat hunters informed. Reports of new tactics are particularly valuable. The threat hunter can ask whether existing systems protect against them and investigate more closely if there are doubts.
Threat hunting is about looking when you don’t know what to look for. If you knew, you could create a test for it in software. It’s a matter of following up hints and suggestions, much as a detective looks for unspecified clues that could lead an investigation in a new direction. The analyst looks at the indications and forms a hypothesis. They ask: If this hypothesis is true, what indications should be present? Are they present? If not, might the data lead to a different hypothesis? It’s a cyclic process.
Threat intelligence comes from crowdsourcing; an analyst discovers a new attack and reports its characteristics to the community. A devious technique may evade both signature-based identification and most behavior-based techniques, but eventually someone will spot it.
Myths and misconceptions about cyber threat hunting
The process is a difficult one, and everyone who regularly hunts for threats develops a personal approach. It shouldn’t be surprising that misconceptions have grown about how it works and what it accomplishes. Here are ten myths that get in the way of understanding threat hunting and doing it effectively.
1. “It’s a basically manual process.”
Finding threats in a network requires going through a huge amount of data. Trying to do this without powerful tools would be hopeless. Expert hunters rely on analytic tools that show changes from baseline activity and present information in an easy-to-view form. They review reports generated by SIEM software to identify issues that are suspicious but don’t rise to the level of an alert.
Intelligence-driven tools sift through threat intelligence data to find the hostile techniques and tactics that are most applicable to an organization’s situation. Banks have a different set of security priorities from hospitals, which have different priorities from retail sites, and so on. Threat hunters use this information to form hypotheses relevant to their networks and figure out what to look for when testing them.
Educated guessing is an important part of the process, but without the right tools, the guesses won’t be educated.
2. “It can be automated with artificial intelligence.”
At the other extreme, some people think that an AI system can do the work of threat hunters with equal or better results. Someday this may be true, but it hasn’t happened yet. The hardest task for AI is figuring out human psychology. Understanding the motives and plans of intruders is a key part of threat hunting, and people do it best.
Analysts can look at their network, consider its assets and public visibility, and ask, “If I wanted to break into this system, what would I be after and how would I do it?” People can do that better than software.
3. “It’s an ad hoc, seat-of-the-pants process.”
This myth is a different way of running to the opposite extreme. Threat hunting involves guesswork, but it requires structure and discipline. Analysts run their tools regularly, compare the results with previous ones, and go through lists of threat intelligence data.
There’s a definite methodology for investigating possible threats. The first step is formulating a hypothesis based on the situation. An investigation follows, looking for data that will confirm or contradict the hypothesis. If it’s confirmed, specific identification and remediation follow. If it turns out to be mistaken, a record is entered to avoid redundant investigation.
4. “It’s a replacement for IDS or SIEM.”
This myth is a dangerous one. The majority of attacks use known methods, and software can catch them faster and with less effort than human investigators can. Tools such as firewalls, intrusion detection systems (IDS), and SIEM (security information and event management) are a network’s primary defense.
Threat hunting is supposed to spot exploitation of the gaps in a network’s defense. Ideally, there are no gaps, and the search will come up empty. In reality, there are always weak spots, but the attacks focusing on them are few compared with the overall spectrum. They’re the most dangerous attacks, though, and analysts should be able to focus on them while the software handles the more routine ones.
5. “It’s a one-time activity.”
No one who understands cybersecurity thinks one round of threat hunting will take care of the problem for all time, but making it a regular part of the process and budget could take some pushing. A period of hunting that doesn’t turn anything up might create the impression that it’s not accomplishing anything.
Threat hunting is a long-term commitment. The people who practice it get better over time as they test hypotheses and learn where software is most likely to miss problems. Catching just one threat that would otherwise have gone undetected can save a company millions. Dry periods aren’t a reason to drop it. New threat intelligence shows up every day, and if possible, analysts should do a round of threat hunting every day.
6. “Penetration testing is better.”
Pen testing is a valuable technique for discovering weaknesses in one’s own network. It has some features in common with threat hunting. It takes a theory about weaknesses and puts it to the test. People with experience in one area will have an advantage when they try to learn the other.
But it’s not an either-or choice. Pen testing has the advantage of showing that a weakness exists and demonstrating how it can be exploited. However, it can’t tell whether actual hostile activity is taking place. Finding weaknesses before they’re exploited is best, but finding exploits in progress is urgent if they exist. Discovering and fixing weaknesses using simulated attacks means the threat hunter has less to do, but it doesn’t eliminate the need for the task. Having both provides a defense in depth.
7. “It’s basically a hacker vs. hacker game.”
Like many myths, this one has a bit of truth to it. The defender has to anticipate the attacker’s thoughts. Some threat hunters even find their way onto criminal discussion boards to discover their current schemes. But that description makes the job sound a lot more adventurous than it is.
It helps to “think like a criminal” when forming hypotheses, but the process from there is entirely different. The goal isn’t to exploit a weakness but to remove a threat. It isn’t just to find a vulnerability but to close it. It’s challenging work, but not the James Bond stuff which some imagine.
8. “It’s only about discovering active penetration.”
Discovering malware and compromised accounts is important, but it’s better to discover a penetration attempt in progress before it can do any harm. Discovering ongoing attempts and footholds that haven’t yet done damage is important, too. Many threats rely on the downloading of additional malware and lateral movement to other systems before they give their creators anything of value.
Threat hunting may turn up previously undiscovered weaknesses that haven’t been exploited yet. Whatever it finds is an opportunity to fix a problem and make the systems safer.
9. “It’s something any tech person can pick up.”
This claim isn’t entirely false. Anyone with decent IT skills can do some threat hunting and may find hostile activity that needs remediation. Doing it well, though, is a specialized skill that requires education and practice. A skilled hunter knows what to look for and is familiar with the tools,
Analytics and SIEM systems bring a huge mass of information down to a comprehensible set of data. It takes practice to get the most out of them. It takes the ability to go through threat intelligence digests and extract the information which is relevant to their situation. It takes forensic skills to proceed from a general idea to the discovery of the data that will confirm or reject it.
10. “It isn’t worth the time and effort.”
It’s true that cyber threat hunting is a labor-intensive process that won’t turn up problems as regularly as automated processes do. This might seem better to put the entire cybersecurity budget into making software protection as robust as possible. This mindset doesn’t consider the fact that the most advanced and dangerous attacks are the ones that bypass existing defenses. The people who go to that much trouble are interested in a serious return on their investment.
Threat hunting may not discover as many issues as monitoring, SIEM, and IDS do. However, if those systems miss an attack vector, they’re missing something important. We can’t rely entirely on machines to keep us safe from devious penetration schemes. Human understanding has to be part of the defensive plan.
Threat hunting and threat intelligence
The role of cyber threat intelligence in cyber threat hunting is a crucial one. It lets threat hunters learn from each other’s discoveries. A central threat intelligence data warehouse pools the discoveries of many analysts, shortening the time from the discovery of novel threats to the creation of remedies.
This is where SIEM and the search for threats come together. Firewalls and monitoring systems report some information, but SIEM is designed for thorough examination and analysis of all logged activity. It should be set up to receive regular threat intelligence updates. By analyzing network log data in the light of known threats, it can report anomalies and match them up to specific tactics and techniques.
If hostile activity is evident, SIEM will issue an alert and may initiate an automated remedy. In addition, it presents summaries and reports for analysts. Security analysts can identify patterns and changes that suggest dangerous activity. The data collected will speed the testing of their hypotheses.
For example, SIEM may find that certain email messages fit the profile of a newly reported threat, but without enough certainty to block them automatically. Analysts can look at the information to decide if they need to add a filter blocking such messages. At the same time, they can discover who is being targeted and how the spammers are tailoring their messages. They’ll be able to respond not just to spear phishing in the abstract, but to the exact tactics being used against their organization.
Cyber threat hunting is a difficult task, requiring the best people and software tools. Most organizations don’t have the resources to do it properly on their own. It requires a top-notch security operations center (SOC) and state-of-the-art analytic tools. BitLyft provides SOC as a Service, giving you the expertise you need to identify and stop threats before they can do damage. It’s part of the BitLyft AIR® managed detection and response service and platform, a complete solution of technical expertise and tools to identify and eliminate cyber threats. With expert threat hunters on your side, you can be confident your network is getting the best protection available.