Best Practices in Threat Intelligence

A common difficulty in threat intelligence is to improve existing analyses and especially how to do it efficiently. One of the main questions to ask is:

"What will be the target audience of the improved analysis and the objective thereof?"

The following three answers could come to mind.

In the 1st case, MISP includes a mechanism to propose changes to the original creator, a mechanism MISP refers to as proposals. By using proposals, you can propose a change to the value or the context of an attribute (such as a typographic error in an IP address, missing contextual information, type of the information, the category or the removal of an IDS flag). The proposal will be sent back to the original author who can decide to accept or discard it.

The advantages of using the proposal system include the lack of a need to create a new event as well as the process itself being very simple and fast. However, it assumes that the party providing the improvements is willing to lose control over the proposed data. This is pretty efficient for small changes but for more comprehensive changes, especially those that include non-attribute information such as galaxy clusters or objects, the event extension is more appropriate.

Apart from being more suitable for more comprehensive changes, the 2nd scenario is also a great fit for the extended event functionality, allowing users wanting to provide additional information or an alternate view-point with the opportunity of creating a self-contained event (which can have its own custom distribution rules) that references the original analysis. This information can be shared back to the original author or kept within a limited distribution scope such as a specific sector, a trust group or as internal information for the organisation providing the additional information.

In the 3rd scenario your use-case might be highly automated, e.g. scripted processing of events and attributes via PyMISP and the end-consumer is mainly another automated process, e.g. Intrusion Detection System, 3rd part visualization tool etc. This, for automagic reasons, becomes exponentially unreliable. What is primal in this case is to fully understand what the IDS flag in MISP does and how it impacts attributes. Further on, it is even more important to fully understand the entire tool-chain, cradle-to-grave style. Where does the data come from (cradle) where does it go to (grave) and what processes "touch" the data as it flows through, small diagrams can help tremendously to visualize the actual data-flow. Those diagrams will mostly be of use once unexpected results occur, or other errors appear somewhere in the chain.

Contribution comes in various shapes and sizes.

Information which is often distributed within sharing communities are the following:

There are several factors to successful and efficient intelligence sharing. Certainly, one major aspect is the quality of the indicators (or observable depending on the definition you use), stored as attributes within a MISP event itself. However, it does not stop there. Even the most viable information gained by a shared event can render itself complete useless if not classified and tagged accordingly. One feature which enables a uniformed classification is implemented in MISP as tags. Currently, there are two types of tags, which differ in the respective place they are set.

A frequent use-case for placing additional tags on attribute level would be to lower the confidence in certain attributes. If the event is classified with a high confidence tag, some indicators e.g. legit-but-compromised domains or popular filenames should be labeled with a lowered confidence class. There are several real world examples where this or similar attribute specific tagging has proven to be worthwhile.

Most of the tags are organised in dedicated MISP Taxonomies. Those schema dictate how tags should look like and how they are to be applied in certain conditions. There are many general details on this topic which can be read up on in the main MISP Taxonomy GitHub repository. Currently, there are more than 60 different taxonomies available, each of them containing a number of different tags, which are steadily increasing and evolving. There are a lot of advantages in having such a vast variety of tags, e.g. there is one tag for each known associated malware type.

However, this sheer amount of tags can lead to two main concerns, over-tagging and miss-tagging. Beginners can be overwhelmed with the large number of available tags, and might miss exactly the required taxonomy to properly label the to be shared data. As a site administrator it is thus important to enable the taxonomies that are known to the users on the MISP instance, (or to remote organizations you might sync with).

Over-tagging in most cases only leads to an overwhelming visual appearance. Miss-tagging, however, is a critical step into mis-usage of shared data. The best and most devastating example would be the miss classification of an event. In dedicated and private sharing groups it is quite usual to share intelligence labeled as „for your company only“. This data must not leave the boundaries of this virtual border of the recipient’s firm.

To prevent this kind of mistake, the traffic light protocol (aka TLP) and its respective taxonomy can be used and thus complementing the mitigation in the note below.

There are multiple solutions to solve the issue of missing additional information about the shared content. One of them is the following list of tags which are deemed to be the minimal subset at the start of any event or the individual attributes. sharing platform. The list below is in order of importance.

Analysis or reports are often shared together with technical details, but often lack the associated overall confidence level. To ascertain this confidence level you can use for example the MISP MISP Taxonomies called admiralty-scale and/or estimative-language. This is a very human way to describe either globally an event or individual indicators of an event, with a set of easy to read human tags. (e.g: admiralty-scale:source-reliability="a/b/c…​", estimative-language:likelihood-probability="almost-no-chance", estimative-language:confidence-in-analytic-judgment="moderate") Generally it is good practice to do this globally for the event as this will enrich the trust/value if set. Using this in an automated way is also possible but without human intervention, or AI that actually works, not recommended. Also, on events with hundreds of attributes this is cumbersome and perhaps unfeasible and will just frustrate operators. The obvious side-effect of this approach is that automation will be the overall benefactor too upping the trust on that level too.

Thus, adding confidence or estimative probability has multiple advantages such as:

Complement analysis with contrary evidences is also very welcome to ensure the original analysis and the hypotheses are properly evaluated.

Keeping track of the advancement of an analysis, of what has been done or still needs to be done, is important in order not to forget anything on either side and to ensure work is not performed redundantly by accident. It is essential to have a method to keep these information clear and concise.

One of the possible methodologies is to use tags to mark the information and convey the current state of an analysis.

For instance the MISP Workflow [Taxonomy] allows the user to describe the state of an analysis, as complete or incomplete. Moreover, it can be used to clearly specify what still needs to be done using the todo tags. The workflow taxonomy is separated into two parts. One part is related to the actions to be done (todo) and the other part is about the current state of the analysis(state) such as incomplete, draft or complete.

The first tool we can use to classify information are tags and taxonomies