Harpoon is a tool to automate threat intelligence and open source intelligence tasks. It is written in Python 3 and organised in plugins so the idea is to have one plugin per platform or task. The code is on Github, feel free to open issues and propose Pull Requests.

Harpoon ?

For the past year and a half I have been pretty busy doing threat intelligence and open source intelligence on several malware operations. Threat Intelligence mainly relies on passive DNS/malware databases on one side, and databases on malicious activity on the other side. The objective is to map an attack’s infrastructure and if possible link it with other malicious activities. Some threat intelligence platforms are accessible to everyone (like OTX or RobTex) while others are commercial with or without free access (like VirusTotal or PassiveTotal). In the end, a large part of this activity is about looking for information in different platforms. Many people have tried to create a platform centralizing information from other platforms, but we always end up having yet another platform to consider during the research.

New standards problem totally applies to Threat Intelligence (xkcd 927)

Open Source Intelligence (aka OSINT) on the other side is way more diverse. The objective is to get as much information as possible on someone or a group of people using any data source openly accessible on Internet. Of course there are some interesting platforms (like SpyOnWeb) but you may also have to do research in social media, cache platforms and many other random tasks depending where your investigation leads you.

In all that, a large number of tasks are completely manual and it sucks. At first, I tried to create random python scripts to automate some of these tasks but it became quickly a mess : too many scripts, some in python 2 some in python 3, some using config files and some getting API key in parameters… After a while, I started to organize these scripts as modules for a tool called Harpoon and after some months of using it, I think it is time to make it open source to see if it can help other people.

Some notes on the principles behind this tool :

• Python 3 only, python 2 is dead, I am not even trying to support it
• Many OSINT tools try to gather as much information as possible from an indicator (domain or email) without really any interest on where it comes from. Harpoon does not follow this philosophy. It mostly allows you to realise a single task per command (with a couple of more general commmands using several tools). I think it is really important during an investigation to understand where an information comes from and how reliable it is.
• In many cases, I also wanted to explore APIs to see what was possible with them, and I ended up rewriting some libraries (like SpyOnWeb) just because I wanted to understand exactly what it did and how. So I reinvented the wheel many times and I am totally fine with it.
• Harpoon is organized into subcommands that are easy to implement, these commands rely on internal or external libraries. These commands also use a single configuration file that you need to complete manually when an API key is needed.
• This tool is not perfect, it only does what I needed to automate, it is likely buggy and has a long list of things I would like to implement one day (but it may never happen). Feel free to open issues or propose Pull Requests.

This post will be boring, mainly because I try to be as exhaustive as I can to complete the limited existing documentation about the tool. Feel free to jump to the sections you are interested to read