In this blog post you will learn what OSINT is ( or Open Source Intelligence) and why it is the best way to analyze and understand a scenario.
It is the method used by intelligence agencies to counter threats to national security. In the same way, it is also used by companies to defend their assets and gain a competitive advantage in the market.
But that's not all!
Thanks to the exponential growth of the internet and technologies, even professionals, consultants and law enforcement can use this discipline to achieve results in their work.
What is OSINT (Open Source Intelligence)?
OSINT is a methodology used mainly by institutions and companies. It aims to collect and analyze data from open sources.
Open sources are considered to be all those sources that:
- they have a large audience
- they are public and accessible to everyone
Decades ago, the only open sources were television, radio, newspapers, scientific publications, institutional documents and libraries.
The OSINT scenario continues to evolve with the exponential growth of technology.
Today, OSINT's main source is the internet.
Using the web it is even possible to analyze and indirectly collect other sources of information.
Yes, because open sources are not the only sources of information.
The other sources of information
Governments and corporations have enough resources to afford too:
- HUMINT. Human Intelligence is the collection of information from human sources.
This includes: government and diplomatic personnel, spies and secret services, non-governmental organizations, companies and anything else that allows you to make contact with people who can provide exclusive and confidential information;
- GEOINT and IMINT - Geospatial Intelligence and Imagery Intelligence include all the information that is produced by satellites or reconnaissance aircrafts.
The result of these sources are for maps, satellite images or aerial photos;
- MASINT - Measurement and signature intelligence are collected thanks to tools that allow to pick up physical, biological, nuclear, radio frequency, laser signals, etc..
Here we talk about advanced measurement instruments such as radar or laser;
- SIGINT - Signals Intelligence concerns the interception of signals from communications.
Yes, the American NSA is probably the leader in this field;
- TECHINIT - Technical Intelligence allows to extrapolate information with the analysis of the weapons or instruments of a potential adversary or enemy;
- CYBINT - Cyber Intelligence concerns data coming from cyberspace.
In recent years, the governments of the major world powers have focused on this particular category.
- FININT - Financial Intelligence concerns the analysis of economic transitions.
As you can surely understand most of these sources can be within the reach of exclusive bodies and organizations such as the army or space agencies.
However, if you don't have huge resources, keep reading because I'm about to reveal how thanks to OSINT you can access sources that were once reserved only for multinationals and governments.
But before to deep into the abyss of this fascinating topic, I will explain the dawn of the analysis of open sources
How OSINT was born
In 1941 Japan attacked the American base at Pearl Harbor. The Americans are unprepared and the attack pushes the nation towards a new direction of international affairs.
An old department is renovated and renamed Office of Strategic Services (OSS).
The OSS is nothing more than the forerunner of the current Central Intelligence Agency (CIA). The office deals with special operations abroad with the aim of defending the United States of America from potential threats.
The most interesting thing is that the OSS has a particular research and analysis office that meticulously collects and studies dozens of newspapers, radio channels, publications and books from other countries.
World War II ends and governments and intelligence agencies from around the world begin to classify and categorize various sources of information including OSINT.
The advent of the Cold War and then the 11th of September make us lose interest in this discipline while on the contrary it grows in the private sector thanks to the advent of the Internet.
But in 2009 something happens.
In Iran thousands of citizens take to the streets to protest against the government: the green revolution.
After only one week since the beginning of the protest, more than 60% of the posts and contents on Twitter concern Iran and the Green Revolution.
Before then, no protest had been so well documented and told by ordinary citizens through social media.
After Iran, social media and OSINT are growing. They also play a decisive role during the Arab Spring and continue to do so with the current protest in Hong Kong.
Meanwhile, governments and organizations sharpen their cybernetic weapons using open sources to launch attacks against ISIS, design disinformation campaigns, influence public opinion, study competition or analyze corporate reputation.
Problems with Open Source Intelligence
The most painful problem is the amount of information to be examined.
Security threats have increased exponentially.
In the digital world alone, an organization has thousands of potential cyber threats every day from cybercriminals, thieves, or enemy states.
If we add traditional threats such as organized crime, terrorism, espionage and common crimes, our brains are not able to examine all potential threats at once.
The latest technologies such as Machine Learning and Natural Processing Language can simplify and automate part of our analysis work but to make the decision on what and how to focus on a specific threat is the human being.
The second problem with open sources is their reliability and validity.
Content disseminated through social media or forums is not necessarily valid or even true.
We can use some analytical and technological tools to verify information, however this requires time and patience which unfortunately sometimes risks turning a potential threat into a real one.
One last aspect is speed. We live in a completely hyper-connected way and the delay of only one day can mean the advantage of a competitor or worse, a terrorist attack.
Unfortunately, the overabundance of information and verification results in a longer reaction time to threats.
The advantages of Open Sources and how they can be useful
Despite the disadvantages, open sources can bring great benefits and help us in our work.
Most open sources are free.
Paid sources such as databases (chambers of commerce, scientific publications, etc.) are cheaper than other sources.
Open sources can be used to make important decisions.
The government could use them to understand the impact of a reform.
A company can understand if it is appropriate to develop a new product.
There are those who have used them to help political candidates to be elected.
They are accessible to everyone. You don't need to be a government agency to collect social media data. Anyone can do it with the right tools and technical knowledge.
That's why I want to list some practical examples of how professionals in different roles can use open sources for different purposes.
Open Source Intelligence
Advantages and disadvantages of using open sources
Law enforcement, military and government agencies
Law enforcement, military and government agencies can use open sources to investigate terrorist organisations and organised crime.
In our webinar, we tell you some concrete examples of how we have helped the police of a European state to locate and apprehend suspects. It is also possible to monitor a public event and ensure the safety of participants. Technologies such as Computer Vision even allow us to scan multimedia content in real time and alert us to the presence of a weapon.
But you can also locate how and where a content was posted or check for the presence of an internal webcam without protection and avoid ambushes or the installation of cameras and microphones for surveillance.
Journalists, particularly investigative journalists, can use open sources to assess the reliability of a news item, they can monitor an event or public event at a distance. They can also use open sources for their own journalistic inquiries.
A few years ago I collaborated with Wired for the an investigation into the assets belonging to criminal organizations. The whole work was done with only open sources and completely free with data visualization tools!
Private investigators, lawyers and forensic professionals
Every day billions of pieces of information are shared on the Internet. Such information can help private investigators solve cases of industrial espionage or absenteeism at work. Lawyers and forensic professionals can collect data online for litigation. Although actually printing or making a screenshot of a web page is not enough and does not guarantee validity in court. A real acquisition and crystallization of the digital artefact is necessary.
Every day, researchers of all fields use data from open sources to analyse social and physical phenomena. It used to be an obstacle to even work with ISTAT data. Now thousands of public and private institutions publish online reports, datasets and multimedia content. There are also several websites where you can find datasets to solve some specific problems.
Our analysts often use sites like Kaggle to power our machine learning and deep learning models and we extract public data from social media on a daily basis to analyse our clients' cases.
Physical and logical security professionals can benefit from open sources. Security managers, security analysts and CISOs can scan the web for potential data breaches in their organizations and finally cyber threat intelligence is no longer just a business for multinationals.
In the social media and dark web there are real illegal markets in which personal data are illegally sold and stolen from banks, health organizations, governments etc..
Cyber treat intelligence allows to prevent and mitigate potential threats to the security of the organization.
How to learn with a practical and guided approach
The first time I heard the word Open Source Intelligence was about 10 years ago. I was still an university graduate. I was hooked but books on the subject were scarce, for some teachers the subject was rather agnostic and university and professional courses did nothing but talk about OSINT theory, semantics and arguments that were far from practical.
The only valid courses were restricted to members of some organizations such as NATO and the United Nations.
10 years later some things have improved, there are books, there are good investigators and trainers but there is still no valid and recognized training path.
From what I see inside the university there are finally interesting degree but still teachers too focused on theory.
Analysis of open sources is a multidisciplinary subject and requires a range of skills in computer security, journalism, data science and other more advanced such as international relations, economics or law depending on the organization you work for or the purpose of collecting and analyzing open sources.
I have personally spent 10 years of my life perfecting this method.
Years spent working first as a researcher, security analyst, journalist and project officer for various organizations including the United Nations.
A path that I have summarized and simplified in an online program called Investigation Masterclass.
The program includes 6 modules and over 30 video lessons in which I guide you step by step from the installation of your OSINT lab to the analysis of data passing through the collection of data online in the most hidden corners of the web as the dark web.
It is by far the most complete online program on Open Source Intelligence and Digital Investigations.
Take a look by clicking here now.
I hope you can feel the same interest I have in this incredible discipline.
See you in the next tutorial!