In April 2018, The Wall Street Journal reported missile strikes on Syrian government bases that killed dozens not long after Israel had been blamed for attacking an Iranian air station. The article stated, “An Israeli open-source intelligence site posted purported satellite imagery on Twitter, saying that the target of the attack was an Iranian base recently erected north of Hama airport.”
Indeed, open-source intelligence (OSINT) frameworks and tools are on the rise. The term refers to data collected - via various scripts, scraping and multiple tools often working in framework- from available sources to be used by those seeking information in an intelligence context. Be it the blogs we browse, broadcasts we watch, or publications we read, there is an endless supply of information available that is hidden within the content or linkable to or from the content.
OSINT draws from the internet, traditional mass media (e.g., television, radio, newspapers, magazines), specialized journals, conference proceedings, think tank studies, photos and geospatial information (e.g., maps and commercial imagery products), as well as other information repositories.
Justin Nordine is an IT security professional who created an OSINT framework that allows anyone to use the associated tools of the framework to gather informational artifacts from a multitude of sources for a variety of purposes. It gathers information about and from sources related to a cadre of information repositories and sources which include training, documentation, OpSec, threat intelligence, classifieds, public records, IP addresses, business records and many more. Developers can use such a framework, as can journalists, forensic investigators, and anyone else seeking to deep dive into the archaeology of something or someone.
Often the framework can be used not just for the information it provides, but also to find holes or vulnerabilities within an enterprise or network that can be patched or filled for security purposes. “Blue Team” security professionals can use it to find competitive information or seek intelligence. Companies may use it to gather corporate intelligence.
Open-Source Spirit, Collaborative Results
Nordine, in a podcast on Timothy Deblock’s Technology and Media site, posits that since the creation of his OSINT framework, response from others, in IT and other disciplines, has been overwhelming.
He welcomes visitors, outside of infosec, to use it and aid in the collaboration of building it further for the good of all. He’s hoping it can be, in the spirit of open source, a fully collaborative effort whereby others contribute and add to the development of the framework. The code is posted on www.github.com, where anyone can access and clone OSINT applications.
As Nordine describes, his OSINT framework began after he listened to a presentation by Johnny Long about “no-tech” hacking; he described the many ways he was getting information without attacking another computer system. This happened in the 2006-2007 timeframe. Long went on to write a book in 2008, along with some co-authors, entitled No Tech Hacking: A Guide to Social Engineering, Dumpster Diving, and Shoulder Surfing.
Nordine said his OSINT framework had its genesis from his own exploration into how to proceed in finding useful information. More specifically, if he could find one piece of information such as an email address, IP address, or domain name, how could it be used to “tile back in,” as he calls it, into one big picture?
From that genesis, he developed disparate tools that can work within the framework to perform other functions. Many of these tools were overlapping. Then he narrowed the tools down by contemplating other tools that he had successfully used. This evolved into a “mind map” that subsequently morphed into a framework that served his intention by allowing him to enter information and then use it to branch off to more information.
This mind map has tiers of information. Any observer who wants to know more information about anything then starts digging. "This is the 'no-tech hacking' form of thinking," he says on the podcast. For example, finding user names may be cross-referenced into different sites to gain more information. Nordine notes that anyone can use this tact to gather information. The framework originated from his information security background where he gathered information to know what to defend against.
Since the framework was offered, others have begun using it. This includes fact checkers, journalists, background checkers, and many different industry interests that Nordine says he wasn’t exactly expecting. He believes it can be used in a more formal way to gather information and can be used as a tool to find information.
Nordine says future plans for the framework include building it further by adding new tools continuously. Then he expects to spend more time on some of the site’s code and functionality upgrades. “Blue Team” security professionals can use it to find competitive information or seek intelligence. Companies may use it to gather corporate intelligence.
So, what are some real-world examples of how OSINT is being used in today's world?
As stated, many tools have been developed that use OSINT. Here’s a sampling of other tools that use OSINT for specific functions of search and information extraction:
Says Christian Berg, CEO at Paliscope: “It’s a way of solving crimes, but also a means of knowing who you are dealing with and separating facts from false claims. However, it is not a matter of just collecting and storing large amounts of data; online investigations have to be carried out with structure and purpose, especially in light of the new European General Data Protection Regulation (GDPR).”
While many tools built from OSINT are commercially available, the framework and its capabilities, in the spirit of open source, is free. Justin Nordine says his framework requires constant updating, but hopes such a tool will continue the collaboration of open-source developers to make OSINT a valuable asset for the many who stand to benefit from it and its capabilities, just as the Israeli site did when it posted satellite imagery of the Syrian missile attack on Twitter.
Want more? Visit MISTI's upcoming InfoSec World Conference & Expo in Orlando, Florida where some of the top leaders in the security industry will be sharing their knowledge.