Media Attribution for Open Source Investigations

Open source investigations leverage publicly available information, typically derived from web-based sources like websites, web scraping, social media posts, and APIs to research topics of public interest such as the prosecution of wars, criminal behavior, government wrongdoing and much else.

This work is largely being done manually by both paid investigators and a global network of volunteers. One major task of open source investigations involves volunteers painstakingly reviewing media collected in the field and verifying its authenticity and details through a series of manual steps, including comparing it to public and known image repositories, cross-referencing details in the image or video with known details of the suspected location and time of the media such as the weather and the location where it was taken, and even profiling the source of the media. This is a necessary component both of identifying the nature of media that is part of an investigation and in validating that any attribution (e.g., a claimed source or a description provided by a source,) is true or false. This consumes massive amounts of time and effort in human terms.

Our tool allows users to submit images via an intuitive, human-usable interface and receive predictions via a suite of trained classification models about key qualities of the scene shown in the image: where was the image taken and what are the weather, season, and time of day depicted in the image? These predictions are then cross-checked against a suspected geographic and time/date attribution provided by the user to provide a useful indicator to investigators about the likelihood that that attribution is correct. Fundmentally, investigators will use our tool to help them answer the following question: “Is it likely that this image depicts the place and time I suspect it does?” This will greatly streamline the process of media attribution, making this effort more accurate and less time-consuming, contributing to the reach and quality of open source investigations.