On average, cyberattackers operate on networks for 120 days before detection, and typically they are found by outsiders. This inability to spot attackers already on networks contributes significantly to the worldwide the cost of cyber-crime, which was $600 Billion in 2017, followed by 2018 - the year cyber-crime metastasized into attacks on civil infrastructure. Cyber-crime effects not only economics, but also erodes our democratic societies, emboldens authoritarian despots and will factor deeply in the coming global order. This is a crisis of organizations, human talent and decision-making more than a gap in technology. The FOURSight platform from FOUR18 Intelligence is the first incentive-driven crowdsourcing market platform to solve these problems systematically by engaging cyber experts and targeted enterprises in collaborating, predicting, hunting and assessing cyber threats continuously for sport and profit. It is the epitome of boldness, promising to use crowdsourcing to turn the tables on cyber criminals, and no other system like it has ever existed before.
In 2018, under support from the US Department of Homeland Security, FOUR18 Intelligence successfully piloted FOURSight in performing Digital Forensics and Incident Response (DFIR) on network data containing evidence of the tactics used in sophisticated cyberattacks. The pilot demonstrated that experienced cyber analysts could be engaged to collaboratively analyze a dataset from a simulated cyberattack, accurately identify and confirm the tactics and indicators of attack left behind by simulated attackers. At the same time it demonstrated the benefits of a unique quantitative form of crowdsourcing in improving these experts' understanding and confidence in the attack analysis by sharing findings via an experience that is at once collaborative and competitive. This proved our crowdsourcing platform can successfully overcome the key obstacles of inadequate access to skilled people and technology that prevent many enterprises from being able to identify and respond to cyberattacks on their networks. And because FOURSight was architected using best practice frameworks that are in general use across the cybersecurity analysis community, it scales to address the full breadth of cyber threats. For the first time a platform exists to proactively and continuously crowdsource attack probabilities, detection methods, mitigation response plans and better talent and processes - driven by the most lasting of motivations: economics, competition and self-assessment.
With only 2 weeks of recruiting a total of 37 individuals participated in the pilot by creating accounts on the system - about 20% of these became active users. Users participated by betting points provided to them on the probabilities of outcomes in return for winning a market return determined by their speed, value of information, accuracy and confidence. Using this game model, we generated in one week, from a cold start, 315 bets in our system on a simulated attack on 143 different questions. The platform and network correctly identified attacker techniques in every tactic used by our simulated attackers and created initial estimates on the effectiveness of 95 different countermeasures against these techniques. We also proved that cyber analysts would share their best insights by collecting 165 insightful comments and 38 artifacts indicating compromises in the network. This blending of qualitative and probabilistic analysis is a unique strength of the FOURSight crowdsourcing model as one of our power users commented: "I absolutely think that commenting on your rationale for trades was one of the greatest parts of the whole experience for me."
The platform and network performed impressively in terms of discrimination metrics. As a system it produced a True Positive rate of .76 and True Negative rate of .79. The False Positive rate was .24 and the False Negative rate was .18. Significantly, Brier Score metrics were collected for all questions and showed strong error reduction as the crowdsourcing proceeded with successive bets.
Just as important as the network metrics, the system proved that the incentive model based on gamification and market economics can alter the status quo of the critically under resourced cybersecurity labor market. Our professional cyber analyst users made this point clearly: “I had a terrific experience with the pilot! ...I really like the idea of being rewarded for being fast with initial findings, especially as more people get involved, simply because it’s easier to tell what the direction of something might be based on what other people are offering. ... I found the concept of thinking in probabilities… was extremely useful when determining how to proceed with the investigation. … Overall, I had a ton of fun working though elements of an actual incident… I can definitely see myself using the system to train and improve my method... ” FOURSight and crowdsourcing can manufacture capacity from critically under-resourced cybersecurity labor pools.
Finally, the platform proved that co-opetive crowdsourcing can be a powerful force in improving human performance. The platform was extremely effective at scoring and discerning talent. As one user put it: "(it) made it really clear to me where my weaker areas were in terms of DFIR, and I already have training that I’m looking to go through to help shore up those aspects." The value of crowdsourcing was well recognized too: "I really felt there was a collaborative push to get the right info out and to help clarify why someone was trading the way they were, and I think it will only help everyone develop a common language and approach to IR (Incident Response), which is the goal." "I think prompting the user to provide their rationale for a bet was a good way to try and keep people honest in their betting but it also provided the ability for other users to assess whether they agreed or disagreed with the other person’s rationale. This could then be factored in to how they placed their own bets." This proved important in assessing analyst skill too as it was easy to judge skill by whether an analyst's trades and rationale were coherent and confident or disjointed. There is no hiding in FOURSight, but with every user anonymous analysts are safe to learn!
The FOURSight project and its powerful, bold, bounty-based crowdsourcing platform is positioned to change the game in 2019 and bring crowdsourcing to the fore as a vital new tool in perhaps the most important and defining struggle of the digital age.