Evaluating Dual Use Risk in ML Projects
You'll learn: How to systematically assess whether your ML system could be weaponized or misused before deployment.
The Core Challenge
Every powerful ML capability you build is a double-edged sword. A face recognition model could help reunite lost children with families—or enable mass surveillance without consent. The question isn't whether dual use risk exists, but how much and what to do about it.
Framework for Risk Assessment
Threat modeling adapts cybersecurity practices to ML contexts. Start by asking:
- Who are the threat actors? (Nation-states, criminals, activists, competitors)
- What harmful capabilities does this model enable? (Generate deepfakes, automate phishing, bypass safety systems)
- What's the barrier to entry? (Open weights + code = low; API-only access = medium; closed research = high)
- What's the magnitude of potential harm? (Individual privacy loss vs. destabilizing democracy)
Risk assessment matrices help you score each dimension. Think of it like airport security: you're triaging threats based on likelihood × impact.
Key Questions During Development
- Could fine-tuning repurpose this model for harm with <$1000 compute?
- Does the model lower the skill floor for dangerous activities (e.g., writing exploit code)?
- Are there similar capabilities already publicly available?
- What's the worst-case scenario if access controls fail?
Consider a protein-folding model: beneficial for drug discovery, but potentially exploitable for toxin design. Your assessment might recommend restricted access, watermarking outputs, or monitoring usage patterns.
Key Takeaway: Dual use risk evaluation is not a checkbox—it's an ongoing threat modeling process that weighs technical capabilities against adversarial misuse scenarios, informing deployment decisions before harm occurs.