Scoring Methodology
Every SamuAI score is built from observable, verifiable signals — not opinions. Here's exactly how we calculate risk scores for the tools you use every day.
Methodology v2.0 · Last updated June 2, 2026
How It Works
When you scan a tool, SamuAI pulls its publicly available metadata — manifest files, permission declarations, store listings, dependency trees, and developer information. We never execute extension code or access your personal data.
That metadata is analyzed across four pillars, each producing a score from 0 to 10. These are combined into a single overall score using a weighted formula that prioritizes the most concrete, verifiable signals.
The Four Pillars
Each pillar scores a different dimension of risk. The overall score is a weighted composite:
Permission Risk
35%What access does this tool request, and does it make sense for what the tool does? A password manager needs access to all websites — a calculator doesn't.
What we analyze
- Every declared permission, classified by risk tier (critical, high, medium, low)
- Contextual analysis: is each permission expected for this type of tool?
- Chrome: manifest permissions and host permissions
- npm/MCP: risky dependencies (child_process, keytar) and install scripts
- VS Code: activation events, contributes capabilities, bundled dependencies
Standards referenced
Developer Trust
20%How identifiable and established is the publisher? A verified developer with millions of users and years of history is more accountable than an anonymous first-time publisher.
What we analyze
- Platform verification status (Chrome Web Store, VS Code Marketplace)
- Known entity recognition (Google, Microsoft, 1Password, etc.)
- Install count / download volume as a proxy for community vetting
- Account or package age — how long they've been in the ecosystem
- Web presence and contact information
Standards referenced
Data Privacy
30%What data leaves the tool, where does it go, and is it encrypted? We trace data flows from permissions and dependencies — not from developer self-reports.
What we analyze
- Third-party data destinations (external APIs, analytics, ad networks)
- Encryption status of data in transit
- Sensitive data types: credentials, browsing history, email, clipboard
- Network-capable dependencies with no documented data flows
- OAuth scopes and externally connectable domains
Standards referenced
Policy Compliance
15%Does this tool have a privacy policy, and does the policy match what we observe? A policy that claims 'we collect no data' while the tool requests cookies and browsing history is a red flag.
What we analyze
- Privacy policy existence
- Claim-vs-behavior mismatch detection
- Data collection transparency
- Library exception: npm packages without install scripts aren't penalized for missing policies
Standards referenced
The Formula
Permission Risk and Data Privacy together account for 65% of the overall score because these are the concrete, directly observable signals. We can verify exactly what permissions a tool requests and what data flows exist. Developer Trust and Policy Compliance are important but rely on more indirect signals.
Contextual Permission Analysis
Not all permissions are equal in all contexts. SamuAI detects what category a tool belongs to and adjusts permission scoring accordingly:
An ad blocker requesting webRequest — it literally cannot function without it. Minimal score impact (0.15x weight).
A password manager requesting clipboardRead — common and reasonable. Reduced score impact (0.4x weight).
A writing assistant requesting <all_urls> — not typical, worth reviewing. Full score impact (1.0x weight).
A tab manager requesting geolocation — no legitimate reason for this. Amplified score impact (1.5x weight).
This contextual system currently covers 14 tool categories. Tools that don't match a known category are analyzed with the default (no context adjustment) weights.
Industry Standards Referenced
| Standard | Publisher | Applies To |
|---|---|---|
| OWASP Top 10 (2021) | OWASP Foundation | Permission Risk, Data Privacy |
| MITRE ATT&CK | MITRE Corporation | Permission Risk |
| NIST SP 800-53 Rev. 5 | NIST | Developer Trust |
| NIST Privacy Framework | NIST | Data Privacy |
| SLSA v1.0 | OpenSSF / Google | Developer Trust |
| CWE | MITRE Corporation | Permission Risk |
| SSDF | CISA / NIST | Developer Trust |
| GDPR | European Union | Data Privacy, Policy Compliance |
| CCPA | State of California | Policy Compliance |
| FTC Act Section 5 | FTC | Policy Compliance |
Limitations
SamuAI provides automated risk assessment based on publicly available metadata. It is not a substitute for a full security audit. Key limitations:
- Static analysis only
We analyze manifests, metadata, and dependency trees. We do not execute code or perform dynamic/behavioral analysis.
- Inferred data flows
Data flows are inferred from permissions and dependencies, not observed at runtime. A tool with a network library may not actually make external requests.
- No code review
We do not decompile, deobfuscate, or review source code. Malicious behavior hidden in obfuscated code would not be detected.
- Surface-level policy analysis
Privacy policy checks verify existence and basic claim-vs-behavior consistency, not full legal compliance review.
- Store availability dependency
Chrome extension analysis relies on Chrome Web Store and CRX download availability. If both fail, only a partial analysis is possible.
Have questions about how a specific score was calculated?