extract: 2025-08-00-mccaslin-stream-chembio-evaluation-reporting #1363

Closed
leo wants to merge 1 commit from extract/2025-08-00-mccaslin-stream-chembio-evaluation-reporting into main
4 changed files with 50 additions and 1 deletions
Showing only changes of commit 666da253ac - Show all commits

View file

@ -27,6 +27,12 @@ The structural point is about threat proximity. AI takeover requires autonomy, r
The International AI Safety Report 2026 (multi-government committee, February 2026) confirms that 'biological/chemical weapons information accessible through AI systems' is a documented malicious use risk. While the report does not specify the expertise level required (PhD vs amateur), it categorizes bio/chem weapons information access alongside AI-generated persuasion and cyberattack capabilities as confirmed malicious use risks, giving institutional multi-government validation to the bioterrorism concern.
### Additional Evidence (extend)
*Source: [[2025-08-00-mccaslin-stream-chembio-evaluation-reporting]] | Added: 2026-03-19*
STREAM framework proposes standardized reporting for ChemBio dangerous capability evaluations, directly addressing the disclosure gap in the domain where AI bioweapon capability is most concerning. The multi-stakeholder process (including government experts) signals potential for eventual regulatory adoption of standardized ChemBio evaluation disclosure.
---
Relevant Notes:

View file

@ -35,6 +35,12 @@ The alignment implication: transparency is a prerequisite for external oversight
Expert consensus identifies 'external scrutiny, proactive evaluation and transparency' as the key principles for mitigating AI systemic risks, with third-party audits as the top-3 implementation priority. The transparency decline documented by Stanford FMTI is moving in the opposite direction from what 76 cross-domain experts identify as necessary.
### Additional Evidence (extend)
*Source: [[2025-08-00-mccaslin-stream-chembio-evaluation-reporting]] | Added: 2026-03-19*
The need for STREAM demonstrates that even when labs conduct dangerous capability evaluations, lack of standardized disclosure formats prevents independent verification of evaluation quality. The 23-expert consensus that standardized reporting is necessary confirms that current model reports contain insufficient detail for third-party assessment.
---
Relevant Notes:

View file

@ -0,0 +1,24 @@
{
"rejected_claims": [
{
"filename": "ai-model-reports-lack-standardized-dangerous-capability-evaluation-disclosure-preventing-independent-assessment.md",
"issues": [
"missing_attribution_extractor"
]
}
],
"validation_stats": {
"total": 1,
"kept": 0,
"fixed": 1,
"rejected": 1,
"fixes_applied": [
"ai-model-reports-lack-standardized-dangerous-capability-evaluation-disclosure-preventing-independent-assessment.md:set_created:2026-03-19"
],
"rejections": [
"ai-model-reports-lack-standardized-dangerous-capability-evaluation-disclosure-preventing-independent-assessment.md:missing_attribution_extractor"
]
},
"model": "anthropic/claude-sonnet-4.5",
"date": "2026-03-19"
}

View file

@ -7,9 +7,13 @@ date: 2025-08-01
domain: ai-alignment
secondary_domains: []
format: paper
status: unprocessed
status: enrichment
priority: medium
tags: [evaluation-infrastructure, dangerous-capabilities, standardized-reporting, ChemBio, transparency, STREAM]
processed_by: theseus
processed_date: 2026-03-19
enrichments_applied: ["AI lowers the expertise barrier for engineering biological weapons from PhD-level to amateur which makes bioterrorism the most proximate AI-enabled existential risk.md", "AI transparency is declining not improving because Stanford FMTI scores dropped 17 points in one year while frontier labs dissolved safety teams and removed safety language from mission statements.md"]
extraction_model: "anthropic/claude-sonnet-4.5"
---
## Content
@ -53,3 +57,12 @@ PRIMARY CONNECTION: [[AI lowers the expertise barrier for engineering biological
WHY ARCHIVED: Provides evidence of emerging standardization for dangerous capability evaluation reporting. The multi-stakeholder process (government, academia, AI companies) signals potential for eventual adoption.
EXTRACTION HINT: Focus on the disclosure gap: labs currently report their own dangerous capability evaluations without standardized format, preventing independent assessment of rigor.
## Key Facts
- STREAM stands for 'Standard for Transparently Reporting Evaluations in AI Model Reports'
- STREAM was developed by 23 experts from government, civil society, academia, and frontier AI companies
- STREAM initial focus is on chemical and biological (ChemBio) dangerous capability evaluations
- STREAM includes a 3-page reporting template and 'gold standard' examples
- STREAM was proposed in August 2025
- STREAM has two stated purposes: (1) practical guidance for AI developers presenting evaluation results with clarity, (2) enabling third parties to assess whether model reports contain sufficient detail about ChemBio evaluation rigor