ControlSift
ControlSift studies a common security-review problem: a document can sound relevant to a control without proving the control actually worked.
The project compares a traditional text classifier with a small language model on synthetic cybersecurity evidence examples.
It publishes both the successes and the failures, while keeping human review authoritative.
One important caveat: the traditional model and the completed language-model runs were tested on different versions of the benchmark. Both sets of results are published, but they aren't treated as a fair head-to-head ranking.
Synthetic research · failures kept · comparison limits disclosed · human judgment stays authoritative
ControlSift
ControlSift tests whether a traditional text classifier and a small AI model can distinguish strong cybersecurity evidence from weak or misleading paperwork. It publishes what worked, what failed, and what the results do and don't show.
Synthetic study · failures kept · comparison limits disclosed · human review required
ControlSift asks whether AI can help a reviewer separate evidence that proves a security control worked from documents that merely sound relevant.
A policy can describe what should happen without proving that it did happen. A screenshot may show one account but not the full population. A system export or log can provide much stronger operational proof. ControlSift studies those differences.
The project built a synthetic set of security-evidence examples and tested several approaches: a simple baseline, a traditional text classifier, and a small language model used three different ways. Partway through the work, the benchmark was made harder, so some results come from version 1.0 and others from version 1.1.
On the original test set, giving the language model a few examples worked better than either giving it no examples or fine-tuning it. On the harder v1.1 test set, the traditional classifier scored about 0.53 on the project's main metric. Because those results came from different benchmark versions, ControlSift doesn't claim an overall winner.
Start with the question you care about. The technical detail is there when you want it.
All published scores, which benchmark version each score came from, and which comparisons are fair.
How the synthetic test cases were built, how each approach was tested, and how performance was measured.
Specific cases where the approaches disagree or fail. Start here if the mistakes matter more to you than the average score.
What the project is for, where it can fail, how the research was governed, and why a human reviewer remains in charge.
Environment details and commands for validating the data and rerunning the public experiments.
The complete MMC / Google DeepMind AI Research Foundations submission, including the report, sources, slides, video, and supporting work.
Source code, notebooks, machine-readable results, governance files, scripts, tests, and project history.
Suppose the control says every privileged account must use multi-factor authentication.
An identity-system export showing every privileged account, whether MFA is enabled, when the data was captured, and the population covered.
SUFFICIENTA written standard that says privileged accounts must use MFA, but provides no account-level evidence that the requirement was actually followed.
INSUFFICIENTAmong the Gemma tests run on v1.0.0, giving the model a few examples worked better than either giving it no examples or fine-tuning it. The fine-tuned run also had trouble returning usable labels consistently. On the harder v1.1.0 benchmark, the traditional TF-IDF model scored about 0.53 macro F1. Because those results come from different benchmark versions, ControlSift doesn't claim that the traditional model beats Gemma overall. That comparison still needs a same-version rerun.
Every important claim points back to a public artifact.