We found that although CTF challenges are common in security red team testing, asking agents to "find the flag" may waste effort on tasks that do not actually test the target's security boundaries.
We found that although CTF challenges are common in security red team testing, asking agents to "find the flag" may waste effort on tasks that do not actually test the target's security boundaries.
Security-specific model training also did not guarantee better results on our tasks.