Ashwin Sreevatsa
AI security researcher
about
I am interested in research to ensure that AI systems stay aligned and beneficial as they become more capable. Currently, I work on designing and red-teaming safeguards to monitor and prevent AI systems from taking misaligned or dangerous actions.
Most recently, I have worked on AI control as an ERA:AI Fellow and as a CBAI Fellow.
Before getting into AI security, I built machine learning infrastructure at Google for 3 years. I received my Bachelor’s degree in Computer Science from University of Michigan.
Research interests: alignment, ai control, scalable oversight, alignment auditing, multi-agent dynamics
research
For a full list of publications, see my Google Scholar page.
* denotes equal contribution
-
Can Base Models be Effective Monitors for AI Systems?
(Preliminary investigation) We explore whether base models (i.e. models without post-training) can effectively monitor the behavior of AI agents. We find that such monitors trained via a midtraining + SFT pipeline can often match the performance of post-trained models. These base models show minimal degradation in general performance and outperform post-trained models on alignment evals.
-
Basic Legibility Protocols Improve Trusted Monitoring
One natural approach to monitoring a strong model we don't trust is to have a weaker model that we do trust evaluate all of the untrusted model's actions. However, this weaker model will often fail to understand the stronger, untrusted model's actions. We study "legibility protocols" that force the stronger model to take actions that are easier for the weaker model to understand and oversee. We show that these protocols can improve the effectiveness of the weaker model's monitoring.
-
Prompt Attacks Reveal Superficial Knowledge Removal in Unlearning Methods
We investigate whether popular unlearning methods actually remove knowledge from a model or merely suppress it. We find that simple black-box prompt attacks can often reveal "unlearned" knowledge, indicating that these methods may not be as effective as claimed.