of SFI IT Academic Festival

I will present experiments, where language models of different architectures need to solve a multi-step task, but doing all the steps in memory, without writing them down. Recurrent architectures enable such hidden reasoning, which is risky, because it means that we don't always have access into the model's "thoughts". On the other hand transformers (f.e. GPT) are forced to write down intermediate steps, which usually gives us such access (but not always)