All of daniel kornis's Comments + Replies

If you used dropout during training, it might help to explain why redundancy of roles is so common, You are covering some features every training step.

2Neel Nanda
These models were not trained with dropout. Nice idea though!