Contextual information and the architecture of RNNs