torch:subsequent-mask

(torch:subsequent-mask sequence-length)

Returns the causal (look-ahead) mask of a sequence: 1.0 strictly above the diagonal, shaped (1 sequence-length sequence-length) so it broadcasts over the batch. Position i may not attend to any j > i.

Like torch:padding-mask it is a raw linalg array. Filling the masked scores with -infinity before torch:softmax is the masked-attention idiom, and the masked weight comes out as exactly 0.0.