tokenizer:decode
(tokenizer:decode tk ids)
The text the token ids stand for. It takes a WHOLE sequence, so that it round-trips tokenizer:encode: a multi-byte character straddles two tokens routinely, and a SentencePiece encode opens with a dummy prefix space that this takes back off. A generation loop that decodes one token at a time wants tokenizer:decode-bytes instead.