GitHub Repo stars Token Monster is an ungreedy subword tokenizer and vocabulary generator, enabling language models to run faster, cheaper, smarter and generate longer streams of text.

image

Share link! 📋
Link copied!
See the main site!