huggingface-tokenizers

github:eyadsibai/ltk · first committed 2026-01-15 · source · badge

Use when "tokenizers", "HuggingFace tokenizer", "BPE", "WordPiece", or asking about "train tokenizer", "custom vocabulary", "tokenization", "subword", "fast tokenizer", "encode text

Skill Health: not yet scored

Listed, not yet scored. This skill has no public model test yet. It is in the scoring queue; submitted skills go first, then the most-installed.

Version history

1 commit to this file on GitHub (newest first).

2026-01-15 · commit 8cb168d · refactor: Reorganize into 7 domain-based plugins with ltk: command prefix

Archived snapshots