5 Commits (b897ff5aa832a8ec11fea71769f4cb4d644ffb0c)

Author SHA1 Message Date
  Reece H. Dunn 1c8ed9c190 tokenizer.c: Support mac newlines. 8 years ago
  Reece H. Dunn 7602c9ac18 tokenizer.c: Support linux newlines. 8 years ago
  Reece H. Dunn bce44316bb Create a basic tokenizer API using a structure that mirrors the TtsTokenizer interface in the tts-dev-studio project. 8 years ago
  Reece H. Dunn 5c6bc0e556 Armenian emphasis mark (U+055B) is used for interjections, so treat it as an exclamation mark. 8 years ago
  Reece H. Dunn 1c4ce3dcd3 tokenizer.c: create and use a clause_type_from_codepoint function, with tests. 8 years ago