3 Commits (e885aff70defbc95510c545ab586389842c910f9)

Author SHA1 Message Date
  Reece H. Dunn dd90d3812d tokenizer.c: Support general symbol tokens. 8 years ago
  Reece H. Dunn 786575c6ed tokenizer.c: Support general punctuation tokens. 8 years ago
  Reece H. Dunn 683579f403 Make the tokenizer.h API public. 8 years ago
  Reece H. Dunn 55bfbb4754 tokenizer.c: Support ellipsis tokens. 8 years ago
  Reece H. Dunn b847df63b5 tokenizer.c: Support semicolon tokens. 8 years ago
  Reece H. Dunn af7e8fc5a3 tokenizer.c: Support colon tokens. 8 years ago
  Reece H. Dunn 7560070dcd tokenizer.c: Support comma tokens. 8 years ago
  Reece H. Dunn c9199cfacb tokenizer.c: Support exclamation mark tokens. 8 years ago
  Reece H. Dunn 128ceaff6a tokenizer.c: Support question mark tokens. 8 years ago
  Reece H. Dunn 8f62e18324 tokenizer.c: Support full stop tokens. 8 years ago
  Reece H. Dunn d50f3f2fa5 tokenizer.c: Support word tokens. 8 years ago
  Reece H. Dunn d093513b65 tokenizer.c: Add an options parameter to the tokenizer_reset API. 8 years ago
  Reece H. Dunn c41ac642fa tokenizer.c: Tokenise Zp codepoints as paragraphs. 8 years ago
  Reece H. Dunn df6ca7a22c tokenizer.c: Support whitespace tokens. 8 years ago
  Reece H. Dunn 7602c9ac18 tokenizer.c: Support linux newlines. 8 years ago
  Reece H. Dunn bce44316bb Create a basic tokenizer API using a structure that mirrors the TtsTokenizer interface in the tts-dev-studio project. 8 years ago
  Reece H. Dunn 5118891f20 windows: Visual Studio 2013 compatibility. 8 years ago
  Reece H. Dunn 527f2bc6b2 Add M_PI compatibility when using the msvc standard library. 9 years ago
  Reece H. Dunn 9c5820f72a Create an initial compatibility shim for unistd.h. 9 years ago
  Reece H. Dunn c22550afb5 Use the NetBSD getopt_long implementation on Windows. 9 years ago
  Reece H. Dunn 9c7179189b Add a stdint.h compatibility shim. 9 years ago