How to register porter tokenizer in python

266 Views Asked by At

How is the default tokenizer 'porter' in fts3 module registered.

one way to register user defined tokenizers is fts.register_tokenizer() but what are its arguments?

since porter being a in built tokenizer does it even need to be registered?

1

There are 1 best solutions below

2
CL. On BEST ANSWER

If your Python has the Porter tokenizer compiled in, you do not need to register it.

To register a user-defined tokenizer, you have to call the fts3_tokenizer() SQL function with a pointer to a C structure containing C function pointers; this cannot be easily done from Python.