Class WordSegmenter

Construct a WordSegmenter with automatically selecting the best available LSTM or dictionary payload data, using compiled data. This does not assume any content locale.

Note: currently, it uses dictionary for Chinese and Japanese, and LSTM for Burmese, Khmer, Lao, and Thai.

See the Rust documentation for new_auto for more information.

Construct a WordSegmenter with automatically selecting the best available LSTM or dictionary payload data, using compiled data.

Note: currently, it uses dictionary for Chinese and Japanese, and LSTM for Burmese, Khmer, Lao, and Thai.

See the Rust documentation for try_new_auto for more information.

Construct a WordSegmenter with automatically selecting the best available LSTM or dictionary payload data, using a particular data source.

Note: currently, it uses dictionary for Chinese and Japanese, and LSTM for Burmese, Khmer, Lao, and Thai.

See the Rust documentation for try_new_auto for more information.

Construct a WordSegmenter with dictionary payload data for Chinese, Japanese, Burmese, Khmer, Lao, and Thai, using compiled data. This does not assume any content locale.

Note: currently, it uses dictionary for Chinese and Japanese, and dictionary for Burmese, Khmer, Lao, and Thai.

See the Rust documentation for new_dictionary for more information.

Construct a WordSegmenter with dictionary payload data for Chinese, Japanese, Burmese, Khmer, Lao, and Thai, using compiled data.

Note: currently, it uses dictionary for Chinese and Japanese, and dictionary for Burmese, Khmer, Lao, and Thai.

See the Rust documentation for try_new_dictionary for more information.

Construct a WordSegmenter with dictionary payload data for Chinese, Japanese, Burmese, Khmer, Lao, and Thai, using a particular data source.

Note: currently, it uses dictionary for Chinese and Japanese, and dictionary for Burmese, Khmer, Lao, and Thai.

See the Rust documentation for try_new_dictionary for more information.

Construct a WordSegmenter with no support for scripts requiring complex context dependent word breaks (Chinese, Japanese, Burmese, Khmer, Lao, and Thai), using compiled data. This does not assume any content locale.

See the Rust documentation for new_for_non_complex_scripts for more information.

Construct a WordSegmenter with no support for scripts requiring complex context dependent word breaks (Chinese, Japanese, Burmese, Khmer, Lao, and Thai), using compiled data.

See the Rust documentation for try_new_for_non_complex_scripts for more information.

Construct a WordSegmenter with no support for scripts requiring complex context dependent word breaks (Chinese, Japanese, Burmese, Khmer, Lao, and Thai), using a particular data source.

See the Rust documentation for try_new_for_non_complex_scripts for more information.

Construct a WordSegmenter with LSTM payload data for Burmese, Khmer, Lao, and Thai, using compiled data. This does not assume any content locale.

Note: currently, it uses dictionary for Chinese and Japanese, and LSTM for Burmese, Khmer, Lao, and Thai.

See the Rust documentation for new_lstm for more information.

Construct a WordSegmenter with LSTM payload data for Burmese, Khmer, Lao, and Thai, using compiled data.

Note: currently, it uses dictionary for Chinese and Japanese, and LSTM for Burmese, Khmer, Lao, and Thai.

See the Rust documentation for try_new_lstm for more information.

Construct a WordSegmenter with LSTM payload data for Burmese, Khmer, Lao, and Thai, using a particular data source.

Note: currently, it uses dictionary for Chinese and Japanese, and LSTM for Burmese, Khmer, Lao, and Thai.

See the Rust documentation for try_new_lstm for more information.

Segments a string.

Ill-formed input is treated as if errors had been replaced with REPLACEMENT CHARACTERs according to the WHATWG Encoding Standard.

See the Rust documentation for segment_utf16 for more information.