Class TokenInfoDictionaryBuilder
java.lang.Object
org.apache.lucene.analysis.ko.dict.TokenInfoDictionaryBuilder
-
Field Summary
FieldsModifier and TypeFieldDescriptionprivate final String
private final Normalizer.Form
private int
Internal word id - incrementally assigned as entries are read and added. -
Constructor Summary
Constructors -
Method Summary
Modifier and TypeMethodDescriptionprivate TokenInfoDictionaryWriter
buildDictionary
(List<Path> csvFiles)
-
Field Details
-
offset
private int offsetInternal word id - incrementally assigned as entries are read and added. This will be byte offset of dictionary file -
encoding
-
normalForm
-
-
Constructor Details
-
TokenInfoDictionaryBuilder
TokenInfoDictionaryBuilder(String encoding, boolean normalizeEntries)
-
-
Method Details
-
build
- Throws:
IOException
-
buildDictionary
- Throws:
IOException
-