ngrammake − Make ngram model from input count file.
Make ngram model from input count file.
ngrammake [−−options] [in.fst [out.fst]]
−−backoff: type = bool, default = false |
Use backoff smoothing (default: method dependent
−−backoff_label: type = int64, default = 0 |
Backoff label
−−bins: type = int64, default = −1 |
Number of bins for katz or absolute discounting
−−check_consistency: type = bool, default = false |
Check model consistency
−−count_of_counts: type = string, default = "" |
Read count−of−counts from file
−−discount_D: type = double, default = −1 |
Absolute discount value D to use
−−interpolate: type = bool, default = false |
Use interpolated smoothing (default: method dependent)
−−method: type = string, default = "katz" |
One of: "absolute", "katz", "kneser_ney", "presmoothed", "unsmoothed", "katz_frac", "witten_bell" (default)
−−norm_eps: type = double, default = 0.001 |
Normalization check epsilon
−−witten_bell_k: type = double, default = 1 |
Witten−Bell hyperparameter K
−−help: type = bool, default = false |
show usage information
−−helpshort: type = bool, default = false |
show brief usage information
−−tmpdir: type = string, default = "/tmp" |
temporary directory
−−v: type = int32, default = 0 |
verbose level
−−fst_align: type = bool, default = false |
Write FST data aligned where appropriate
−−fst_default_cache_gc: type = bool, default = true |
Enable garbage collection of cache
−−fst_default_cache_gc_limit: type = int64, default = 1048576 |
Cache byte size that triggers garbage collection
−−fst_read_mode: type = string, default = "read" |
Default file reading mode for mappable files
−−fst_verify_properties: type = bool, default = false |
Verify fst properties queried by TestProperties
−−fst_weight_parentheses: type = string, default = "" |
Characters enclosing the first weight of a printed composite weight (e.g. pair weight, tuple weight and derived classes) to ensure proper I/O of nested composite weights; must have size 0 (none) or 2 (open and close parenthesis)
−−fst_weight_separator: type = string, default = "," |
Character separator between printed composite weights; must be a single character
−−save_relabel_ipairs: type = string, default = "" |
Save input relabel pairs to file
−−save_relabel_opairs: type = string, default = "" |
Save output relabel pairs to file
−−end_symbol: type = string, default = "</s>" |
Class label for sentence start
−−start_symbol: type = string, default = "<s>" |
Class label for sentence start
−−fst_compat_symbols: type = bool, default = true |
Require symbol tables to match when appropriate
−−fst_field_separator: type = string, default = " " |
Set of characters used as a separator between printed fields
−−fst_error_fatal: type = bool, default = true |
FST errors are fatal; o.w. return objects flagged as bad:
e.g., FSTs − kError prop. true, FST weights − not a Member()
−−ngram_error_fatal: type = bool, default = true |
NGram errors are fatal; o.w. return objects flagged as bad:
e.g., NGramModel::Error() returns true.