Grammalecte  History of graphspell-js/tokenizer.js of 6e9947d6aa172770

History of file graphspell-js/tokenizer.js at check-in 6e9947d6aa172770

2020-12-02
07:57
[graphspell] tokenizer update file: [5dde8e1191] check-in: [9678e9208c] user: olr, branch: trunk, size: 5504 [annotate] [blame] [check-ins using] [diff]
2020-11-30
15:15
[graphspell][fx] update tokenizer and lexicographer: add symbols and emojis file: [8e6d24c94a] check-in: [b3448ac17f] user: olr, branch: trunk, size: 5289 [annotate] [blame] [check-ins using] [diff]
2020-11-25
20:50
[graphspell][fr][fx] rename tokens file: [9c02b80583] check-in: [6aae160f81] user: olr, branch: trunk, size: 5079 [annotate] [blame] [check-ins using] [diff]
2020-10-02
09:32
[graphspell] tokenizer: token UNDERSCORE file: [7838839417] check-in: [e2313363fe] user: olr, branch: trunk, size: 5088 [annotate] [blame] [check-ins using] [diff]
2020-10-01
14:50
[graphspell] tokenizer: exclude underscore from WORD token [fr] ajustements, écriture inclusive file: [4d88fd06ec] check-in: [cfbaf0ad4e] user: olr, branch: trunk, size: 5018 [annotate] [blame] [check-ins using] [diff]
2020-09-02
09:07
[graphspell] tokenizer: token OTHER as fallback file: [2ede633a83] check-in: [e201630bf5] user: olr, branch: trunk, size: 4970 [annotate] [blame] [check-ins using] [diff]
2020-05-07
10:35
[graphspell] tokenizer and suggestion engine: other apostrophes file: [0e7b889227] check-in: [b68161b398] user: olr, branch: trunk, size: 4909 [annotate] [blame] [check-ins using] [diff]
2020-04-20
18:02
[graphspell] tokenizer: combining diacritics recognition and NFC normalization file: [efabea9cdf] check-in: [3ef2bdb736] user: olr, branch: trunk, size: 4895 [annotate] [blame] [check-ins using] [diff]
2020-01-12
09:33
[graphspell][js] fix tokenizer for HTML markers file: [16e7826100] check-in: [b6c3593f76] user: olr, branch: trunk, size: 4817 [annotate] [blame] [check-ins using] [diff]
2019-09-01
08:22
[graphspell] tokenizer: handles all kinds of apostrophes file: [5a3085a08e] check-in: [1bdedd3133] user: olr, branch: trunk, size: 5025 [annotate] [blame] [check-ins using] [diff]
2019-08-30
09:45
[graphspell] tokenizer: consider presqu’ and quelqu’ as separate words file: [0f58ee1cf3] check-in: [0f0bc77645] user: olr, branch: trunk, size: 5015 [annotate] [blame] [check-ins using] [diff]
2019-07-30
20:06
[graphspell][fr] update tokenizer: ordinals file: [2f6a31a87a] check-in: [dcdb32b057] user: olr, branch: trunk, size: 5001 [annotate] [blame] [check-ins using] [diff]
2019-06-09
06:32
[graphspell] tokenizer: update HOUR file: [282849cd0e] check-in: [1bc78ce87f] user: olr, branch: trunk, size: 4969 [annotate] [blame] [check-ins using] [diff]
2019-05-22
08:24
[graphspell][js] tokenizer: tag SEPARATOR -> PUNC file: [3e1b39918e] check-in: [75bf92c9c2] user: olr, branch: trunk, size: 4939 [annotate] [blame] [check-ins using] [diff]
07:59
[core][graphspell][js] fix regex for \w substitution file: [5b1f96af0f] check-in: [e40149ad94] user: olr, branch: trunk, size: 4949 [annotate] [blame] [check-ins using] [diff]
2019-05-14
15:19
[graphspell] tokenizer: update for HOUR tokens file: [4d861c8e0e] check-in: [63672ef096] user: olr, branch: trunk, size: 4949 [annotate] [blame] [check-ins using] [diff]
2019-05-02
08:16
[graphspell] tokinizer: update file: [f8ee073676] check-in: [7d30bbec37] user: olr, branch: trunk, size: 4946 [annotate] [blame] [check-ins using] [diff]
07:50
[graphspell] tokinizer: update file: [d9b8ecbdba] check-in: [ed3b7acf68] user: olr, branch: trunk, size: 4918 [annotate] [blame] [check-ins using] [diff]
2019-02-22
11:53
[graphspell][fr] tokenisation: +signes €$# (faux positif) file: [c05f88b98c] check-in: [365d3554c7] user: olr, branch: trunk, size: 4900 [annotate] [blame] [check-ins using] [diff]
2018-12-26
18:25
[graphspell][js] fucking \w substitution again file: [1d91386df2] check-in: [7f03f6c55a] user: olr, branch: trunk, size: 4890 [annotate] [blame] [check-ins using] [diff]
18:10
[graphspell][js] fucking \w substitution again file: [eb39282111] check-in: [78254a6629] user: olr, branch: trunk, size: 4874 [annotate] [blame] [check-ins using] [diff]
2018-10-11
15:13
[js] Revert syntax change file: [1dc7e255ce] check-in: [9ae8f0a042] user: IllusionPerdu, branch: nodejs, size: 4864 [annotate] [blame] [check-ins using] [diff]
2018-10-10
09:19
Some change to javascript to work in node file: [541689c69f] check-in: [a3687f4fd3] user: IllusionPerdu, branch: nodejs, size: 4863 [annotate] [blame] [check-ins using] [diff]
2018-09-17
12:06
[graphspell][js] tokenizer: update \w replacement again file: [88bacac87d] check-in: [72f63bddd2] user: olr, branch: rg, size: 4819 [annotate] [blame] [check-ins using] [diff]
10:26
[graphspell] tokenizer: new regex for ordinals (JS still sucks) file: [d579c92281] check-in: [a479b36582] user: olr, branch: rg, size: 4815 [annotate] [blame] [check-ins using] [diff]
09:00
[graphspell] tokenizer: add chars to \w replacement (JS still sucks) file: [49d1f9c490] check-in: [c185b3fc04] user: olr, branch: rg, size: 4803 [annotate] [blame] [check-ins using] [diff]
2018-09-14
09:48
[graphspell][js] tokenizer: fix var init file: [aac6560c8a] check-in: [52d64fd395] user: olr, branch: rg, size: 4775 [annotate] [blame] [check-ins using] [diff]
2018-09-11
18:55
[graphspell][js] tokenizer: don’t use spaces as tokens, yield information token (start/end) file: [d2289444f9] check-in: [d12872816f] user: olr, branch: rg, size: 4799 [annotate] [blame] [check-ins using] [diff]
2018-08-07
08:09
[graphspell][core][fx][tb][js] get rid of console.log() substitutes, assuming now default console is available from everywhere (eh, print now works as expected in JS! The JS pile of shit is decreasing a little) file: [04c83f2aa3] check-in: [8e5a2f5900] user: olr, branch: trunk, size: 4322 [annotate] [blame] [check-ins using] [diff]
2018-07-17
06:42
[graphspell] tokenizer: remove hyphen in number detection (always considered as a separate sign) file: [4a5b091820] check-in: [6950f5898f] user: olr, branch: rg, size: 4387 [annotate] [blame] [check-ins using] [diff]
2018-06-28
08:26
[graphspell][core] tokenizer: rename ACRONYM tokens to WORD_ACRONYM file: [2fadfb42f5] check-in: [ccbbecbd1b] user: olr, branch: rg, size: 4391 [annotate] [blame] [check-ins using] [diff]
08:00
[graphspell] tokenizer: rename ORDINAL tokens to WORD_ORDINAL file: [a185b00a68] check-in: [20dbc28ded] user: olr, branch: rg, size: 4381 [annotate] [blame] [check-ins using] [diff]
07:53
[graphspell][core] tokenizer: rename ELPFX tokens to WORD_ELIDED file: [8dd855b1b3] check-in: [a1b165e276] user: olr, branch: rg, size: 4376 [annotate] [blame] [check-ins using] [diff]
2018-06-18
20:13
[graphspell][fix] tokenizer: new signs file: [5f94dc04ea] check-in: [303c416286] user: olr, branch: rg, size: 4370 [annotate] [blame] [check-ins using] [diff]
20:12
[graphspell] tokenizer: new signs file: [1ab7d18bed] check-in: [da0d308818] user: olr, branch: rg, size: 4372 [annotate] [blame] [check-ins using] [diff]
2018-06-17
13:11
[graphspell] tokenizer: update ordinals file: [2e21533239] check-in: [4be13a74c3] user: olr, branch: rg, size: 4280 [annotate] [blame] [check-ins using] [diff]
2018-05-18
13:11
[graphspell] tokenizer: add token index and avoid punctuations aggregation file: [9bd60cca8a] check-in: [be6d99bbdc] user: olr, branch: rg, size: 4232 [annotate] [blame] [check-ins using] [diff]
2018-02-20
08:40
[graphspell] spellchecker: add parseParagraph() file: [bdd895b918] check-in: [7616aa7ef9] user: olr, branch: trunk, size: 4443 [annotate] [blame] [check-ins using] [diff]
2018-02-13
15:44
[core][cli][server][graphspell][fx] use spellchecker instead of ibdawg file: [c3f0ee8c90] check-in: [18db5d65f0] user: olr, branch: multid, size: 4756 [annotate] [blame] [check-ins using] [diff]
2017-12-25
08:52
[build][tb] fix imports for Thunderbird file: [d6429837c4] check-in: [5822c38ee9] user: olr, branch: graphspell, size: 4740 [annotate] [blame] [check-ins using] [diff]
2017-12-24
18:58
Renamed gc_core/js/tokenizer.js → graphspell-js/tokenizer.js. [build][js] move files from gc_core to graphspell file: [a34a81c6e5] check-in: [bdfc6fd5e9] user: olr, branch: graphspell, size: 4729 [annotate] [blame] [check-ins using] [diff]
2017-11-12
13:22
[core][fx] tokenizer: +acronyms file: [a34a81c6e5] check-in: [fa1205c098] user: olr, branch: Lexicographe, size: 4729 [annotate] [blame] [check-ins using] [diff]
2017-11-02
10:58
Ajout dans le tokenizer du ~ dans la detection des dossier linux, et distintion entre les deux types de dossier windows/linux avec le changement dans le lexicographe file: [9d996e312d] check-in: [11f1414b5b] user: IllusionPerdu, branch: Lexicographe, size: 4613 [annotate] [blame] [check-ins using] [diff]
2017-10-26
05:49
[core] tokenizer: better regex for URLs and folders file: [9bb6ea03fb] check-in: [843c0244bc] user: olr, branch: trunk, size: 4601 [annotate] [blame] [check-ins using] [diff]
2017-10-25
18:34
[core][bug] fix tokenizer for URL file: [de468b4358] check-in: [ee7d44a3ee] user: olr, branch: trunk, size: 4583 [annotate] [blame] [check-ins using] [diff]
2017-10-24
11:00
[core] tokenization: folders file: [9622b0a610] check-in: [35c48d42a8] user: olr, branch: trunk, size: 4561 [annotate] [blame] [check-ins using] [diff]
2017-08-08
19:11
[core] add inline config for JSHint assuming es6 synthax and some global file: [81088a0177] check-in: [b89dc82bc4] user: IllusionPerdu, branch: webext2_fix, size: 3799 [annotate] [blame] [check-ins using] [diff]
16:49
[core][fr] change typeof(exports) before the require block file: [c0e5d0e035] check-in: [6c8ee6edcf] user: IllusionPerdu, branch: webext2_fix, size: 3748 [annotate] [blame] [check-ins using] [diff]
2017-08-06
00:44
[core][js] ajout des ; oubliés et ajustement des ; en trop (commit erroné) file: [fcd058bf6a] check-in: [b08f2ef338] user: IllusionPerdu, branch: webext2_illusion, size: 3748 [annotate] [blame] [check-ins using] [diff]
2017-08-02
06:12
[core][js] conditional requires and variable renaming to avoid overridding file: [02266194ef] check-in: [7e545790c3] user: olr, branch: webext2, size: 3749 [annotate] [blame] [check-ins using] [diff]
2017-07-29
08:06
[core][js] test if <exports> exists file: [4c307bc446] check-in: [c55ae247e7] user: olr, branch: webext, size: 3044 [annotate] [blame] [check-ins using] [diff]
2017-07-27
10:10
[core][js] test if variable <exports> exists file: [a6594366c3] check-in: [f9a034e6ce] user: olr, branch: trunk, size: 3695 [annotate] [blame] [check-ins using] [diff]
2017-07-13
07:08
[core][js] tokenizer: yield separator one by one file: [8720e6c367] check-in: [492169bd6f] user: olr, branch: kill_innerHTML, size: 3649 [annotate] [blame] [check-ins using] [diff]
2017-07-12
11:03
[fr][js] màj: lexicographe, tokenizer file: [4eb5311054] check-in: [074cb33c80] user: olr, branch: kill_innerHTML, size: 3338 [annotate] [blame] [check-ins using] [diff]
2017-04-25
11:51
Added: commit 1 file: [d06151ccd7] check-in: [2fd7dc4dd5] user: olr, branch: trunk, size: 2998 [annotate] [blame] [check-ins using]