History of file gc_core/js/tokenizer.js at check-in 8835b5ebba820828
|
2020-12-02
| ||
| 07:57 | [graphspell] tokenizer update file: [5dde8e1191] check-in: [9678e9208c] user: olr, branch: trunk, size: 5504 [annotate] [blame] [check-ins using] [diff] | |
|
2020-11-30
| ||
| 15:15 | [graphspell][fx] update tokenizer and lexicographer: add symbols and emojis file: [8e6d24c94a] check-in: [b3448ac17f] user: olr, branch: trunk, size: 5289 [annotate] [blame] [check-ins using] [diff] | |
|
2020-11-25
| ||
| 20:50 | [graphspell][fr][fx] rename tokens file: [9c02b80583] check-in: [6aae160f81] user: olr, branch: trunk, size: 5079 [annotate] [blame] [check-ins using] [diff] | |
|
2020-10-02
| ||
| 09:32 | [graphspell] tokenizer: token UNDERSCORE file: [7838839417] check-in: [e2313363fe] user: olr, branch: trunk, size: 5088 [annotate] [blame] [check-ins using] [diff] | |
|
2020-10-01
| ||
| 14:50 | [graphspell] tokenizer: exclude underscore from WORD token [fr] ajustements, écriture inclusive file: [4d88fd06ec] check-in: [cfbaf0ad4e] user: olr, branch: trunk, size: 5018 [annotate] [blame] [check-ins using] [diff] | |
|
2020-09-02
| ||
| 09:07 | [graphspell] tokenizer: token OTHER as fallback file: [2ede633a83] check-in: [e201630bf5] user: olr, branch: trunk, size: 4970 [annotate] [blame] [check-ins using] [diff] | |
|
2020-05-07
| ||
| 10:35 | [graphspell] tokenizer and suggestion engine: other apostrophes file: [0e7b889227] check-in: [b68161b398] user: olr, branch: trunk, size: 4909 [annotate] [blame] [check-ins using] [diff] | |
|
2020-04-20
| ||
| 18:02 | [graphspell] tokenizer: combining diacritics recognition and NFC normalization file: [efabea9cdf] check-in: [3ef2bdb736] user: olr, branch: trunk, size: 4895 [annotate] [blame] [check-ins using] [diff] | |
|
2020-01-12
| ||
| 09:33 | [graphspell][js] fix tokenizer for HTML markers file: [16e7826100] check-in: [b6c3593f76] user: olr, branch: trunk, size: 4817 [annotate] [blame] [check-ins using] [diff] | |
|
2019-09-01
| ||
| 08:22 | [graphspell] tokenizer: handles all kinds of apostrophes file: [5a3085a08e] check-in: [1bdedd3133] user: olr, branch: trunk, size: 5025 [annotate] [blame] [check-ins using] [diff] | |
|
2019-08-30
| ||
| 09:45 | [graphspell] tokenizer: consider presqu’ and quelqu’ as separate words file: [0f58ee1cf3] check-in: [0f0bc77645] user: olr, branch: trunk, size: 5015 [annotate] [blame] [check-ins using] [diff] | |
|
2019-07-30
| ||
| 20:06 | [graphspell][fr] update tokenizer: ordinals file: [2f6a31a87a] check-in: [dcdb32b057] user: olr, branch: trunk, size: 5001 [annotate] [blame] [check-ins using] [diff] | |
|
2019-06-09
| ||
| 06:32 | [graphspell] tokenizer: update HOUR file: [282849cd0e] check-in: [1bc78ce87f] user: olr, branch: trunk, size: 4969 [annotate] [blame] [check-ins using] [diff] | |
|
2019-05-22
| ||
| 08:24 | [graphspell][js] tokenizer: tag SEPARATOR -> PUNC file: [3e1b39918e] check-in: [75bf92c9c2] user: olr, branch: trunk, size: 4939 [annotate] [blame] [check-ins using] [diff] | |
| 07:59 | [core][graphspell][js] fix regex for \w substitution file: [5b1f96af0f] check-in: [e40149ad94] user: olr, branch: trunk, size: 4949 [annotate] [blame] [check-ins using] [diff] | |
|
2019-05-14
| ||
| 15:19 | [graphspell] tokenizer: update for HOUR tokens file: [4d861c8e0e] check-in: [63672ef096] user: olr, branch: trunk, size: 4949 [annotate] [blame] [check-ins using] [diff] | |
|
2019-05-02
| ||
| 08:16 | [graphspell] tokinizer: update file: [f8ee073676] check-in: [7d30bbec37] user: olr, branch: trunk, size: 4946 [annotate] [blame] [check-ins using] [diff] | |
| 07:50 | [graphspell] tokinizer: update file: [d9b8ecbdba] check-in: [ed3b7acf68] user: olr, branch: trunk, size: 4918 [annotate] [blame] [check-ins using] [diff] | |
|
2019-02-22
| ||
| 11:53 | [graphspell][fr] tokenisation: +signes €$# (faux positif) file: [c05f88b98c] check-in: [365d3554c7] user: olr, branch: trunk, size: 4900 [annotate] [blame] [check-ins using] [diff] | |
|
2018-12-26
| ||
| 18:25 | [graphspell][js] fucking \w substitution again file: [1d91386df2] check-in: [7f03f6c55a] user: olr, branch: trunk, size: 4890 [annotate] [blame] [check-ins using] [diff] | |
| 18:10 | [graphspell][js] fucking \w substitution again file: [eb39282111] check-in: [78254a6629] user: olr, branch: trunk, size: 4874 [annotate] [blame] [check-ins using] [diff] | |
|
2018-10-11
| ||
| 15:13 | [js] Revert syntax change file: [1dc7e255ce] check-in: [9ae8f0a042] user: IllusionPerdu, branch: nodejs, size: 4864 [annotate] [blame] [check-ins using] [diff] | |
|
2018-10-10
| ||
| 09:19 | Some change to javascript to work in node file: [541689c69f] check-in: [a3687f4fd3] user: IllusionPerdu, branch: nodejs, size: 4863 [annotate] [blame] [check-ins using] [diff] | |
|
2018-09-17
| ||
| 12:06 | [graphspell][js] tokenizer: update \w replacement again file: [88bacac87d] check-in: [72f63bddd2] user: olr, branch: rg, size: 4819 [annotate] [blame] [check-ins using] [diff] | |
| 10:26 | [graphspell] tokenizer: new regex for ordinals (JS still sucks) file: [d579c92281] check-in: [a479b36582] user: olr, branch: rg, size: 4815 [annotate] [blame] [check-ins using] [diff] | |
| 09:00 | [graphspell] tokenizer: add chars to \w replacement (JS still sucks) file: [49d1f9c490] check-in: [c185b3fc04] user: olr, branch: rg, size: 4803 [annotate] [blame] [check-ins using] [diff] | |
|
2018-09-14
| ||
| 09:48 | [graphspell][js] tokenizer: fix var init file: [aac6560c8a] check-in: [52d64fd395] user: olr, branch: rg, size: 4775 [annotate] [blame] [check-ins using] [diff] | |
|
2018-09-11
| ||
| 18:55 | [graphspell][js] tokenizer: don’t use spaces as tokens, yield information token (start/end) file: [d2289444f9] check-in: [d12872816f] user: olr, branch: rg, size: 4799 [annotate] [blame] [check-ins using] [diff] | |
|
2018-08-07
| ||
| 08:09 | [graphspell][core][fx][tb][js] get rid of console.log() substitutes, assuming now default console is available from everywhere (eh, print now works as expected in JS! The JS pile of shit is decreasing a little) file: [04c83f2aa3] check-in: [8e5a2f5900] user: olr, branch: trunk, size: 4322 [annotate] [blame] [check-ins using] [diff] | |
|
2018-07-17
| ||
| 06:42 | [graphspell] tokenizer: remove hyphen in number detection (always considered as a separate sign) file: [4a5b091820] check-in: [6950f5898f] user: olr, branch: rg, size: 4387 [annotate] [blame] [check-ins using] [diff] | |
|
2018-06-28
| ||
| 08:26 | [graphspell][core] tokenizer: rename ACRONYM tokens to WORD_ACRONYM file: [2fadfb42f5] check-in: [ccbbecbd1b] user: olr, branch: rg, size: 4391 [annotate] [blame] [check-ins using] [diff] | |
| 08:00 | [graphspell] tokenizer: rename ORDINAL tokens to WORD_ORDINAL file: [a185b00a68] check-in: [20dbc28ded] user: olr, branch: rg, size: 4381 [annotate] [blame] [check-ins using] [diff] | |
| 07:53 | [graphspell][core] tokenizer: rename ELPFX tokens to WORD_ELIDED file: [8dd855b1b3] check-in: [a1b165e276] user: olr, branch: rg, size: 4376 [annotate] [blame] [check-ins using] [diff] | |
|
2018-06-18
| ||
| 20:13 | [graphspell][fix] tokenizer: new signs file: [5f94dc04ea] check-in: [303c416286] user: olr, branch: rg, size: 4370 [annotate] [blame] [check-ins using] [diff] | |
| 20:12 | [graphspell] tokenizer: new signs file: [1ab7d18bed] check-in: [da0d308818] user: olr, branch: rg, size: 4372 [annotate] [blame] [check-ins using] [diff] | |
|
2018-06-17
| ||
| 13:11 | [graphspell] tokenizer: update ordinals file: [2e21533239] check-in: [4be13a74c3] user: olr, branch: rg, size: 4280 [annotate] [blame] [check-ins using] [diff] | |
|
2018-05-18
| ||
| 13:11 | [graphspell] tokenizer: add token index and avoid punctuations aggregation file: [9bd60cca8a] check-in: [be6d99bbdc] user: olr, branch: rg, size: 4232 [annotate] [blame] [check-ins using] [diff] | |
|
2018-02-20
| ||
| 08:40 | [graphspell] spellchecker: add parseParagraph() file: [bdd895b918] check-in: [7616aa7ef9] user: olr, branch: trunk, size: 4443 [annotate] [blame] [check-ins using] [diff] | |
|
2018-02-13
| ||
| 15:44 | [core][cli][server][graphspell][fx] use spellchecker instead of ibdawg file: [c3f0ee8c90] check-in: [18db5d65f0] user: olr, branch: multid, size: 4756 [annotate] [blame] [check-ins using] [diff] | |
|
2017-12-25
| ||
| 08:52 | [build][tb] fix imports for Thunderbird file: [d6429837c4] check-in: [5822c38ee9] user: olr, branch: graphspell, size: 4740 [annotate] [blame] [check-ins using] [diff] | |
|
2017-12-24
| ||
| 18:58 | Renamed gc_core/js/tokenizer.js → graphspell-js/tokenizer.js. [build][js] move files from gc_core to graphspell file: [a34a81c6e5] check-in: [bdfc6fd5e9] user: olr, branch: graphspell, size: 4729 [annotate] [blame] [check-ins using] [diff] | |
|
2017-11-12
| ||
| 13:22 | [core][fx] tokenizer: +acronyms file: [a34a81c6e5] check-in: [fa1205c098] user: olr, branch: Lexicographe, size: 4729 [annotate] [blame] [check-ins using] [diff] | |
|
2017-11-02
| ||
| 10:58 | Ajout dans le tokenizer du ~ dans la detection des dossier linux, et distintion entre les deux types de dossier windows/linux avec le changement dans le lexicographe file: [9d996e312d] check-in: [11f1414b5b] user: IllusionPerdu, branch: Lexicographe, size: 4613 [annotate] [blame] [check-ins using] [diff] | |
|
2017-10-26
| ||
| 05:49 | [core] tokenizer: better regex for URLs and folders file: [9bb6ea03fb] check-in: [843c0244bc] user: olr, branch: trunk, size: 4601 [annotate] [blame] [check-ins using] [diff] | |
|
2017-10-25
| ||
| 18:34 | [core][bug] fix tokenizer for URL file: [de468b4358] check-in: [ee7d44a3ee] user: olr, branch: trunk, size: 4583 [annotate] [blame] [check-ins using] [diff] | |
|
2017-10-24
| ||
| 11:00 | [core] tokenization: folders file: [9622b0a610] check-in: [35c48d42a8] user: olr, branch: trunk, size: 4561 [annotate] [blame] [check-ins using] [diff] | |
|
2017-08-08
| ||
| 19:11 | [core] add inline config for JSHint assuming es6 synthax and some global file: [81088a0177] check-in: [b89dc82bc4] user: IllusionPerdu, branch: webext2_fix, size: 3799 [annotate] [blame] [check-ins using] [diff] | |
| 16:49 | [core][fr] change typeof(exports) before the require block file: [c0e5d0e035] check-in: [6c8ee6edcf] user: IllusionPerdu, branch: webext2_fix, size: 3748 [annotate] [blame] [check-ins using] [diff] | |
|
2017-08-06
| ||
| 00:44 | [core][js] ajout des ; oubliés et ajustement des ; en trop (commit erroné) file: [fcd058bf6a] check-in: [b08f2ef338] user: IllusionPerdu, branch: webext2_illusion, size: 3748 [annotate] [blame] [check-ins using] [diff] | |
|
2017-08-02
| ||
| 06:12 | [core][js] conditional requires and variable renaming to avoid overridding file: [02266194ef] check-in: [7e545790c3] user: olr, branch: webext2, size: 3749 [annotate] [blame] [check-ins using] [diff] | |
|
2017-07-29
| ||
| 08:06 | [core][js] test if <exports> exists file: [4c307bc446] check-in: [c55ae247e7] user: olr, branch: webext, size: 3044 [annotate] [blame] [check-ins using] [diff] | |
|
2017-07-27
| ||
| 10:10 | [core][js] test if variable <exports> exists file: [a6594366c3] check-in: [f9a034e6ce] user: olr, branch: trunk, size: 3695 [annotate] [blame] [check-ins using] [diff] | |
|
2017-07-13
| ||
| 07:08 | [core][js] tokenizer: yield separator one by one file: [8720e6c367] check-in: [492169bd6f] user: olr, branch: kill_innerHTML, size: 3649 [annotate] [blame] [check-ins using] [diff] | |
|
2017-07-12
| ||
| 11:03 | [fr][js] màj: lexicographe, tokenizer file: [4eb5311054] check-in: [074cb33c80] user: olr, branch: kill_innerHTML, size: 3338 [annotate] [blame] [check-ins using] [diff] | |
|
2017-04-25
| ||
| 11:51 | Added: commit 1 file: [d06151ccd7] check-in: [2fd7dc4dd5] user: olr, branch: trunk, size: 2998 [annotate] [blame] [check-ins using] | |