Grammalecte  Check-in [1ef304f49a]

Overview
Comment:[core] gc engine: text processor delete tokens replaced by empty strings or * command
Downloads: Tarball | ZIP archive | SQL archive
Timelines: family | ancestors | descendants | both | core | rg
Files: files | file ages | folders
SHA3-256: 1ef304f49ad60ab3b8c8ebfa9eb46b2ff653a0b6875a5ff36e8c76f21174cb95
User & Date: olr on 2018-06-27 17:04:24
Other Links: branch diff | manifest | tags
Context
2018-06-27
17:05
[fr] rewrite rule rewriting subjets check-in: 345078f4f4 user: olr tags: fr, rg
17:04
[core] gc engine: text processor delete tokens replaced by empty strings or * command check-in: 1ef304f49a user: olr tags: core, rg
16:02
[fr] conversion: regex rules -> graph rules check-in: 7ada852885 user: olr tags: fr, rg
Changes

Modified gc_core/py/lang_core/gc_engine.py from [8b03067d70] to [f29cb1d51a].

935
936
937
938
939
940
941



942
943
944



945
946
947
948
949
950
951
935
936
937
938
939
940
941
942
943
944



945
946
947
948
949
950
951
952
953
954







+
+
+
-
-
-
+
+
+







            else:
                # several tokens
                lTokenValue = sWhat.split("|")
                if len(lTokenValue) != (nTokenRewriteEnd - nTokenRewriteStart + 1):
                    print("Error. Text processor: number of replacements != number of tokens.")
                    return
                for i, sValue in zip(range(nTokenRewriteStart, nTokenRewriteEnd+1), lTokenValue):
                    if not sValue or sValue == "*":
                        self.lToken[i]["bToRemove"] = True
                    else:
                    if bUppercase:
                        sValue = sValue[0:1].upper() + sValue[1:]
                    self.lToken[i]["sNewValue"] = sValue
                        if bUppercase:
                            sValue = sValue[0:1].upper() + sValue[1:]
                        self.lToken[i]["sNewValue"] = sValue

    def rewrite (self, bDebug=False):
        "rewrite the sentence, modify tokens, purge the token list"
        lNewToken = []
        nMergeUntil = 0
        dTokenMerger = None
        for dToken in self.lToken: