o EJe:@sdZddlZddlZddlZddlmZmZddlmZddl m Z m Z m Z m Z mZddlmZmZmZmZmZmZddlmZgdZed Zgd Zed d ZGd ddeZGdddedZ Gddde Z!Gddde"Z#GdddZ$e$Z%Gddde&Z'GdddZ(ddZ)GdddZ*e*Z+d d!Z,Gd"d#d#Z-Gd$d%d%eZ.Gd&d'd'eZ/Gd(d)d)e e/dZ0Gd*d+d+Z1Gd,d-d-e0Z2d.d/Z3Gd0d1d1e/Z4Gd2d3d3e0e4dZ5dS)4z pygments.lexer ~~~~~~~~~~~~~~ Base lexer classes. :copyright: Copyright 2006-2023 by the Pygments team, see AUTHORS. :license: BSD, see LICENSE for details. N) apply_filtersFilter)get_filter_by_name)ErrorTextOther Whitespace _TokenType) get_bool_opt get_int_opt get_list_optmake_analysatorFuture guess_decode) regex_opt) Lexer RegexLexerExtendedRegexLexerDelegatingLexer LexerContextincludeinheritbygroupsusingthisdefaultwordsline_rez.*? ))sutf-8)szutf-32)szutf-32be)szutf-16)szutf-16becCdS)Nxr!r!h/home/seandotau/au.memorygallery.club/gallery/lib/python3.10/site-packages/pip/_vendor/pygments/lexer.py"r%c@eZdZdZddZdS) LexerMetaz This metaclass automagically converts ``analyse_text`` methods into static methods which always return float values. cCs(d|vr t|d|d<t||||S)N analyse_text)r type__new__)Zmcsnamebasesdr!r!r$r++szLexerMeta.__new__N)__name__ __module__ __qualname____doc__r+r!r!r!r$r(%s r(c@s^eZdZdZdZgZgZgZgZdZ dZ ddZ ddZ dd Z d d Zdd dZddZdS)rau Lexer for a specific language. See also :doc:`lexerdevelopment`, a high-level guide to writing lexers. Lexer classes have attributes used for choosing the most appropriate lexer based on various criteria. .. autoattribute:: name :no-value: .. autoattribute:: aliases :no-value: .. autoattribute:: filenames :no-value: .. autoattribute:: alias_filenames .. autoattribute:: mimetypes :no-value: .. autoattribute:: priority Lexers included in Pygments should have an additional attribute: .. autoattribute:: url :no-value: You can pass options to the constructor. The basic options recognized by all lexers and processed by the base `Lexer` class are: ``stripnl`` Strip leading and trailing newlines from the input (default: True). ``stripall`` Strip all leading and trailing whitespace from the input (default: False). ``ensurenl`` Make sure that the input ends with a newline (default: True). This is required for some lexers that consume input linewise. .. versionadded:: 1.3 ``tabsize`` If given and greater than 0, expand tabs in the input (default: 0). ``encoding`` If given, must be an encoding name. This encoding will be used to convert the input string to Unicode, if it is not already a Unicode string (default: ``'guess'``, which uses a simple UTF-8 / Locale / Latin1 detection. Can also be ``'chardet'`` to use the chardet library, if it is installed. ``inencoding`` Overrides the ``encoding`` if given. NrcKs||_t|dd|_t|dd|_t|dd|_t|dd|_|dd |_|d p-|j|_g|_ t |d d D]}| |q8d S)a This constructor takes arbitrary options as keyword arguments. Every subclass must first process its own options and then call the `Lexer` constructor, since it processes the basic options like `stripnl`. An example looks like this: .. sourcecode:: python def __init__(self, **options): self.compress = options.get('compress', '') Lexer.__init__(self, **options) As these options must all be specifiable as strings (due to the command line usage), there are various utility functions available to help with that, see `Utilities`_. stripnlTstripallFensurenltabsizerencodingguess inencodingfiltersr!N) optionsr r3r4r5r r6getr7r:r add_filter)selfr;filter_r!r!r$__init__s zLexer.__init__cCs$|jr d|jj|jfSd|jjS)Nzz)r; __class__r/r>r!r!r$__repr__s  zLexer.__repr__cKs*t|ts t|fi|}|j|dS)z8 Add a new stream filter to this lexer. N) isinstancerrr:append)r>r?r;r!r!r$r=s zLexer.add_filtercCr)a A static method which is called for lexer guessing. It should analyse the text and return a float in the range from ``0.0`` to ``1.0``. If it returns ``0.0``, the lexer will not be selected as the most probable one, if it returns ``1.0``, it will be selected immediately. This is used by `guess_lexer`. The `LexerMeta` metaclass automatically wraps this function so that it works like a static method (no ``self`` or ``cls`` parameter) and the return value is automatically converted to `float`. If the return value is an object that is boolean `False` it's the same as if the return values was ``0.0``. Nr!)textr!r!r$r)r&zLexer.analyse_textFc stts{jdkrt\}nwjdkrgzddlm}Wnty/}ztd|d}~wwd}tD]\}}|rKt |d |d}nq4|durd| dd}  | d pad d}|n! jd rzt d dn d rt d d d d  dd jrnjrd jdkrjjrd sd 7fdd} | } |st| j} | S)ae This method is the basic interface of a lexer. It is called by the `highlight()` function. It must process the text and return an iterable of ``(tokentype, value)`` pairs from `text`. Normally, you don't need to override this method. The default implementation processes the options recognized by all lexers (`stripnl`, `stripall` and so on), and then yields all tokens from `get_tokens_unprocessed()`, with the ``index`` dropped. If `unfiltered` is set to `True`, the filtering mechanism is bypassed even if filters are defined. r8chardetr)rGzkTo enable chardet encoding guessing, please install the chardet library from http://chardet.feedparser.org/Nreplaceir7ruz   c3s&D] \}}}||fVqdSN)get_tokens_unprocessed)_tvr>rFr!r$streamers z"Lexer.get_tokens..streamer)rDstrr7rZ pip._vendorrG ImportError _encoding_map startswithlendecodeZdetectr<rHr4stripr3r6 expandtabsr5endswithrr:) r>rFZ unfilteredrMrGeZdecodedZbomr7encrQstreamr!rPr$ get_tokenssZ              zLexer.get_tokenscCst)aS This method should process the text and return an iterable of ``(index, tokentype, value)`` tuples where ``index`` is the starting position of the token within the input text. It must be overridden by subclasses. It is recommended to implement it as a generator to maximize effectiveness. )NotImplementedErrorrPr!r!r$rLs zLexer.get_tokens_unprocessed)F)r/r0r1r2r,aliases filenamesalias_filenames mimetypespriorityurlr@rCr=r)r^rLr!r!r!r$r1s4  @r) metaclassc@s$eZdZdZefddZddZdS)ra  This lexer takes two lexer as arguments. A root lexer and a language lexer. First everything is scanned using the language lexer, afterwards all ``Other`` tokens are lexed using the root lexer. The lexers from the ``template`` lexer package use this base lexer. cKs<|di||_|di||_||_tj|fi|dSNr!) root_lexerlanguage_lexerneedlerr@)r>Z _root_lexerZ_language_lexerZ_needler;r!r!r$r@szDelegatingLexer.__init__cCsd}g}g}|j|D]$\}}}||jur(|r#|t||fg}||7}q ||||fq |r<|t||ft||j|S)N)rirLrjrErV do_insertionsrh)r>rFZbuffered insertionsZ lng_bufferirNrOr!r!r$rLs   z&DelegatingLexer.get_tokens_unprocessedN)r/r0r1r2rr@rLr!r!r!r$r s rc@eZdZdZdS)rzI Indicates that a state should include rules from another state. Nr/r0r1r2r!r!r!r$r4src@r')_inheritzC Indicates the a state should inherit from its superclass. cCr)Nrr!rBr!r!r$rC?z_inherit.__repr__N)r/r0r1r2rCr!r!r!r$rq;s rqc@s eZdZdZddZddZdS)combinedz: Indicates a state combined from multiple states. cGs t||SrK)tupler+)clsargsr!r!r$r+Js zcombined.__new__cGsdSrKr!)r>rvr!r!r$r@Mszcombined.__init__N)r/r0r1r2r+r@r!r!r!r$rsEs rsc@sFeZdZdZddZdddZdddZdd d Zd d Zd dZ dS) _PseudoMatchz: A pseudo match object constructed from a string. cCs||_||_dSrK)_text_start)r>startrFr!r!r$r@Ws z_PseudoMatch.__init__NcCs|jSrK)ryr>argr!r!r$rz[sz_PseudoMatch.startcCs|jt|jSrK)ryrVrxr{r!r!r$end^sz_PseudoMatch.endcCs|rtd|jS)Nz No such group) IndexErrorrxr{r!r!r$groupasz_PseudoMatch.groupcCs|jfSrK)rxrBr!r!r$groupsfsz_PseudoMatch.groupscCsiSrKr!rBr!r!r$ groupdictirrz_PseudoMatch.groupdictrK) r/r0r1r2r@rzr}rrrr!r!r!r$rwRs    rwcsdfdd }|S)zL Callback that yields multiple actions for each group in the match. Nc3stD]O\}}|durqt|tur)||d}|r(||d||fVq||d}|durT|r>||d|_||t||d||D]}|rS|VqLq|r^||_dSdS)N) enumerater*r rrzposrwr})lexermatchctxrnactiondataitemrvr!r$callbackqs, zbygroups..callbackrKr!)rvrr!rr$rmsrc@ro)_ThiszX Special singleton used for indicating the caller class. Used by ``using``. Nrpr!r!r!r$rsrc slidvrd}t|ttfr|d<nd|fd<tur+dfdd }|Sdfdd }|S) a Callback that processes the match with a different lexer. The keyword arguments are forwarded to the lexer, except `state` which is handled separately. `state` specifies the state that the new lexer will start in, and can be an enumerable such as ('root', 'inline', 'string') or a simple string which is assumed to be on top of the root state. Note: For that to work, `_other` must not be an `ExtendedRegexLexer`. statestackrootNc3sxr|j|jdi}n|}|}|j|fiD] \}}}||||fVq#|r:||_dSdSrg)updater;rArzrLrr}rrrrZlxsrnrNrO) gt_kwargskwargsr!r$rs  zusing..callbackc3sl|jdi}|}|j|fiD] \}}}||||fVq|r4||_dSdSrg)rr;rzrLrr}rr_otherrrr!r$rs  rK)poprDlistrtr)rrrrr!rr$rs     rc@r')rz Indicates a state or state action (e.g. #pop) to apply. For example default('#pop') is equivalent to ('', Token, '#pop') Note that state tuples may be used as well. .. versionadded:: 2.0 cCs ||_dSrK)r)r>rr!r!r$r@s zdefault.__init__N)r/r0r1r2r@r!r!r!r$rs rc@s"eZdZdZdddZddZdS) rz Indicates a list of literal words that is transformed into an optimized regex that matches any of the words. .. versionadded:: 2.0 rkcCs||_||_||_dSrK)rprefixsuffix)r>rrrr!r!r$r@s zwords.__init__cCst|j|j|jdS)Nrr)rrrrrBr!r!r$r<sz words.getN)rkrk)r/r0r1r2r@r<r!r!r!r$rs  rc@sJeZdZdZddZddZddZdd Zdd d Zd dZ ddZ d S)RegexLexerMetazw Metaclass for RegexLexer, creates the self._tokens attribute from self.tokens on the first instantiation. cCs t|tr |}t||jS)zBPreprocess the regular expression component of a token definition.)rDrr<recompiler)ruregexrflagsrr!r!r$_process_regexs zRegexLexerMeta._process_regexcCs&t|tust|sJd|f|S)z5Preprocess the token component of a token definition.z2token type must be simple type or callable, not %r)r*r callable)rutokenr!r!r$_process_tokenszRegexLexerMeta._process_tokencCst|tr/|dkr dS||vr|fS|dkr|S|dddkr)t|dd SJd|t|trbd |j}|jd 7_g}|D]}||ksPJd ||||||qD|||<|fSt|tr||D]}||vsy|d vsyJd |qi|SJd|)z=Preprocess the state transition action of a token definition.#pop#pushNz#pop:Fzunknown new state %rz_tmp_%drzcircular state ref %r)rrzunknown new state zunknown new state def %r)rDrRintrs_tmpnameextend_process_statert)ru new_state unprocessed processedZ tmp_stateitokensZistater!r!r$_process_new_states<        z!RegexLexerMeta._process_new_statec Cst|tus Jd||ddksJd|||vr ||Sg}||<|j}||D]}t|trK||ks>Jd|||||t|q-t|trQq-t|trk| |j ||}| t djd|fq-t|tuswJd|z ||d||}Wnty} z td |d||| f| d} ~ ww||d } t|d krd}n | |d ||}| || |fq-|S) z%Preprocess a single state definition.zwrong state name %rr#zinvalid state name %rzcircular state reference %rrkNzwrong rule def %rz+uncompilable regex %r in state %r of %r: %sr)r*rRflagsrDrrrrqrrrrErrrrtr Exception ValueErrorrrV) rurrrtokensrZtdefrrexerrrr!r!r$rsL        zRegexLexerMeta._process_stateNcCs<i}|j|<|p |j|}t|D] }||||q|S)z-Preprocess a dictionary of token definitions.) _all_tokensrrr)rur, tokendefsrrr!r!r$process_tokendef?s  zRegexLexerMeta.process_tokendefc Csi}i}|jD]_}|jdi}|D]Q\}}||}|dur;|||<z|t}Wn ty5Yqw|||<q||d}|durFq||||d<z|t} Wn ty^Yqw|| ||<qq|S)a Merge tokens from superclasses in MRO order, returning a single tokendef dictionary. Any state that is not defined by a subclass will be inherited automatically. States that *are* defined by subclasses will, by default, override that state in the superclass. If a subclass wishes to inherit definitions from a superclass, it can use the special value "inherit", which will cause the superclass' state definition to be included at that point in the state. rNr)__mro____dict__r<itemsindexrrr) rur inheritablectoksrrZcuritemsZ inherit_ndxZ new_inh_ndxr!r!r$ get_tokendefsGs6      zRegexLexerMeta.get_tokendefscOsRd|jvri|_d|_t|dr|jrn |d||_tj |g|Ri|S)z:Instantiate cls after preprocessing its token definitions._tokensrtoken_variantsrk) rrrhasattrrrrrr*__call__)rurvkwdsr!r!r$rxs zRegexLexerMeta.__call__rK) r/r0r1r2rrrrrrrr!r!r!r$rs# , 1rc@s$eZdZdZejZiZdddZdS)rz Base for simple stateful regular expression-based lexers. Simplifies the lexing process so that you need only provide a list of states and regular expressions. rc csd}|j}t|}||d} |D]\}}} |||} | r|dur:t|tur2||| fVn||| EdH| }| durt| trm| D]"} | dkrZt|dkrY| qI| dkrf| |dqI| | qIn+t| t rt | t|kr|dd=n|| d=n| dkr| |dnJd| ||d}n3qz'||d krd g}|d }|t d fV|d7}Wq|t||fV|d7}Wn tyYdSwq) z~ Split ``text`` into (tokentype, text) pairs. ``stack`` is the initial stack (default: ``['root']``) rrrNrrFwrong state def: %rrIr)rrr*r rr}rDrtrVrrErabsrrr~) r>rFrrrZ statestack statetokensrexmatchrrmrr!r!r$rLs`           #    z!RegexLexer.get_tokens_unprocessedNr) r/r0r1r2r MULTILINErrrLr!r!r!r$rs rc@s"eZdZdZdddZddZdS)rz9 A helper object that holds lexer position data. NcCs*||_||_|p t||_|pdg|_dS)Nr)rFrrVr}r)r>rFrrr}r!r!r$r@szLexerContext.__init__cCsd|j|j|jfS)NzLexerContext(%r, %r, %r))rFrrrBr!r!r$rCs zLexerContext.__repr__NN)r/r0r1r2r@rCr!r!r!r$rs  rc@seZdZdZdddZdS)rzE A RegexLexer that uses a context object to store its state. Nc cs|j}|st|d}|d}n |}||jd}|j} |D]\}}}|||j|j} | r|durYt|turG|j|| fV| |_n||| |EdH|sY||jd}|durt |t r|D]'} | dkrwt |jdkrv|j qd| dkr|j |jdqd|j | qdn0t |trt|t |jkr|jdd=n|j|d=n|dkr|j |jdnJd |||jd}nHqz;|j|jkrWdS||jd krdg|_|d}|jtd fV|jd7_Wq|jt||jfV|jd7_Wn ty YdSwq) z Split ``text`` into (tokentype, text) pairs. If ``context`` is given, use this lexer context instead. rrrrNrrFrrI)rrrrFrr}r*r rrDrtrVrrErrrrr~) r>rFcontextrrrrrrrrr!r!r$rLsn        # z)ExtendedRegexLexer.get_tokens_unprocessedr)r/r0r1r2rLr!r!r!r$rsrc cst|}zt|\}}Wnty|EdHYdSwd}d}|D]{\}}}|dur.|}d} |r|t||kr|| ||} | rP||| fV|t| 7}|D]\} } } || | fV|t| 7}qR||} zt|\}}Wn ty{d}Yn w|r|t||ks:| t|kr|||| dfV|t|| 7}q#|r|pd}|D]\}}}|||fV|t|7}qzt|\}}Wn tyd}YdSw|sdSdS)ag Helper for lexers which must combine the results of several sublexers. ``insertions`` is a list of ``(index, itokens)`` pairs. Each ``itokens`` iterable should be inserted at position ``index`` into the token stream given by the ``tokens`` argument. The result is a combined token stream. TODO: clean up the code here. NTrF)iternext StopIterationrV)rmrrrZrealposZinsleftrnrNrOZoldiZtmpvalZit_indexZit_tokenZit_valuepr!r!r$rl?s\          rlc@r')ProfilingRegexLexerMetaz>Metaclass for ProfilingRegexLexer, collects regex timing info.csLt|trt|j|j|jdn|t|tjffdd }|S)Nrcs`jdfddg}t}|||}t}|dd7<|d||7<|S)Nrrr r) _prof_data setdefaulttimer)rFrendposinfot0rest1rucompiledrrr!r$ match_funcsz:ProfilingRegexLexerMeta._process_regex..match_func) rDrrrrrrsysmaxsize)rurrrrr!rr$rs   z&ProfilingRegexLexerMeta._process_regexN)r/r0r1r2rr!r!r!r$rs rc@s"eZdZdZgZdZdddZdS)ProfilingRegexLexerzFDrop-in replacement for RegexLexer that does profiling of its regexes.rc#sjjit||EdHjj}tdd|Dfdddd}tdd|D}t t djj t ||ft d t d d t d |D]}t d |qSt d dS)NcssP|]#\\}}\}}|t|ddddd|d|d||fVqdS)zu'z\\\NAi)reprrXrH).0rrnrNr!r!r$ s  z=ProfilingRegexLexer.get_tokens_unprocessed..cs |jSrK)_prof_sort_indexr"rBr!r$r%s z.T)keyreversecss|]}|dVqdS)Nr!)rr#r!r!r$rsz2Profiling result for %s lexing %d chars in %.3f mszn==============================================================================================================z$%-20s %-64s ncalls tottime percall)rrzn--------------------------------------------------------------------------------------------------------------z%-20s %-65s %5d %8.4f %8.4f) rArrErrLrsortedrsumprintr/rV)r>rFrZrawdatarZ sum_totalr.r!rBr$rLs*    z*ProfilingRegexLexer.get_tokens_unprocessedNr)r/r0r1r2rrrLr!r!r!r$rs r)6r2rrrZpip._vendor.pygments.filterrrZpip._vendor.pygments.filtersrpip._vendor.pygments.tokenrrrrr pip._vendor.pygments.utilr r r r rrZpip._vendor.pygments.regexoptr__all__rrrT staticmethodZ_default_analyser*r(rrrRrrqrrtrsrwrrrrrrrrrrrlrrr!r!r!r$sH       ]' 2 )aH@