ruạṛ
a �u:j\$ � @ s* z d dl ZW n ey& d dlZY n0 d dlZd dlZd dlmZ d dlmZ d dl m Z d dl m Z d dl mZmZmZmZmZ d dlmZ dd lmZmZmZmZmZmZ e ed �e e d �d d ��Z e ed �e e d �dd��Z e ed �e ee d �dd��Z e ed �e e d �dd��Z!e e d �dd�Z"e ed �e e d �dd��Z#e ed �e e d �dd��Z$e ed �e e d �dd��Z%e ed �e e d �d d ��Z&e ed �e e d �d d ��Z'e e d �d d!�Z(e ed �e e d �d"d#��Z)e ed �e e d �d$d%��Z*e ed �e e d �d&d'��Z+e ed �e e d �d(d)��Z,e ed �e e d �d*d+��Z-e e.e�d �e e d,�d-d.��Z/dNe0e1ee d0�d1d2�Z2e d3d �e e d4�d5d6��Z3e0eee e0f d7�d8d9�Z4e e d:�d;d<�Z5dOe e e d>�d?d@�Z6e ee dA�dBdC�Z7e e e8dD�dEdF�Z9e e e dD�dGdH�Z:dIej;dJfe e1e ddK�dLdM�Z<dS )P� N)�IncrementalDecoder)�aliases)� lru_cache)�findall)�List�Optional�Set�Tuple�Union)�MultibyteIncrementalDecoder� )�ENCODING_MARKS�IANA_SUPPORTED_SIMILAR� RE_POSSIBLE_ENCODING_INDICATION�UNICODE_RANGES_COMBINED� UNICODE_SECONDARY_RANGE_KEYWORD�UTF8_MAXIMAL_ALLOCATION)�maxsize)� character�returnc C sT zt �| �}W n ty" Y dS 0 d|v pRd|v pRd|v pRd|v pRd|v pRd|v S )NFz WITH GRAVEz WITH ACUTEz WITH CEDILLAzWITH DIAERESISzWITH CIRCUMFLEXz WITH TILDE�� unicodedata�name� ValueError�r � description� r �K/opt/bart/bart_venv/lib/python3.9/site-packages/charset_normalizer/utils.py�is_accentuated s �����r c C s. t �| �}|s| S |�d�}tt|d d��S )N� r � )r � decomposition�split�chr�int)r Z decomposed�codesr r r � remove_accent* s r& c C s. t | �}t�� D ]\}}||v r| S qdS )zK Retrieve the Unicode range official name from a single character. N)�ordr �items)r Z character_ord� range_nameZ ord_ranger r r � unicode_range5 s r* c C s, zt �| �}W n ty" Y dS 0 d|v S )NFZLATINr r r r r �is_latinC s r+ c C s( z| � d� W n ty" Y dS 0 dS )N�asciiFT)�encode�UnicodeEncodeError�r r r r �is_asciiL s r0 c C s2 t �| �}d|v rdS t| �}|d u r*dS d|v S )N�PTF� Punctuation�r �categoryr* �r �character_category�character_ranger r r �is_punctuationT s r8 c C s: t �| �}d|v sd|v r dS t| �}|d u r2dS d|v S )N�S�NTFZFormsr3 r5 r r r � is_symbolc s r; c C s t | �}|d u rdS d|v S )NFZ Emoticons)r* )r r7 r r r � is_emoticonr s r< c C s& | � � s| dv rdS t�| �}d|v S )N> �+�;�<u ｜�,�>T�Z)�isspacer r4 �r r6 r r r � is_separator| s rE c C s | � � | �� kS �N)�islower�isupperr/ r r r �is_case_variable� s rI c C s t �| �}|dkS )N�Co)r r4 rD r r r �is_private_use_only� s rK c C s, zt �| �}W n ty" Y dS 0 d|v S )NFZCJKr �r Zcharacter_namer r r �is_cjk� s rM c C s, zt �| �}W n ty" Y dS 0 d|v S )NFZHIRAGANAr rL r r r � is_hiragana� s rN c C s, zt �| �}W n ty" Y dS 0 d|v S )NFZKATAKANAr rL r r r � is_katakana� s rO c C s, zt �| �}W n ty" Y dS 0 d|v S )NFZHANGULr rL r r r � is_hangul� s rP c C s, zt �| �}W n ty" Y dS 0 d|v S )NFZTHAIr rL r r r �is_thai� s rQ )r) r c s t � fdd�tD ��S )Nc 3 s | ]}|� v V qd S rF r )�.0�keyword�r) r r � <genexpr>� � z-is_unicode_range_secondary.<locals>.<genexpr>)�anyr rT r rT r �is_unicode_range_secondary� s rX � )�sequence� search_zoner c C s� t | t�st�t| �}tt| dt||�� jddd��}t|�dkrHdS |D ]N}|�� � dd�}t � � D ]0\}}||kr�| S ||krh| S qhqLdS )zW Extract using ASCII-only decoder any specified encoding in the first n-bytes. Nr, �ignore��errorsr �-�_) � isinstance�bytes� TypeError�lenr r �min�decode�lower�replacer r( )rZ r[ Zseq_len�resultsZspecified_encoding�encoding_alias� encoding_ianar r r �any_specified_encoding� s" � rl � )r r c C s | dv p t t�d�| ��jt�S )zQ Verify is a specific encoding is a multi byte one based on it IANA name > �utf_7Z utf_8_sig�utf_8� utf_16_le�utf_16�utf_32� utf_16_be� utf_32_be� utf_32_le� encodings.{})� issubclass� importlib� import_module�formatr r )r r r r �is_multi_byte_encoding� s ��r{ )rZ r c C sJ t D ]@}t | }t|t�r |g}|D ] }| �|�r$||f S q$qdS )z9 Identify and extract SIG/BOM in given sequence. )NrV )r ra rb � startswith)rZ � iana_encodingZmarks�markr r r �identify_sig_or_bom� s r )r} r c C s | dvS )N> rr rq r )r} r r r �should_strip_sig_or_bom s r� T)�cp_name�strictr c C sL | � � �dd�} t�� D ] \}}| ||fv r| S q|rHtd�| ���| S )Nr_ r` z Unable to retrieve IANA for '{}')rg rh r r( r rz )r� r� rj rk r r r � iana_name s r� )�decoded_sequencer c C s4 t � }| D ] }t|�}|d u r q |�|� q t|�S rF )�setr* �add�list)r� �rangesr r7 r r r � range_scan s r� )� iana_name_a� iana_name_br c C s� t | �st |�rdS t�d�| ��j}t�d�|��j}|dd�}|dd�}d}td�D ]*}t|g�}|�|�|�|�krX|d7 }qX|d S ) Ng rv r\ r] r � r � )r{ rx ry rz r �rangerb rf ) r� r� Z decoder_aZ decoder_bZid_aZid_bZcharacter_match_count�iZ to_be_decodedr r r � cp_similarity+ s r� c C s | t v o|t | v S )z� Determine if two code page are at least 80% similar. IANA_SUPPORTED_SIMILAR dict was generated using the function cp_similarity. )r )r� r� r r r � is_cp_similar@ s �r� Zcharset_normalizerz)%(asctime)s | %(levelname)s | %(message)s)r �level� format_stringr c C s: t �| �}|�|� t �� }|�t �|�� |�|� d S rF )�logging� getLogger�setLevel� StreamHandler� setFormatter� Formatter� addHandler)r r� r� �logger�handlerr r r �set_logging_handlerK s r� )rY )T)=Z unicodedata2r � ImportErrorrx r� �codecsr Zencodings.aliasesr � functoolsr �rer �typingr r r r r Z_multibytecodecr Zconstantr r r r r r �str�boolr r& r* r+ r0 r8 r; r< rE rI rK rM rN rO rP rQ rd rX rb r$ rl r{ r r� r� r� �floatr� r� �INFOr� r r r r �<module> s| ��
cải xoăn