OwlCyberSecurity - MANAGER
Edit File: horizontal_shard.cpython-312.pyc
� ̓�g3A � � � d Z ddlmZ ddlmZ ddlmZ ddlmZ ddlmZ ddlmZ ddlm Z dd lm Z dd lm Z dd lm Z dd lm Z d dlmZ d dlmZ d dlmZ d dlmZ d dlmZ d dlmZ d dlmZ d dlmZ d dlm Z d dl m Z d dl m Z d dl m Z d dl!m"Z" d dl!m#Z# e r`d d l$m%Z% d d l$m&Z& d d l$m'Z' d d l(m)Z) d d l(m*Z* d d!lm+Z+ d d"lm,Z, d d#l-m.Z. d d$l/m0Z0 d d%l m1Z1 d d&l m2Z2 d d'l m3Z3 d d(l4m5Z5 d d)l6m7Z7 d d*l8m9Z9 d d+l:m;Z; d,d-gZ< e d.e�/� Z=e>Z? G d0� d1e"� Z@ G d2� d3e"� ZA G d4� d-e e= � ZB G d5� d,e � ZC G d6� d7e� ZD d:d8�ZEy9);a� Horizontal sharding support. Defines a rudimental 'horizontal sharding' system which allows a Session to distribute queries and persistence operations across multiple databases. For a usage example, see the :ref:`examples_sharding` example included in the source distribution. .. deepalchemy:: The horizontal sharding extension is an advanced feature, involving a complex statement -> database interaction as well as use of semi-public APIs for non-trivial cases. Simpler approaches to refering to multiple database "shards", most commonly using a distinct :class:`_orm.Session` per "shard", should always be considered first before using this more complex and less-production-tested system. � )� annotations)�Any)�Callable)�Dict)�Iterable)�Optional)�Tuple)�Type)� TYPE_CHECKING)�TypeVar)�Union� )�event)�exc)�inspect)�util)� PassiveFlag)�OrmExecuteOptionsParameter)� ORMOption)�Mapper)�Query)�_BindArguments)�_PKIdentityArgument)�Session)�Protocol)�Self)� Connection)�Engine)� OptionEngine)�IteratorResult)�Result)�LoaderCallableStatus)�_O)�BulkUDCompileState)� QueryContext)�_EntityBindKey)� _SessionBind)�ORMExecuteState)� InstanceState)� Executable)�_TP)� ClauseElement�ShardedSession� ShardedQuery�_T)�boundc �$ � e Zd Z dd�Zy)� ShardChooserc � � y �N� )�self�mapper�instance�clauses �P/opt/hc_python/lib64/python3.12/site-packages/sqlalchemy/ext/horizontal_shard.py�__call__zShardChooser.__call__Q s � � � N)r7 �Optional[Mapper[_T]]r8 r r9 �Optional[ClauseElement]�returnr ��__name__� __module__� __qualname__r; r5 r< r: r2 r2 P s- � ��$�� �� (� � � r< r2 c �0 � e Zd Z dd�Zy)�IdentityChooserc � � y r4 r5 )r6 r7 � primary_key�lazy_loaded_from�execution_options�bind_arguments�kws r: r; zIdentityChooser.__call__Z s � � r< N�r7 z Mapper[_T]rG r rH � Optional[InstanceState[Any]]rI r rJ r rK r r? r r@ r5 r< r: rE rE Y sK � � �� � )� � 7� � 6� � '� � � � � r< rE c �, � � e Zd ZdZd� fd� Zdd�Z� xZS )r. a Query class used with :class:`.ShardedSession`. .. legacy:: The :class:`.ShardedQuery` is a subclass of the legacy :class:`.Query` class. The :class:`.ShardedSession` now supports 2.0 style execution via the :meth:`.ShardedSession.execute` method. c �� �� t �| � |i |�� t | j t � sJ �| j j | _ | j j | _ d | _ y r4 )�super�__init__� isinstance�sessionr- �identity_chooser�execute_chooser� _shard_id)r6 �args�kwargs� __class__s �r: rQ zShardedQuery.__init__o sT �� � ���$�)�&�)��$�,�,��7�7�7� $� � � =� =�� � #�|�|� ;� ;�� � ��r< c �& � | j |�� S )a� Return a new query, limited to a single shard ID. All subsequent operations with the returned query will be against the single shard regardless of other state. The shard_id can be passed for a 2.0 style execution to the bind_arguments dictionary of :meth:`.Session.execute`:: results = session.execute(stmt, bind_arguments={"shard_id": "my_shard"}) )� _sa_shard_id)rI )r6 �shard_ids r: � set_shardzShardedQuery.set_shardw s � � �%�%�8�%�<�<r< )rW r rX r r? �None)r\ �ShardIdentifierr? r )rA rB rC �__doc__rQ r] � __classcell__�rY s @r: r. r. f s �� �� � =r< c �P � � e Zd ZU ded< ded< ded< dddefddd� d� fd �Zdej dej df d� fd � Z dd �Z d dd �Z ddddd � dd�Z dd�Z� xZS )r- r2 � shard_chooserrE rT z*Callable[[ORMExecuteState], Iterable[Any]]rU N)� id_chooser� query_chooserc �( �� � � � t �� � dd|i|�� t j � dt d�� |� _ |r5|� t j dd� d� � fd� } | � _ n |r|� _ nt j d� �|r=|� t j d d � |rt j d � � d� fd � } |�| }|�t j d� �|� _ i � _ |� |D ] } � j | || � � y y )a� Construct a ShardedSession. :param shard_chooser: A callable which, passed a Mapper, a mapped instance, and possibly a SQL clause, returns a shard ID. This id may be based off of the attributes present within the object, or on some round-robin scheme. If the scheme is based on a selection, it should set whatever state on the instance to mark it in the future as participating in that shard. :param identity_chooser: A callable, passed a Mapper and primary key argument, which should return a list of shard ids where this primary key might reside. .. versionchanged:: 2.0 The ``identity_chooser`` parameter supersedes the ``id_chooser`` parameter. :param execute_chooser: For a given :class:`.ORMExecuteState`, returns the list of shard_ids where the query should be issued. Results from all shards returned will be combined together into a single listing. .. versionchanged:: 1.4 The ``execute_chooser`` parameter supersedes the ``query_chooser`` parameter. :param shards: A dictionary of string shard names to :class:`~sqlalchemy.engine.Engine` objects. � query_cls�do_orm_executeT)�retvalzLThe ``id_chooser`` parameter is deprecated; please use ``identity_chooser``.z2.0c �^ �� �j | � }|r|j |� } �||� S r4 )�query�_set_lazyload_from) r7 rG rH rI rJ rK �q� _id_chooserr6 s ��r: �_legacy_identity_chooserz9ShardedSession.__init__.<locals>._legacy_identity_chooser� s4 �� � �J�J�v�&��#��,�,�-=�>�A�"�1�k�2�2r< z*identity_chooser or id_chooser is requiredzNThe ``query_chooser`` parameter is deprecated; please use ``execute_chooser``.z1.4z>Can't pass query_chooser and execute_chooser at the same time.c �( �� �| j � S r4 )� statement)� orm_context�_query_choosers �r: �_default_execute_chooserz9ShardedSession.__init__.<locals>._default_execute_chooser� s �� � &�k�&;�&;�<�<r< Nz,execute_chooser or query_chooser is requiredr5 rL )rs r( r? z Iterable[Any])rP rQ r �listen�execute_and_instancesrd r �warn_deprecatedrT r � ArgumentErrorrU �_ShardedSession__shards� bind_shard)r6 rd rT rU �shardsrh re rf rX rp ru �kro rt rY s ` @@�r: rQ zShardedSession.__init__� s �� �X ���7�9�7��7� � � � �"�$9�$� � +��� �$�K� � � �3�� � 3�"� 3� 0� 3� #?� 3� $>� 3� !/� 3� � 3� � 3� %=�D� !� �$4�D� !��#�#�<�� � � *�N� � � �2�� � ��'�'�(�� � =� ,� =� � =� �&�":�� � "��#�#�>�� � /�� �=?�� � � �������6�!�9�-� � r< c � �� |�t � | � ||fd|i|��} | S | j |||||r t |� ni �� D ] } t � | � ||f| |d�|��} | ��| c S y)a_ override the default :meth:`.Session._identity_lookup` method so that we search for a given non-token primary key identity across all possible identity tokens (e.g. shard ids). .. versionchanged:: 1.4 Moved :meth:`.Session._identity_lookup` from the :class:`_query.Query` object to the :class:`.Session`. N�identity_token)rH rI rJ )r rH )rP �_identity_lookuprT �dict) r6 r7 �primary_key_identityr �passiverH rI rJ rK �objr\ �obj2rY s �r: r� z ShardedSession._identity_lookup� s� �� �( � %��'�*��$�� .�� � �C� �J� � 1� 1��$�!1�"3�7E�t�N� 3�2� 2� �� �w�/��(�� $,�%5� � � �� �#� �K� �" r< c �� � |�Dt |� }|j r|j d }|�J �|S |j r |j S t |t � sJ � | j ||fi |��}|�|_ |S )Nr )r �keyr rR r rd )r6 r7 r8 rK �state�tokenr\ s r: �_choose_shard_and_assignz'ShardedSession._choose_shard_and_assign, s� � � � ��H�%�E��y�y� � � �!� �� �(�(�(� � ��%�%� �+�+�+��&�&�)�)�)�%�4�%�%�f�h�=�"�=�� � �#+�E� ��r< c �, � |�| j ||� }| j � r'| j � }|�J �|j ||�� S | j |||�� }t |t � r |j di |��S t |t � sJ �|S )zaProvide a :class:`_engine.Connection` to use in the unit of work flush process. )r\ )r7 r\ r8 r5 ) r� �in_transaction�get_transaction� connection�get_bindrR r �connectr )r6 r7 r8 r\ rK �trans�binds r: �connection_callablez"ShardedSession.connection_callableA s� � � � ��4�4�V�X�F�H� � � � ��(�(�*�E��$� $�$��#�#�F�X�#�>� >��=�=� ��8� !� �D� �$��'�#�t�|�|�)�b�)�)�!�$� �3�3�3�� r< )r\ r8 r9 c �T � |�| j |||�� }|�J �| j | S )N)r8 r9 )r� rz )r6 r7 r\ r8 r9 rK s r: r� zShardedSession.get_bind_ sC � � � ��4�4���&� 5� �H� �'� '�'��}�}�X�&�&r< c �" � || j |< y r4 )rz )r6 r\ r� s r: r{ zShardedSession.bind_shardo s � � #'�� � �h� r< )rd r2 rT zOptional[IdentityChooser]rU z4Optional[Callable[[ORMExecuteState], Iterable[Any]]]r| zOptional[Dict[str, Any]]rh zType[Query[_T]]re z<Optional[Callable[[Query[_T], Iterable[_T]], Iterable[Any]]]rf z/Optional[Callable[[Executable], Iterable[Any]]]rX r r? r^ )r7 z Mapper[_O]r� zUnion[Any, Tuple[Any, ...]]r � Optional[Any]r� r rH rM rI r rJ zOptional[_BindArguments]rK r r? z)Union[Optional[_O], LoaderCallableStatus])r7 � Optional[_EntityBindKey[_O]]r8 r rK r r? r )NNN) r7 r= r8 r� r\ �Optional[ShardIdentifier]rK r r? r r4 ) r7 r� r\ r� r8 r� r9 r> rK r r? r' )r\ r_ r� zUnion[Engine, OptionEngine]r? r^ )rA rB rC �__annotations__r. rQ r � PASSIVE_OFFr � EMPTY_DICTr� r� r� r� r{ ra rb s @r: r- r- � s� �� � � �%�%�?�?� 7;� �+/�%1�n.� �IM�n.�#�n.� 4�n.� � n.� )�n.� #�n.� �n.� G�n.� � n.� � n.�h )-� *� 6� 6�9=�8<���37�/��/� :�/� &� /� � /� 7� /� 6�/� 1�/� �/� 3�/�b�,�� �� � � � �. (,�"&�.2� �$� � � � ,� � � � � �@ 04�'� /3�"&�*.� '�,�'� ,� '� � '� (� '� �'� �'� '�'�'�/J�'� �'r< c �& � e Zd ZdZdZ d dd�Zy)� set_shard_ida a loader option for statements to apply a specific shard id to the primary query as well as for additional relationship and column loaders. The :class:`_horizontal.set_shard_id` option may be applied using the :meth:`_sql.Executable.options` method of any executable statement:: stmt = ( select(MyObject) .where(MyObject.name == "some name") .options(set_shard_id("shard1")) ) Above, the statement when invoked will limit to the "shard1" shard identifier for the primary query as well as for all relationship and column loading strategies, including eager loaders such as :func:`_orm.selectinload`, deferred column loaders like :func:`_orm.defer`, and the lazy relationship loader :func:`_orm.lazyload`. In this way, the :class:`_horizontal.set_shard_id` option has much wider scope than using the "shard_id" argument within the :paramref:`_orm.Session.execute.bind_arguments` dictionary. .. versionadded:: 2.0.0 �r\ �propagate_to_loadersc � � || _ || _ y)aH Construct a :class:`_horizontal.set_shard_id` option. :param shard_id: shard identifier :param propagate_to_loaders: if left at its default of ``True``, the shard option will take place for lazy loaders such as :func:`_orm.lazyload` and :func:`_orm.defer`; if False, the option will not be propagated to loaded objects. Note that :func:`_orm.defer` always limits to the shard_id of the parent row in any case, so the parameter only has a net effect on the behavior of the :func:`_orm.lazyload` strategy. Nr� )r6 r\ r� s r: rQ zset_shard_id.__init__� s � � !�� �$8��!r< N)T)r\ r_ r� �bool)rA rB rC r` � __slots__rQ r5 r< r: r� r� u s* � ��8 5�I� GK�9�'�9�?C�9r< r� c � � � � j r � j }n'� j s � j r � j }nd }� j }t |t � sJ � d� fd� }� j D ] }t |t � s�|j } nZ |r|j � |j }n>d� j v r� j d }n d� j v r� j d }nd }|� ||� S g }|j � � D ] } ||� }|j |� � |d j |dd � S )Nr\ c � �� t �j � }| |d< �j | �� �j |�� S )Nr\ )r )rJ )r� rJ �update_execution_options�invoke_statement)r\ rJ rs s �r: �iter_for_shardz-execute_and_instances.<locals>.iter_for_shard� sD �� � �k� 8� 8�9��%-��z�"��,�,�H�,�E��+�+�>�+�J�Jr< r[ r � )r\ r_ r? �&Union[Result[_T], IteratorResult[_TP]])� is_select� load_options� is_update� is_delete�update_delete_optionsrS rR r- �_non_compile_orm_optionsr� r\ �_identity_tokenrI rJ rU �append�merge)rs �active_optionsrS r� �orm_optr\ �partial�result_s ` r: rw rw � sU �� � � � �$�1�1�� � � �+�"7�"7�$�:�:�� ���!�!�G� �g�~� .�.� .�K�!�K� /�K� �7�7�� �g�|� ,� �'�'�H� � 8� �n� <� <� H�%�5�5�H� �{� <� <� <�"�4�4�^�D�H� �;�5�5� 5�"�1�1�*�=�H��H��� �h�'�'��� �/�/� �<�H�$�X�.�G� �N�N�7� #� =� �w�q�z� � �����-�-r< N)rs r( r? r� )Fr` � __future__r �typingr r r r r r r r r r � r r r r �ormr � orm._typingr �orm.interfacesr � orm.mapperr � orm.queryr � orm.sessionr r r � util.typingr r � engine.baser r r � engine.resultr r! r" r# �orm.bulk_persistencer$ � orm.contextr% r& r' r( � orm.stater) �sqlr* � sql._typingr+ � sql.elementsr, �__all__r/ �strr_ r2 rE r. r- r� rw r5 r< r: �<module>r� s ���$ #� � � � � � � � � � � � � � � � 4� &� � � (� -� !� "� ��(�$�*�.�&�*� �9�*�,�*�-�)� �!�,� �^� ,�� �T�� �� ���8� � �h� � =�5��9� =�@l'�W� l'�^/9�9� /9�d7.� �7.�+�7.r<