¯wgNRdZddlZddlZddlZddlmZmZmZmZm Z m Z m Z m Z m Z mZddlmZddlZddlmZddlmcmZddlmZgdZe dZe d Ze d Ze d ZeZeZeZeege eeeffZeeeegefZ eeeegefZ!eZ"eege"fZ#ee"gefZ$e ed fZ%eege ee eefeffZ&d e de!fdZ'dddddde edede de e(de e#de e$de e&ddfdZ)ede*ddddde edede de e(de e#de e$ddfdZ+ddddde edede de e(de e#de e$ddfdZ, dadede eege-fde eeeffd Z.d!eed"edefd#Z/ dadede eege-fdeefd$Z0 dadede eege-fdeefd%Z1 dadede eege-fdefd&Z2dd'd ed efded(ede eege-fdef d)Z3dd'd ed efded(ede eege-fdef d*Z4e e ee efZ5e e ee ee efZ6ejnd+k\ree ee e ed fejpfZ9nee ee e ed ffZ9eeeefgefZ:eeeeefgefZ;eegefZe d,e5eefde>e:eeeffd-Z?e d,e6eeefde>e;eeeeffd.Z?e d,e ede>ee=efd0Z?e d,eege-fde>e=efd1Z?d,ee9eege-ffde>e=efd2Z?e dad,e ed eZB dad=eege-fdede eege-fde-fd?ZCe dad@e ed=e.wrapped^sXd^.v..) functoolswrapsr)r3r<s` r; _reverse_argsr@]s:__T/s/c/c// Nr=)serialized_type_nameto_dumpable_contextfrom_dumpable_contextflatten_with_keys_fncls flatten_fn unflatten_fnrArBrCrDcz| tdt||||||ddlm}|j||||||y)aRegister a container-like type as pytree node. Args: cls (type): A Python type to treat as an internal pytree node. flatten_fn (callable): A function to be used during flattening, taking an instance of ``cls`` and returning a pair, with (1) an iterable for the children to be flattened recursively, and (2) some hashable auxiliary data to be stored in the treespec and to be passed to the ``unflatten_fn``. unflatten_fn (callable): A function taking two arguments: the auxiliary data that was returned by ``flatten_fn`` and stored in the treespec, and the unflattened children. The function should return an instance of ``cls``. serialized_type_name (str, optional): A keyword argument used to specify the fully qualified name used when serializing the tree spec. to_dumpable_context (callable, optional): An optional keyword argument to custom specify how to convert the context of the pytree to a custom json dumpable representation. This is used for json serialization, which is being used in :mod:`torch.export` right now. from_dumpable_context (callable, optional): An optional keyword argument to custom specify how to convert the custom json dumpable representation of the context back to the original context. This is used for json deserialization, which is being used in :mod:`torch.export` right now. Example:: >>> # xdoctest: +SKIP >>> # Registry a Python type with lambda functions >>> register_pytree_node( ... set, ... lambda s: (sorted(s), None, None), ... lambda children, _: set(children), ... ) N-KeyPaths are not yet supported in cxx_pytree.rArBrC)_pytree)NotImplementedError_private_register_pytree_noderL)rErFrGrArBrCrDpythons r;rres[R'!"QRR! 1/3 $ (( 1/3 )r=z`torch.utils._cxx_pytree._register_pytree_node` is deprecated. Please use `torch.utils._cxx_pytree.register_pytree_node` instead.)categoryrJc&t||||||y)aRegister a container-like type as pytree node for the C++ pytree only. The ``namespace`` argument is used to avoid collisions that occur when different libraries register the same Python type with different behaviors. It is recommended to add a unique prefix to the namespace to avoid conflicts with other libraries. Namespaces can also be used to specify the same class in different namespaces for different use cases. .. warning:: For safety reasons, a ``namespace`` must be specified while registering a custom type. It is used to isolate the behavior of flattening and unflattening a pytree node type. This is to prevent accidental collisions between different libraries that may register the same type. Args: cls (type): A Python type to treat as an internal pytree node. flatten_fn (callable): A function to be used during flattening, taking an instance of ``cls`` and returning a pair, with (1) an iterable for the children to be flattened recursively, and (2) some hashable auxiliary data to be stored in the treespec and to be passed to the ``unflatten_fn``. unflatten_fn (callable): A function taking two arguments: the auxiliary data that was returned by ``flatten_fn`` and stored in the treespec, and the unflattened children. The function should return an instance of ``cls``. serialized_type_name (str, optional): A keyword argument used to specify the fully qualified name used when serializing the tree spec. to_dumpable_context (callable, optional): An optional keyword argument to custom specify how to convert the context of the pytree to a custom json dumpable representation. This is used for json serialization, which is being used in :mod:`torch.export` right now. from_dumpable_context (callable, optional): An optional keyword argument to custom specify how to convert the custom json dumpable representation of the context back to the original context. This is used for json deserialization, which is being used in :mod:`torch.export` right now. rJN)rNrErFrGrArBrCs r;_register_pytree_noderTs\" 1/3 r=cttj|s#tj||t|dyy)zThis is an internal function that is used to register a pytree node type for the C++ pytree only. End-users should use :func:`register_pytree_node` instead. torch) namespaceN)optreeis_structseq_classrr@rSs r;rNrNs6  $ $S )##   , '   *r=treeis_leafc4tj||ddS)aFlatten a pytree. See also :func:`tree_unflatten`. The flattening order (i.e., the order of elements in the output list) is deterministic, corresponding to a left-to-right depth-first tree traversal. >>> tree = {'b': (2, [3, 4]), 'a': 1, 'c': None, 'd': 5} >>> tree_flatten(tree) ([1, 2, 3, 4, None, 5], PyTreeSpec({'a': *, 'b': (*, [*, *]), 'c': *, 'd': *}, NoneIsLeaf)) >>> tree_flatten(1) ([1], PyTreeSpec(*, NoneIsLeaf)) >>> tree_flatten(None) ([None], PyTreeSpec(*, NoneIsLeaf)) For unordered dictionaries, :class:`dict` and :class:`collections.defaultdict`, the order is dependent on the **sorted** keys in the dictionary. Please use :class:`collections.OrderedDict` if you want to keep the keys in the insertion order. >>> from collections import OrderedDict >>> tree = OrderedDict([('b', (2, [3, 4])), ('a', 1), ('c', None), ('d', 5)]) >>> tree_flatten(tree) ([2, 3, 4, 1, None, 5], PyTreeSpec(OrderedDict({'b': (*, [*, *]), 'a': *, 'c': *, 'd': *}), NoneIsLeaf)) Args: tree (pytree): A pytree to flatten. is_leaf (callable, optional): An extra leaf predicate function that will be called at each flattening step. The function should have a single argument with signature ``is_leaf(node) -> bool``. If it returns :data:`True`, the whole subtree being treated as a leaf. Otherwise, the default pytree registry will be used to determine a node is a leaf or not. If the function is not specified, the default pytree registry will be used. Returns: A pair ``(leaves, treespec)`` where the first element is a list of leaf values and the second element is a treespec representing the structure of the pytree. TrVr[ none_is_leafrW)rXrrZr[s r;rrs$P      r=leavestreespecc~t|tstdt|dt j ||S)adReconstruct a pytree from the treespec and the leaves. The inverse of :func:`tree_flatten`. >>> tree = {'b': (2, [3, 4]), 'a': 1, 'c': None, 'd': 5} >>> leaves, treespec = tree_flatten(tree) >>> tree == tree_unflatten(leaves, treespec) True Args: leaves (iterable): The list of leaves to use for reconstruction. The list must match the number of leaves of the treespec. treespec (TreeSpec): The treespec to reconstruct. Returns: The reconstructed pytree, containing the ``leaves`` placed in the structure described by ``treespec``. z^tree_unflatten(values, spec): Expected `spec` to be instance of TreeSpec but got item of type .) isinstancer TypeErrortyperXr)r`ras r;rr&sF& h )--1(^,>> tree = {'b': (2, [3, 4]), 'a': 1, 'c': None, 'd': 5} >>> list(tree_iter(tree)) [1, 2, 3, 4, None, 5] >>> list(tree_iter(1)) [1] >>> list(tree_iter(None)) [None] Args: tree (pytree): A pytree to flatten. is_leaf (callable, optional): An extra leaf predicate function that will be called at each flattening step. The function should have a single argument with signature ``is_leaf(node) -> bool``. If it returns :data:`True`, the whole subtree being treated as a leaf. Otherwise, the default pytree registry will be used to determine a node is a leaf or not. If the function is not specified, the default pytree registry will be used. Returns: An iterator over the leaf values. TrVr])rXrr_s r;rrAs#6      r=c4tj||ddS)aDGet the leaves of a pytree. See also :func:`tree_flatten`. >>> tree = {'b': (2, [3, 4]), 'a': 1, 'c': None, 'd': 5} >>> tree_leaves(tree) [1, 2, 3, 4, None, 5] >>> tree_leaves(1) [1] >>> tree_leaves(None) [None] Args: tree (pytree): A pytree to flatten. is_leaf (callable, optional): An extra leaf predicate function that will be called at each flattening step. The function should have a single argument with signature ``is_leaf(node) -> bool``. If it returns :data:`True`, the whole subtree being treated as a leaf. Otherwise, the default pytree registry will be used to determine a node is a leaf or not. If the function is not specified, the default pytree registry will be used. Returns: A list of leaf values. TrVr])rXr r_s r;r r ds#6      r=c4tj||ddS)aGet the treespec for a pytree. See also :func:`tree_flatten`. >>> tree = {'b': (2, [3, 4]), 'a': 1, 'c': None, 'd': 5} >>> tree_structure(tree) PyTreeSpec({'a': *, 'b': (*, [*, *]), 'c': *, 'd': *}, NoneIsLeaf) >>> tree_structure(1) PyTreeSpec(*, NoneIsLeaf) >>> tree_structure(None) PyTreeSpec(*, NoneIsLeaf) Args: tree (pytree): A pytree to flatten. is_leaf (callable, optional): An extra leaf predicate function that will be called at each flattening step. The function should have a single argument with signature ``is_leaf(node) -> bool``. If it returns :data:`True`, the whole subtree being treated as a leaf. Otherwise, the default pytree registry will be used to determine a node is a leaf or not. If the function is not specified, the default pytree registry will be used. Returns: A treespec object representing the structure of the pytree. TrVr])rXr"r_s r;r"r"s#6     r=r[restsc:tj||g||dddS)aMap a multi-input function over pytree args to produce a new pytree. See also :func:`tree_map_`. >>> tree_map(lambda x: x + 1, {'x': 7, 'y': (42, 64)}) {'x': 8, 'y': (43, 65)} >>> tree_map(lambda x: x is None, {'x': 7, 'y': (42, 64), 'z': None}) {'x': False, 'y': (False, False), 'z': True} If multiple inputs are given, the structure of the tree is taken from the first input; subsequent inputs need only have ``tree`` as a prefix: >>> tree_map(lambda x, y: [x] + y, [5, 6], [[7, 9], [1, 2]]) [[5, 7, 9], [6, 1, 2]] Args: func (callable): A function that takes ``1 + len(rests)`` arguments, to be applied at the corresponding leaves of the pytrees. tree (pytree): A pytree to be mapped over, with each leaf providing the first positional argument to function ``func``. rests (tuple of pytree): A tuple of pytrees, each of which has the same structure as ``tree`` or has ``tree`` as a prefix. is_leaf (callable, optional): An extra leaf predicate function that will be called at each flattening step. The function should have a single argument with signature ``is_leaf(node) -> bool``. If it returns :data:`True`, the whole subtree being treated as a leaf. Otherwise, the default pytree registry will be used to determine a node is a leaf or not. If the function is not specified, the default pytree registry will be used. Returns: A new pytree with the same structure as ``tree`` but with the value at each leaf given by ``func(x, *xs)`` where ``x`` is the value at the corresponding leaf in ``tree`` and ``xs`` is the tuple of values at corresponding nodes in ``rests``. TrVr])rXr#r3rZr[rks r;r#r#s6N ??       r=c:tj||g||dddS)aTLike :func:`tree_map`, but do an inplace call on each leaf and return the original tree. See also :func:`tree_map`. Args: func (callable): A function that takes ``1 + len(rests)`` arguments, to be applied at the corresponding leaves of the pytrees. tree (pytree): A pytree to be mapped over, with each leaf providing the first positional argument to function ``func``. rests (tuple of pytree): A tuple of pytrees, each of which has the same structure as ``tree`` or has ``tree`` as a prefix. is_leaf (callable, optional): An extra leaf predicate function that will be called at each flattening step. The function should have a single argument with signature ``is_leaf(node) -> bool``. If it returns :data:`True`, the whole subtree being treated as a leaf. Otherwise, the default pytree registry will be used to determine a node is a leaf or not. If the function is not specified, the default pytree registry will be used. Returns: The original ``tree`` with the value at each leaf is given by the side-effect of function ``func(x, *xs)`` (not the return value) where ``x`` is the value at the corresponding leaf in ``tree`` and ``xs`` is the tuple of values at values at corresponding nodes in ``rests``. TrVr])rXr%rms r;r%r%s78          r= __type_or_types_or_predcyr9rrs r;map_onlyrvr=cyr9rtrus r;rvrvrwr=cyr9rtrus r;rvrvrwr=cyr9rtrus r;rvrv"rwr=cyr9rtrus r;rvrv'rwr=cDtttfs-tjdk\r-tt j rdtdtffd ntrn tddttgtfdttgtfffd }|S)a Suppose you are writing a tree_map over tensors, leaving everything else unchanged. Ordinarily you would have to write: def go(t): if isinstance(t, Tensor): return ... else: return t With this function, you only need to write: @map_only(Tensor) def go(t): return ... You can also directly use 'tree_map_only' roxr4ct|Sr9rd)r}rrs r;predzmap_only..predFsa!89 9r=z9Argument must be a type, a tuple of types, or a callable.r3c`tjdtdtffd }|S)Nr}r4c(|r|S|Sr9rt)r}r3rs r;r<z*map_only..wrapper..wrappedOsAwAwHr=)r>r?r/r)r3r<rs` r;wrapperzmap_only..wrapperNs3   q S    r=) rdrftuplesys version_infotypes UnionTyperboolcallablererr/)rrrrs` @r;rvrv,s*)D%=9 G# . @ :C :D : ) *&STThsCx(XseSj-A Nr=cyr9rtrrr3rZr[s r;r&r&Zr=cyr9rtrs r;r&r&drr=cyr9rtrs r;r&r&nrr=cyr9rtrs r;r&r&xrr=c<tt||||SNrj)r#rvrs r;r&r&s! 5H45d;T7 SSr=cyr9rtrs r;r'r'rr=cyr9rtrs r;r'r'rr=cyr9rtrs r;r'r'rr=cyr9rtrs r;r'r'rr=c<tt||||Sr)r%rvrs r;r'r's! 6X56tz tree_all_only..L1Z?-KtAwL" ")rrrrrZr[rs`` r;r*r*" $0I L L LLr=cyr9rtrs r;r+r+rr=cyr9rtrs r;r+r+rr=cyr9rtrs r;r+r+ rr=cJt||}tfd|DS)Nrjc3HK|]}t|s|ywr9rrs r;rz tree_any_only..rr)rrrs`` r;r+r+rr= prefix_tree full_treec6tj|||ddS)aReturn a list of broadcasted leaves in ``prefix_tree`` to match the number of leaves in ``full_tree``. If a ``prefix_tree`` is a prefix of a ``full_tree``, this means the ``full_tree`` can be constructed by replacing the leaves of ``prefix_tree`` with appropriate **subtrees**. This function returns a list of leaves with the same size as ``full_tree``. The leaves are replicated from ``prefix_tree``. The number of replicas is determined by the corresponding subtree in ``full_tree``. >>> broadcast_prefix(1, [1, 2, 3]) [1, 1, 1] >>> broadcast_prefix([1, 2, 3], [1, 2, 3]) [1, 2, 3] >>> broadcast_prefix([1, 2, 3], [1, 2, 3, 4]) Traceback (most recent call last): ... ValueError: list arity mismatch; expected: 3, got: 4; list: [1, 2, 3, 4]. >>> broadcast_prefix([1, 2, 3], [1, 2, (3, 4)]) [1, 2, 3, 3] >>> broadcast_prefix([1, 2, 3], [1, 2, {'a': 3, 'b': 4, 'c': (None, 5)}]) [1, 2, 3, 3, 3, 3] Args: prefix_tree (pytree): A pytree with the same structure as a prefix of ``full_tree``. full_tree (pytree): A pytree with the same structure as a suffix of ``prefix_tree``. is_leaf (callable, optional): An extra leaf predicate function that will be called at each flattening step. The function should have a single argument with signature ``is_leaf(node) -> bool``. If it returns :data:`True`, the whole subtree being treated as a leaf. Otherwise, the default pytree registry will be used to determine a node is a leaf or not. If the function is not specified, the default pytree registry will be used. Returns: A list of leaves in ``prefix_tree`` broadcasted to match the number of leaves in ``full_tree``. TrVr])rXbroadcast_prefix)rrr[s r;rrs'N  " "  r=ct|tsJtdg|jz|} t |||S#t $rYywxYw)Nrrj)rdrr num_leavesr ValueError)rZrar[rs r;_broadcast_to_and_flattenrVsS h )) )sX%8%88(CIiAA s < AAprotocolct|tstdt|dddlm}m}|tdg|jz|}|||S)z&Serialize a treespec to a JSON string.zVtreespec_dumps(spec): Expected `spec` to be instance of TreeSpec but got item of type rcrK)r"r,r)r) rdrrerfrLr"r,rr)rar_tree_structure_treespec_dumps orig_treespecs r;r,r,csa h )--1(^, bool``. If it returns :data:`True`, the whole subtree being treated as a leaf. Otherwise, the default pytree registry will be used to determine a node is a leaf or not. If the function is not specified, the default pytree registry will be used. Returns: A tuple where the first element is a list of (key path, leaf) pairs, and the second element is a :class:`TreeSpec` representing the structure of the flattened tree. rIrMr_s r;rrs( M NNr=ctd)a8Gets the leaves of a pytree like ``tree_leaves`` and returns each leaf's key path. Args: tree: a pytree. If it contains a custom type, that type must be registered with an appropriate `tree_flatten_with_path_fn` when registered with :func:`register_pytree_node`. is_leaf: An extra leaf predicate function that will be called at each flattening step. The function should have a single argument with signature ``is_leaf(node) -> bool``. If it returns :data:`True`, the whole subtree being treated as a leaf. Otherwise, the default pytree registry will be used to determine a node is a leaf or not. If the function is not specified, the default pytree registry will be used. Returns: A list of (key path, leaf) pairs. rIrr_s r;r!r!s$ M NNr=ctd)aLike :func:`tree_map`, but the provided callable takes an additional key path argument. Args: func: A function that takes ``2 + len(rests)`` arguments, to be applied at the corresponding leaves of the pytrees. The first positional argument to ``func`` is the key path of the leaf in question. The second positional argument is the value of the leaf. tree: A pytree to be mapped over, with each leaf providing the first positional argument to function ``func``. rests: A tuple of pytrees, each of which has the same structure as ``tree`` or has ``tree`` as a prefix. is_leaf: An extra leaf predicate function that will be called at each flattening step. The function should have a single argument with signature ``is_leaf(node) -> bool``. If it returns :data:`True`, the whole subtree being treated as a leaf. Otherwise, the default pytree registry will be used to determine a node is a leaf or not. If the function is not specified, the default pytree registry will be used. Returns A new pytree with the same structure as ``tree`` but with the value at each leaf given by ``func(keypath, x, *xs)`` where ``keypath`` is the key path at the corresponding leaf in ``tree``, ``x`` is the value at that leaf, and ``xs`` is the tuple of values at corresponding nodes in ``rests``. rIrrms r;r$r$s: M NNr=kpctd)z9Given a key path, return a pretty-printed representation.rIr)rs r;rr M NNr=objctd)zAGiven an object and a key path, return the value at the key path.rIr)rrs r;rrrr=Tr9)Y__doc__r>rrtypingrrrrrrr r r r typing_extensionsr rXrtorch.utils._pytreeutilsrLr__all__r/r0r1r2rrrrrOpTreeUnflattenFuncrrrKeyPathFlattenWithKeysFuncr@rr FutureWarningrTrNrrrrr r"r#r%Type2Type3rrTypeAnyFn2Fn3FnFnAny MapOnlyFnrvr&r'r(r)r*r+rrintr,r-rr.rfrrrr!r$rr_cxx_pytree_imported_cxx_pytree_pending_importsr6r7rtr=r;rs     ) %%( F CL CL CL CL   xtCy''9!::; (3-169: #7?@y/9: /!2G!;< # xtE(C-4H/I3/N)OOP *=+/9==A:>> c>> > #3- > ""56 >$$9:>##67> >B I +/9==A0 c00 0 #3- 0 ""56 0$$9:0 0  0p+/9==A c    #3-  ""56  $$9:   437- - hx~. /- 49h -`38C=3H33:37    hx~. / c] J37    hx~. /  #Y J37    hx~. /  N37 . 38 . . .hx~. / .  .j37 # 38 # # #hx~. / #  #L d1gtAw d1gtAwQ'(wDIuT#Y^4eooEFGDIuT#Y^445Gad }a aAg"# qc1f# aS(C5#:.. /  eAqDkiAq#I6O  eAq!Gn3q!QPS|CT9U  d1g)Bq#vJ2G   g)E#J2G  hud{&; %PS*@U +"7HcUD[,A#AB+uSz+\ 37 !!W QV* hx~. /     37 "1a4[ aCi. hx~. /     37 "1a7^ aAsl  hx~. /     37 %seTk2 * hx~. /    37 T"7HcUD[,A#ABT *T Thx~. / T  T 37 !!W QV* hx~. /     37 "1a4[ aCi. hx~. /     37 "1a7^ aAsl  hx~. /     37 %seTk2 * hx~. /    37 U"7HcUD[,A#ABU *U Uhx~. / U  U37% C5$; % %hx~. /% %37% C5$; % %hx~. /% % 37 !W QW+ hx~. /     37 1a4[ aDj/ hx~. /     37 1a7^ aAtm  hx~. /    37 MM +M Mhx~. / M  M 37 !W QW+ hx~. /     37 1a4[ aDj/ hx~. /     37 1a7^ aAtm  hx~. /    37 MM +M Mhx~. / M  M37---hx~. /- #Y -v37    hx~. / d3i  =X =# =# = s x  h3E4>E 7x<737O O hx~. /O 4gsl# $h ./O237O O hx~. /O % O237 O 38 O O Ohx~. / O  O@OwO3O OO'OcO $773LD&!42623r=