diff --git a/LICENSE b/LICENSE new file mode 100644 --- /dev/null +++ b/LICENSE @@ -0,0 +1,661 @@ + GNU AFFERO GENERAL PUBLIC LICENSE + Version 3, 19 November 2007 + + Copyright (C) 2007 Free Software Foundation, Inc. + Everyone is permitted to copy and distribute verbatim copies + of this license document, but changing it is not allowed. + + Preamble + + The GNU Affero General Public License is a free, copyleft license for +software and other kinds of works, specifically designed to ensure +cooperation with the community in the case of network server software. + + The licenses for most software and other practical works are designed +to take away your freedom to share and change the works. By contrast, +our General Public Licenses are intended to guarantee your freedom to +share and change all versions of a program--to make sure it remains free +software for all its users. + + When we speak of free software, we are referring to freedom, not +price. Our General Public Licenses are designed to make sure that you +have the freedom to distribute copies of free software (and charge for +them if you wish), that you receive source code or can get it if you +want it, that you can change the software or use pieces of it in new +free programs, and that you know you can do these things. + + Developers that use our General Public Licenses protect your rights +with two steps: (1) assert copyright on the software, and (2) offer +you this License which gives you legal permission to copy, distribute +and/or modify the software. + + A secondary benefit of defending all users' freedom is that +improvements made in alternate versions of the program, if they +receive widespread use, become available for other developers to +incorporate. Many developers of free software are heartened and +encouraged by the resulting cooperation. However, in the case of +software used on network servers, this result may fail to come about. +The GNU General Public License permits making a modified version and +letting the public access it on a server without ever releasing its +source code to the public. + + The GNU Affero General Public License is designed specifically to +ensure that, in such cases, the modified source code becomes available +to the community. It requires the operator of a network server to +provide the source code of the modified version running there to the +users of that server. Therefore, public use of a modified version, on +a publicly accessible server, gives the public access to the source +code of the modified version. + + An older license, called the Affero General Public License and +published by Affero, was designed to accomplish similar goals. This is +a different license, not a version of the Affero GPL, but Affero has +released a new version of the Affero GPL which permits relicensing under +this license. + + The precise terms and conditions for copying, distribution and +modification follow. + + TERMS AND CONDITIONS + + 0. Definitions. + + "This License" refers to version 3 of the GNU Affero General Public License. + + "Copyright" also means copyright-like laws that apply to other kinds of +works, such as semiconductor masks. + + "The Program" refers to any copyrightable work licensed under this +License. Each licensee is addressed as "you". "Licensees" and +"recipients" may be individuals or organizations. + + To "modify" a work means to copy from or adapt all or part of the work +in a fashion requiring copyright permission, other than the making of an +exact copy. The resulting work is called a "modified version" of the +earlier work or a work "based on" the earlier work. + + A "covered work" means either the unmodified Program or a work based +on the Program. + + To "propagate" a work means to do anything with it that, without +permission, would make you directly or secondarily liable for +infringement under applicable copyright law, except executing it on a +computer or modifying a private copy. Propagation includes copying, +distribution (with or without modification), making available to the +public, and in some countries other activities as well. + + To "convey" a work means any kind of propagation that enables other +parties to make or receive copies. Mere interaction with a user through +a computer network, with no transfer of a copy, is not conveying. + + An interactive user interface displays "Appropriate Legal Notices" +to the extent that it includes a convenient and prominently visible +feature that (1) displays an appropriate copyright notice, and (2) +tells the user that there is no warranty for the work (except to the +extent that warranties are provided), that licensees may convey the +work under this License, and how to view a copy of this License. If +the interface presents a list of user commands or options, such as a +menu, a prominent item in the list meets this criterion. + + 1. Source Code. + + The "source code" for a work means the preferred form of the work +for making modifications to it. "Object code" means any non-source +form of a work. + + A "Standard Interface" means an interface that either is an official +standard defined by a recognized standards body, or, in the case of +interfaces specified for a particular programming language, one that +is widely used among developers working in that language. + + The "System Libraries" of an executable work include anything, other +than the work as a whole, that (a) is included in the normal form of +packaging a Major Component, but which is not part of that Major +Component, and (b) serves only to enable use of the work with that +Major Component, or to implement a Standard Interface for which an +implementation is available to the public in source code form. A +"Major Component", in this context, means a major essential component +(kernel, window system, and so on) of the specific operating system +(if any) on which the executable work runs, or a compiler used to +produce the work, or an object code interpreter used to run it. + + The "Corresponding Source" for a work in object code form means all +the source code needed to generate, install, and (for an executable +work) run the object code and to modify the work, including scripts to +control those activities. However, it does not include the work's +System Libraries, or general-purpose tools or generally available free +programs which are used unmodified in performing those activities but +which are not part of the work. For example, Corresponding Source +includes interface definition files associated with source files for +the work, and the source code for shared libraries and dynamically +linked subprograms that the work is specifically designed to require, +such as by intimate data communication or control flow between those +subprograms and other parts of the work. + + The Corresponding Source need not include anything that users +can regenerate automatically from other parts of the Corresponding +Source. + + The Corresponding Source for a work in source code form is that +same work. + + 2. Basic Permissions. + + All rights granted under this License are granted for the term of +copyright on the Program, and are irrevocable provided the stated +conditions are met. This License explicitly affirms your unlimited +permission to run the unmodified Program. The output from running a +covered work is covered by this License only if the output, given its +content, constitutes a covered work. This License acknowledges your +rights of fair use or other equivalent, as provided by copyright law. + + You may make, run and propagate covered works that you do not +convey, without conditions so long as your license otherwise remains +in force. You may convey covered works to others for the sole purpose +of having them make modifications exclusively for you, or provide you +with facilities for running those works, provided that you comply with +the terms of this License in conveying all material for which you do +not control copyright. Those thus making or running the covered works +for you must do so exclusively on your behalf, under your direction +and control, on terms that prohibit them from making any copies of +your copyrighted material outside their relationship with you. + + Conveying under any other circumstances is permitted solely under +the conditions stated below. Sublicensing is not allowed; section 10 +makes it unnecessary. + + 3. Protecting Users' Legal Rights From Anti-Circumvention Law. + + No covered work shall be deemed part of an effective technological +measure under any applicable law fulfilling obligations under article +11 of the WIPO copyright treaty adopted on 20 December 1996, or +similar laws prohibiting or restricting circumvention of such +measures. + + When you convey a covered work, you waive any legal power to forbid +circumvention of technological measures to the extent such circumvention +is effected by exercising rights under this License with respect to +the covered work, and you disclaim any intention to limit operation or +modification of the work as a means of enforcing, against the work's +users, your or third parties' legal rights to forbid circumvention of +technological measures. + + 4. Conveying Verbatim Copies. + + You may convey verbatim copies of the Program's source code as you +receive it, in any medium, provided that you conspicuously and +appropriately publish on each copy an appropriate copyright notice; +keep intact all notices stating that this License and any +non-permissive terms added in accord with section 7 apply to the code; +keep intact all notices of the absence of any warranty; and give all +recipients a copy of this License along with the Program. + + You may charge any price or no price for each copy that you convey, +and you may offer support or warranty protection for a fee. + + 5. Conveying Modified Source Versions. + + You may convey a work based on the Program, or the modifications to +produce it from the Program, in the form of source code under the +terms of section 4, provided that you also meet all of these conditions: + + a) The work must carry prominent notices stating that you modified + it, and giving a relevant date. + + b) The work must carry prominent notices stating that it is + released under this License and any conditions added under section + 7. This requirement modifies the requirement in section 4 to + "keep intact all notices". + + c) You must license the entire work, as a whole, under this + License to anyone who comes into possession of a copy. This + License will therefore apply, along with any applicable section 7 + additional terms, to the whole of the work, and all its parts, + regardless of how they are packaged. This License gives no + permission to license the work in any other way, but it does not + invalidate such permission if you have separately received it. + + d) If the work has interactive user interfaces, each must display + Appropriate Legal Notices; however, if the Program has interactive + interfaces that do not display Appropriate Legal Notices, your + work need not make them do so. + + A compilation of a covered work with other separate and independent +works, which are not by their nature extensions of the covered work, +and which are not combined with it such as to form a larger program, +in or on a volume of a storage or distribution medium, is called an +"aggregate" if the compilation and its resulting copyright are not +used to limit the access or legal rights of the compilation's users +beyond what the individual works permit. Inclusion of a covered work +in an aggregate does not cause this License to apply to the other +parts of the aggregate. + + 6. Conveying Non-Source Forms. + + You may convey a covered work in object code form under the terms +of sections 4 and 5, provided that you also convey the +machine-readable Corresponding Source under the terms of this License, +in one of these ways: + + a) Convey the object code in, or embodied in, a physical product + (including a physical distribution medium), accompanied by the + Corresponding Source fixed on a durable physical medium + customarily used for software interchange. + + b) Convey the object code in, or embodied in, a physical product + (including a physical distribution medium), accompanied by a + written offer, valid for at least three years and valid for as + long as you offer spare parts or customer support for that product + model, to give anyone who possesses the object code either (1) a + copy of the Corresponding Source for all the software in the + product that is covered by this License, on a durable physical + medium customarily used for software interchange, for a price no + more than your reasonable cost of physically performing this + conveying of source, or (2) access to copy the + Corresponding Source from a network server at no charge. + + c) Convey individual copies of the object code with a copy of the + written offer to provide the Corresponding Source. This + alternative is allowed only occasionally and noncommercially, and + only if you received the object code with such an offer, in accord + with subsection 6b. + + d) Convey the object code by offering access from a designated + place (gratis or for a charge), and offer equivalent access to the + Corresponding Source in the same way through the same place at no + further charge. You need not require recipients to copy the + Corresponding Source along with the object code. If the place to + copy the object code is a network server, the Corresponding Source + may be on a different server (operated by you or a third party) + that supports equivalent copying facilities, provided you maintain + clear directions next to the object code saying where to find the + Corresponding Source. Regardless of what server hosts the + Corresponding Source, you remain obligated to ensure that it is + available for as long as needed to satisfy these requirements. + + e) Convey the object code using peer-to-peer transmission, provided + you inform other peers where the object code and Corresponding + Source of the work are being offered to the general public at no + charge under subsection 6d. + + A separable portion of the object code, whose source code is excluded +from the Corresponding Source as a System Library, need not be +included in conveying the object code work. + + A "User Product" is either (1) a "consumer product", which means any +tangible personal property which is normally used for personal, family, +or household purposes, or (2) anything designed or sold for incorporation +into a dwelling. In determining whether a product is a consumer product, +doubtful cases shall be resolved in favor of coverage. For a particular +product received by a particular user, "normally used" refers to a +typical or common use of that class of product, regardless of the status +of the particular user or of the way in which the particular user +actually uses, or expects or is expected to use, the product. A product +is a consumer product regardless of whether the product has substantial +commercial, industrial or non-consumer uses, unless such uses represent +the only significant mode of use of the product. + + "Installation Information" for a User Product means any methods, +procedures, authorization keys, or other information required to install +and execute modified versions of a covered work in that User Product from +a modified version of its Corresponding Source. The information must +suffice to ensure that the continued functioning of the modified object +code is in no case prevented or interfered with solely because +modification has been made. + + If you convey an object code work under this section in, or with, or +specifically for use in, a User Product, and the conveying occurs as +part of a transaction in which the right of possession and use of the +User Product is transferred to the recipient in perpetuity or for a +fixed term (regardless of how the transaction is characterized), the +Corresponding Source conveyed under this section must be accompanied +by the Installation Information. But this requirement does not apply +if neither you nor any third party retains the ability to install +modified object code on the User Product (for example, the work has +been installed in ROM). + + The requirement to provide Installation Information does not include a +requirement to continue to provide support service, warranty, or updates +for a work that has been modified or installed by the recipient, or for +the User Product in which it has been modified or installed. Access to a +network may be denied when the modification itself materially and +adversely affects the operation of the network or violates the rules and +protocols for communication across the network. + + Corresponding Source conveyed, and Installation Information provided, +in accord with this section must be in a format that is publicly +documented (and with an implementation available to the public in +source code form), and must require no special password or key for +unpacking, reading or copying. + + 7. Additional Terms. + + "Additional permissions" are terms that supplement the terms of this +License by making exceptions from one or more of its conditions. +Additional permissions that are applicable to the entire Program shall +be treated as though they were included in this License, to the extent +that they are valid under applicable law. If additional permissions +apply only to part of the Program, that part may be used separately +under those permissions, but the entire Program remains governed by +this License without regard to the additional permissions. + + When you convey a copy of a covered work, you may at your option +remove any additional permissions from that copy, or from any part of +it. (Additional permissions may be written to require their own +removal in certain cases when you modify the work.) You may place +additional permissions on material, added by you to a covered work, +for which you have or can give appropriate copyright permission. + + Notwithstanding any other provision of this License, for material you +add to a covered work, you may (if authorized by the copyright holders of +that material) supplement the terms of this License with terms: + + a) Disclaiming warranty or limiting liability differently from the + terms of sections 15 and 16 of this License; or + + b) Requiring preservation of specified reasonable legal notices or + author attributions in that material or in the Appropriate Legal + Notices displayed by works containing it; or + + c) Prohibiting misrepresentation of the origin of that material, or + requiring that modified versions of such material be marked in + reasonable ways as different from the original version; or + + d) Limiting the use for publicity purposes of names of licensors or + authors of the material; or + + e) Declining to grant rights under trademark law for use of some + trade names, trademarks, or service marks; or + + f) Requiring indemnification of licensors and authors of that + material by anyone who conveys the material (or modified versions of + it) with contractual assumptions of liability to the recipient, for + any liability that these contractual assumptions directly impose on + those licensors and authors. + + All other non-permissive additional terms are considered "further +restrictions" within the meaning of section 10. If the Program as you +received it, or any part of it, contains a notice stating that it is +governed by this License along with a term that is a further +restriction, you may remove that term. If a license document contains +a further restriction but permits relicensing or conveying under this +License, you may add to a covered work material governed by the terms +of that license document, provided that the further restriction does +not survive such relicensing or conveying. + + If you add terms to a covered work in accord with this section, you +must place, in the relevant source files, a statement of the +additional terms that apply to those files, or a notice indicating +where to find the applicable terms. + + Additional terms, permissive or non-permissive, may be stated in the +form of a separately written license, or stated as exceptions; +the above requirements apply either way. + + 8. Termination. + + You may not propagate or modify a covered work except as expressly +provided under this License. Any attempt otherwise to propagate or +modify it is void, and will automatically terminate your rights under +this License (including any patent licenses granted under the third +paragraph of section 11). + + However, if you cease all violation of this License, then your +license from a particular copyright holder is reinstated (a) +provisionally, unless and until the copyright holder explicitly and +finally terminates your license, and (b) permanently, if the copyright +holder fails to notify you of the violation by some reasonable means +prior to 60 days after the cessation. + + Moreover, your license from a particular copyright holder is +reinstated permanently if the copyright holder notifies you of the +violation by some reasonable means, this is the first time you have +received notice of violation of this License (for any work) from that +copyright holder, and you cure the violation prior to 30 days after +your receipt of the notice. + + Termination of your rights under this section does not terminate the +licenses of parties who have received copies or rights from you under +this License. If your rights have been terminated and not permanently +reinstated, you do not qualify to receive new licenses for the same +material under section 10. + + 9. Acceptance Not Required for Having Copies. + + You are not required to accept this License in order to receive or +run a copy of the Program. Ancillary propagation of a covered work +occurring solely as a consequence of using peer-to-peer transmission +to receive a copy likewise does not require acceptance. However, +nothing other than this License grants you permission to propagate or +modify any covered work. These actions infringe copyright if you do +not accept this License. Therefore, by modifying or propagating a +covered work, you indicate your acceptance of this License to do so. + + 10. Automatic Licensing of Downstream Recipients. + + Each time you convey a covered work, the recipient automatically +receives a license from the original licensors, to run, modify and +propagate that work, subject to this License. You are not responsible +for enforcing compliance by third parties with this License. + + An "entity transaction" is a transaction transferring control of an +organization, or substantially all assets of one, or subdividing an +organization, or merging organizations. If propagation of a covered +work results from an entity transaction, each party to that +transaction who receives a copy of the work also receives whatever +licenses to the work the party's predecessor in interest had or could +give under the previous paragraph, plus a right to possession of the +Corresponding Source of the work from the predecessor in interest, if +the predecessor has it or can get it with reasonable efforts. + + You may not impose any further restrictions on the exercise of the +rights granted or affirmed under this License. For example, you may +not impose a license fee, royalty, or other charge for exercise of +rights granted under this License, and you may not initiate litigation +(including a cross-claim or counterclaim in a lawsuit) alleging that +any patent claim is infringed by making, using, selling, offering for +sale, or importing the Program or any portion of it. + + 11. Patents. + + A "contributor" is a copyright holder who authorizes use under this +License of the Program or a work on which the Program is based. The +work thus licensed is called the contributor's "contributor version". + + A contributor's "essential patent claims" are all patent claims +owned or controlled by the contributor, whether already acquired or +hereafter acquired, that would be infringed by some manner, permitted +by this License, of making, using, or selling its contributor version, +but do not include claims that would be infringed only as a +consequence of further modification of the contributor version. For +purposes of this definition, "control" includes the right to grant +patent sublicenses in a manner consistent with the requirements of +this License. + + Each contributor grants you a non-exclusive, worldwide, royalty-free +patent license under the contributor's essential patent claims, to +make, use, sell, offer for sale, import and otherwise run, modify and +propagate the contents of its contributor version. + + In the following three paragraphs, a "patent license" is any express +agreement or commitment, however denominated, not to enforce a patent +(such as an express permission to practice a patent or covenant not to +sue for patent infringement). To "grant" such a patent license to a +party means to make such an agreement or commitment not to enforce a +patent against the party. + + If you convey a covered work, knowingly relying on a patent license, +and the Corresponding Source of the work is not available for anyone +to copy, free of charge and under the terms of this License, through a +publicly available network server or other readily accessible means, +then you must either (1) cause the Corresponding Source to be so +available, or (2) arrange to deprive yourself of the benefit of the +patent license for this particular work, or (3) arrange, in a manner +consistent with the requirements of this License, to extend the patent +license to downstream recipients. "Knowingly relying" means you have +actual knowledge that, but for the patent license, your conveying the +covered work in a country, or your recipient's use of the covered work +in a country, would infringe one or more identifiable patents in that +country that you have reason to believe are valid. + + If, pursuant to or in connection with a single transaction or +arrangement, you convey, or propagate by procuring conveyance of, a +covered work, and grant a patent license to some of the parties +receiving the covered work authorizing them to use, propagate, modify +or convey a specific copy of the covered work, then the patent license +you grant is automatically extended to all recipients of the covered +work and works based on it. + + A patent license is "discriminatory" if it does not include within +the scope of its coverage, prohibits the exercise of, or is +conditioned on the non-exercise of one or more of the rights that are +specifically granted under this License. You may not convey a covered +work if you are a party to an arrangement with a third party that is +in the business of distributing software, under which you make payment +to the third party based on the extent of your activity of conveying +the work, and under which the third party grants, to any of the +parties who would receive the covered work from you, a discriminatory +patent license (a) in connection with copies of the covered work +conveyed by you (or copies made from those copies), or (b) primarily +for and in connection with specific products or compilations that +contain the covered work, unless you entered into that arrangement, +or that patent license was granted, prior to 28 March 2007. + + Nothing in this License shall be construed as excluding or limiting +any implied license or other defenses to infringement that may +otherwise be available to you under applicable patent law. + + 12. No Surrender of Others' Freedom. + + If conditions are imposed on you (whether by court order, agreement or +otherwise) that contradict the conditions of this License, they do not +excuse you from the conditions of this License. If you cannot convey a +covered work so as to satisfy simultaneously your obligations under this +License and any other pertinent obligations, then as a consequence you may +not convey it at all. For example, if you agree to terms that obligate you +to collect a royalty for further conveying from those to whom you convey +the Program, the only way you could satisfy both those terms and this +License would be to refrain entirely from conveying the Program. + + 13. Remote Network Interaction; Use with the GNU General Public License. + + Notwithstanding any other provision of this License, if you modify the +Program, your modified version must prominently offer all users +interacting with it remotely through a computer network (if your version +supports such interaction) an opportunity to receive the Corresponding +Source of your version by providing access to the Corresponding Source +from a network server at no charge, through some standard or customary +means of facilitating copying of software. This Corresponding Source +shall include the Corresponding Source for any work covered by version 3 +of the GNU General Public License that is incorporated pursuant to the +following paragraph. + + Notwithstanding any other provision of this License, you have +permission to link or combine any covered work with a work licensed +under version 3 of the GNU General Public License into a single +combined work, and to convey the resulting work. The terms of this +License will continue to apply to the part which is the covered work, +but the work with which it is combined will remain governed by version +3 of the GNU General Public License. + + 14. Revised Versions of this License. + + The Free Software Foundation may publish revised and/or new versions of +the GNU Affero General Public License from time to time. Such new versions +will be similar in spirit to the present version, but may differ in detail to +address new problems or concerns. + + Each version is given a distinguishing version number. If the +Program specifies that a certain numbered version of the GNU Affero General +Public License "or any later version" applies to it, you have the +option of following the terms and conditions either of that numbered +version or of any later version published by the Free Software +Foundation. If the Program does not specify a version number of the +GNU Affero General Public License, you may choose any version ever published +by the Free Software Foundation. + + If the Program specifies that a proxy can decide which future +versions of the GNU Affero General Public License can be used, that proxy's +public statement of acceptance of a version permanently authorizes you +to choose that version for the Program. + + Later license versions may give you additional or different +permissions. However, no additional obligations are imposed on any +author or copyright holder as a result of your choosing to follow a +later version. + + 15. Disclaimer of Warranty. + + THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY +APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT +HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY +OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, +THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR +PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM +IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF +ALL NECESSARY SERVICING, REPAIR OR CORRECTION. + + 16. Limitation of Liability. + + IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING +WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS +THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY +GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE +USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF +DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD +PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS), +EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF +SUCH DAMAGES. + + 17. Interpretation of Sections 15 and 16. + + If the disclaimer of warranty and limitation of liability provided +above cannot be given local legal effect according to their terms, +reviewing courts shall apply local law that most closely approximates +an absolute waiver of all civil liability in connection with the +Program, unless a warranty or assumption of liability accompanies a +copy of the Program in return for a fee. + + END OF TERMS AND CONDITIONS + + How to Apply These Terms to Your New Programs + + If you develop a new program, and you want it to be of the greatest +possible use to the public, the best way to achieve this is to make it +free software which everyone can redistribute and change under these terms. + + To do so, attach the following notices to the program. It is safest +to attach them to the start of each source file to most effectively +state the exclusion of warranty; and each file should have at least +the "copyright" line and a pointer to where the full notice is found. + + + Copyright (C) + + This program is free software: you can redistribute it and/or modify + it under the terms of the GNU Affero General Public License as published by + the Free Software Foundation, either version 3 of the License, or + (at your option) any later version. + + This program is distributed in the hope that it will be useful, + but WITHOUT ANY WARRANTY; without even the implied warranty of + MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + GNU Affero General Public License for more details. + + You should have received a copy of the GNU Affero General Public License + along with this program. If not, see . + +Also add information on how to contact you by electronic and paper mail. + + If your software can interact with users remotely through a computer +network, you should also make sure that it provides a way for users to +get its source. For example, if your program is a web application, its +interface could display a "Source" link that leads users to an archive +of the code. There are many ways you could offer source, and different +solutions will be better for different programs; see section 13 for the +specific requirements. + + You should also get your employer (if you work as a programmer) or school, +if any, to sign a "copyright disclaimer" for the program, if necessary. +For more information on this, and how to apply and follow the GNU AGPL, see +. diff --git a/NOTICE b/NOTICE new file mode 100644 --- /dev/null +++ b/NOTICE @@ -0,0 +1,9 @@ +Copyright (c) 2025 Wen Kokke, Matthew Daggitt, and Well-Typed LLP + +This program is free software: you can redistribute it and/or modify it under the terms of the GNU Affero General Public License as published by the Free Software Foundation, only version 3 of the License. + +This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. +See the GNU Affero General Public License for more details. + +You should have received a copy of the GNU Affero General Public License along with this program. +If not, see . diff --git a/hs-onnxruntime-capi.cabal b/hs-onnxruntime-capi.cabal new file mode 100644 --- /dev/null +++ b/hs-onnxruntime-capi.cabal @@ -0,0 +1,73 @@ +cabal-version: 3.4 +build-type: Simple +name: hs-onnxruntime-capi +synopsis: Low-level bindings for ONNX Runtime +description: Low-level bindings for ONNX Runtime. +version: 0.1.0.0 +license: AGPL-3.0-only +author: Wen Kokke +maintainer: wenkokke@users.noreply.github.com +category: AI, Machine Learning +copyright: Copyright (c) 2025 Wen Kokke and Well-Typed LLP +license-files: + LICENSE + NOTICE + +extra-source-files: include/onnxruntime_c_api.h +tested-with: GHC ==9.6.7 || ==9.8.4 || ==9.10.2 || ==9.12.2 + +source-repository head + type: git + location: https://github.com/vehicle-lang/hs-onnxruntime + subdir: hs-onnxruntime-capi + +flag pkg-config + default: True + manual: False + description: + Use @pkg-config@ executable to locate foreign @onnxruntime@ library. + +flag use-bundled-header + default: True + manual: True + description: Use bundled copy of @onnxruntime_c_api.h@ header file + +common language + ghc-options: + -Wall -Wcompat -Wredundant-constraints + -Wunticked-promoted-constructors -Wunused-packages + + default-language: Haskell2010 + default-extensions: + ImportQualifiedPost + RankNTypes + ScopedTypeVariables + StandaloneKindSignatures + TypeApplications + + if impl(ghc <9.6.1) + ghc-options: -optc=-Wno-discarded-qualifiers + + if os(osx) + ghc-options: -optc=-Wno-incompatible-pointer-types-discards-qualifiers + +library + import: language + exposed-modules: Onnxruntime.CApi + build-tool-depends: hsc2hs:hsc2hs + hs-source-dirs: src + build-depends: + , base >=4.13 && <5 + , base-compat-constptr >=0.1 && <0.2 + , bytestring ^>=0.11 || ^>=0.12 + , vector ^>=0.12 || ^>=0.13 + + if flag(use-bundled-header) + includes: onnxruntime_c_api.h + include-dirs: include + + if flag(pkg-config) + pkgconfig-depends: libonnxruntime >=1.21 + + else + extra-libraries: onnxruntime diff --git a/include/onnxruntime_c_api.h b/include/onnxruntime_c_api.h new file mode 100644 --- /dev/null +++ b/include/onnxruntime_c_api.h @@ -0,0 +1,6195 @@ +// Copyright (c) Microsoft Corporation. All rights reserved. +// Licensed under the MIT License. + +// See docs\c_cxx\README.md on generating the Doxygen documentation from this file + +/** \mainpage ONNX Runtime + * + * ONNX Runtime is a high-performance inference and training graph execution engine for deep learning models. + * + * ONNX Runtime's C, C++ APIs offer an easy to use interface to onboard and execute onnx models. + * - \subpage c_cpp_api "Core C, C++ APIs" + * - \subpage training_c_cpp_api "Training C, C++ APIs for on-device training" + * + * \page c_cpp_api Core C, C++ APIs + *

C

+ * + * ::OrtApi - Click here to go to the structure with all C API functions. + * + *

C++

+ * + * ::Ort - Click here to go to the namespace holding all of the C++ wrapper classes + * + * It is a set of header only wrapper classes around the C API. The goal is to turn the C style return value error codes into C++ exceptions, and to + * automate memory management through standard C++ RAII principles. + * + * \addtogroup Global + * ONNX Runtime C API + * @{ + */ + +#pragma once +#include +#include +#include +#include + +/** \brief The API version defined in this header + * + * This value is used by some API functions to behave as this version of the header expects. + */ +#define ORT_API_VERSION 22 + +#ifdef __cplusplus +extern "C" { +#endif + +//! @} +// SAL2 Definitions +#ifndef _MSC_VER +#define _In_ +#define _In_z_ +#define _In_opt_ +#define _In_opt_z_ +#define _Out_ +#define _Outptr_ +#define _Out_opt_ +#define _Inout_ +#define _Inout_opt_ +#define _Frees_ptr_opt_ +#define _Ret_maybenull_ +#define _Ret_notnull_ +#define _Check_return_ +#define _Outptr_result_maybenull_ +#define _In_reads_(X) +#define _Inout_updates_(X) +#define _Out_writes_(X) +#define _Inout_updates_all_(X) +#define _Out_writes_bytes_all_(X) +#define _Out_writes_all_(X) +#define _Success_(X) +#define _Outptr_result_buffer_maybenull_(X) +#define ORT_ALL_ARGS_NONNULL __attribute__((nonnull)) +#else +#include +#define ORT_ALL_ARGS_NONNULL +#endif + +#ifdef _WIN32 +// Define ORT_DLL_IMPORT if your program is dynamically linked to Ort. +// dllexport is not used, we use a .def file. +#ifdef ORT_DLL_IMPORT +#define ORT_EXPORT __declspec(dllimport) +#else +#define ORT_EXPORT +#endif +#define ORT_API_CALL _stdcall +#define ORT_MUST_USE_RESULT +#define ORTCHAR_T wchar_t +#else +// To make symbols visible on macOS/iOS +#ifdef __APPLE__ +#define ORT_EXPORT __attribute__((visibility("default"))) +#else +#define ORT_EXPORT +#endif +#define ORT_API_CALL +#define ORT_MUST_USE_RESULT __attribute__((warn_unused_result)) +#define ORTCHAR_T char +#endif + +/// ORTCHAR_T, ORT_TSTR are reserved specifically for path handling. +/// All other strings are UTF-8 encoded, use char and std::string +#ifndef ORT_TSTR +#ifdef _WIN32 +#define ORT_TSTR(X) L##X +// When X is a macro, L##X is not defined. In this case, we need to use ORT_TSTR_ON_MACRO. +#define ORT_TSTR_ON_MACRO(X) L"" X +#else +#define ORT_TSTR(X) X +#define ORT_TSTR_ON_MACRO(X) X +#endif +#endif + +// On Windows, ORT_FILE is a wchar_t version of the __FILE__ macro. +// Otherwise, ORT_FILE is equivalent to __FILE__. +#ifndef ORT_FILE +#define ORT_FILE_INTERNAL(x) ORT_TSTR(x) +#define ORT_FILE ORT_FILE_INTERNAL(__FILE__) +#endif + +// Any pointer marked with _In_ or _Out_, cannot be NULL. + +// Windows users should use unicode paths when possible to bypass the MAX_PATH limitation +// Every pointer marked with _In_ or _Out_, cannot be NULL. Caller should ensure that. +// for ReleaseXXX(...) functions, they can accept NULL pointer. + +#ifdef __cplusplus +// For any compiler with C++11 support, MSVC 2015 and greater, or Clang version supporting noexcept. +// Such complex condition is needed because compilers set __cplusplus value differently. +#ifndef __has_feature +#define __has_feature(x) 0 +#endif +#if ((__cplusplus >= 201103L) || (_MSC_VER >= 1900) || (defined(__has_feature) && __has_feature(cxx_noexcept))) +#define NO_EXCEPTION noexcept +#else +#define NO_EXCEPTION throw() +#endif +#else +#define NO_EXCEPTION +#endif + +// __VA_ARGS__ on Windows and Linux are different +#define ORT_API(RETURN_TYPE, NAME, ...) RETURN_TYPE ORT_API_CALL NAME(__VA_ARGS__) NO_EXCEPTION + +#define ORT_API_STATUS(NAME, ...) \ + _Success_(return == 0) _Check_return_ _Ret_maybenull_ OrtStatusPtr ORT_API_CALL NAME(__VA_ARGS__) \ + NO_EXCEPTION ORT_MUST_USE_RESULT + +// XXX: Unfortunately, SAL annotations are known to not work with function pointers +#define ORT_API2_STATUS(NAME, ...) \ + _Check_return_ _Ret_maybenull_ OrtStatusPtr(ORT_API_CALL* NAME)(__VA_ARGS__) NO_EXCEPTION ORT_MUST_USE_RESULT + +// Used in *.cc files. Almost as same as ORT_API_STATUS, except without ORT_MUST_USE_RESULT and ORT_EXPORT +#define ORT_API_STATUS_IMPL(NAME, ...) \ + _Success_(return == 0) _Check_return_ _Ret_maybenull_ OrtStatusPtr ORT_API_CALL NAME(__VA_ARGS__) NO_EXCEPTION + +#define ORT_CLASS_RELEASE(X) void(ORT_API_CALL * Release##X)(_Frees_ptr_opt_ Ort##X * input) + +#ifdef __DOXYGEN__ +#undef ORT_API_STATUS +#define ORT_API_STATUS(NAME, ...) OrtStatus* NAME(__VA_ARGS__) +#undef ORT_API2_STATUS +#define ORT_API2_STATUS(NAME, ...) OrtStatus* NAME(__VA_ARGS__) +#undef ORT_CLASS_RELEASE +#define ORT_CLASS_RELEASE(X) void Release##X(Ort##X* input) +#undef NO_EXCEPTION +#define NO_EXCEPTION +#endif +/** \addtogroup Global + * ONNX Runtime C API + * @{ + */ + +/** Copied from TensorProto::DataType + * Currently, Ort doesn't support complex64, complex128 + */ +typedef enum ONNXTensorElementDataType { + ONNX_TENSOR_ELEMENT_DATA_TYPE_UNDEFINED, + ONNX_TENSOR_ELEMENT_DATA_TYPE_FLOAT, // maps to c type float + ONNX_TENSOR_ELEMENT_DATA_TYPE_UINT8, // maps to c type uint8_t + ONNX_TENSOR_ELEMENT_DATA_TYPE_INT8, // maps to c type int8_t + ONNX_TENSOR_ELEMENT_DATA_TYPE_UINT16, // maps to c type uint16_t + ONNX_TENSOR_ELEMENT_DATA_TYPE_INT16, // maps to c type int16_t + ONNX_TENSOR_ELEMENT_DATA_TYPE_INT32, // maps to c type int32_t + ONNX_TENSOR_ELEMENT_DATA_TYPE_INT64, // maps to c type int64_t + ONNX_TENSOR_ELEMENT_DATA_TYPE_STRING, // maps to c++ type std::string + ONNX_TENSOR_ELEMENT_DATA_TYPE_BOOL, + ONNX_TENSOR_ELEMENT_DATA_TYPE_FLOAT16, + ONNX_TENSOR_ELEMENT_DATA_TYPE_DOUBLE, // maps to c type double + ONNX_TENSOR_ELEMENT_DATA_TYPE_UINT32, // maps to c type uint32_t + ONNX_TENSOR_ELEMENT_DATA_TYPE_UINT64, // maps to c type uint64_t + ONNX_TENSOR_ELEMENT_DATA_TYPE_COMPLEX64, // complex with float32 real and imaginary components + ONNX_TENSOR_ELEMENT_DATA_TYPE_COMPLEX128, // complex with float64 real and imaginary components + ONNX_TENSOR_ELEMENT_DATA_TYPE_BFLOAT16, // Non-IEEE floating-point format based on IEEE754 single-precision + // float 8 types were introduced in onnx 1.14, see https://onnx.ai/onnx/technical/float8.html + ONNX_TENSOR_ELEMENT_DATA_TYPE_FLOAT8E4M3FN, // Non-IEEE floating-point format based on IEEE754 single-precision + ONNX_TENSOR_ELEMENT_DATA_TYPE_FLOAT8E4M3FNUZ, // Non-IEEE floating-point format based on IEEE754 single-precision + ONNX_TENSOR_ELEMENT_DATA_TYPE_FLOAT8E5M2, // Non-IEEE floating-point format based on IEEE754 single-precision + ONNX_TENSOR_ELEMENT_DATA_TYPE_FLOAT8E5M2FNUZ, // Non-IEEE floating-point format based on IEEE754 single-precision + // Int4 types were introduced in ONNX 1.16. See https://onnx.ai/onnx/technical/int4.html + ONNX_TENSOR_ELEMENT_DATA_TYPE_UINT4, // maps to a pair of packed uint4 values (size == 1 byte) + ONNX_TENSOR_ELEMENT_DATA_TYPE_INT4 // maps to a pair of packed int4 values (size == 1 byte) +} ONNXTensorElementDataType; + +// Synced with onnx TypeProto oneof +typedef enum ONNXType { + ONNX_TYPE_UNKNOWN, + ONNX_TYPE_TENSOR, + ONNX_TYPE_SEQUENCE, + ONNX_TYPE_MAP, + ONNX_TYPE_OPAQUE, + ONNX_TYPE_SPARSETENSOR, + ONNX_TYPE_OPTIONAL +} ONNXType; + +// These types are synced with internal +// SparseFormatFlags +typedef enum OrtSparseFormat { + ORT_SPARSE_UNDEFINED = 0, + ORT_SPARSE_COO = 0x1, + ORT_SPARSE_CSRC = 0x2, + ORT_SPARSE_BLOCK_SPARSE = 0x4 +} OrtSparseFormat; + +// Enum allows to query sparse tensor indices +enum OrtSparseIndicesFormat { + ORT_SPARSE_COO_INDICES, + ORT_SPARSE_CSR_INNER_INDICES, + ORT_SPARSE_CSR_OUTER_INDICES, + ORT_SPARSE_BLOCK_SPARSE_INDICES +}; + +/** \brief Logging severity levels + * + * In typical API usage, specifying a logging severity level specifies the minimum severity of log messages to show. + */ +typedef enum OrtLoggingLevel { + ORT_LOGGING_LEVEL_VERBOSE, ///< Verbose informational messages (least severe). + ORT_LOGGING_LEVEL_INFO, ///< Informational messages. + ORT_LOGGING_LEVEL_WARNING, ///< Warning messages. + ORT_LOGGING_LEVEL_ERROR, ///< Error messages. + ORT_LOGGING_LEVEL_FATAL, ///< Fatal error messages (most severe). +} OrtLoggingLevel; + +typedef enum OrtErrorCode { + ORT_OK, + ORT_FAIL, + ORT_INVALID_ARGUMENT, + ORT_NO_SUCHFILE, + ORT_NO_MODEL, + ORT_ENGINE_ERROR, + ORT_RUNTIME_EXCEPTION, + ORT_INVALID_PROTOBUF, + ORT_MODEL_LOADED, + ORT_NOT_IMPLEMENTED, + ORT_INVALID_GRAPH, + ORT_EP_FAIL, + ORT_MODEL_LOAD_CANCELED, + ORT_MODEL_REQUIRES_COMPILATION, +} OrtErrorCode; + +typedef enum OrtOpAttrType { + ORT_OP_ATTR_UNDEFINED = 0, + ORT_OP_ATTR_INT, + ORT_OP_ATTR_INTS, + ORT_OP_ATTR_FLOAT, + ORT_OP_ATTR_FLOATS, + ORT_OP_ATTR_STRING, + ORT_OP_ATTR_STRINGS, +} OrtOpAttrType; + +//! @} +#define ORT_RUNTIME_CLASS(X) \ + struct Ort##X; \ + typedef struct Ort##X Ort##X + +/** \addtogroup Global + * ONNX Runtime C API + * @{ + */ +// The actual types defined have an Ort prefix +ORT_RUNTIME_CLASS(Env); +ORT_RUNTIME_CLASS(Status); // nullptr for Status* indicates success +ORT_RUNTIME_CLASS(MemoryInfo); +ORT_RUNTIME_CLASS(IoBinding); +ORT_RUNTIME_CLASS(Session); // Don't call ReleaseSession from Dllmain (because session owns a thread pool) +ORT_RUNTIME_CLASS(Value); +ORT_RUNTIME_CLASS(RunOptions); +ORT_RUNTIME_CLASS(TypeInfo); +ORT_RUNTIME_CLASS(TensorTypeAndShapeInfo); +ORT_RUNTIME_CLASS(MapTypeInfo); +ORT_RUNTIME_CLASS(SequenceTypeInfo); +ORT_RUNTIME_CLASS(OptionalTypeInfo); +ORT_RUNTIME_CLASS(SessionOptions); +ORT_RUNTIME_CLASS(CustomOpDomain); +ORT_RUNTIME_CLASS(ModelMetadata); +ORT_RUNTIME_CLASS(ThreadPoolParams); +ORT_RUNTIME_CLASS(ThreadingOptions); +ORT_RUNTIME_CLASS(ArenaCfg); +ORT_RUNTIME_CLASS(PrepackedWeightsContainer); +ORT_RUNTIME_CLASS(TensorRTProviderOptionsV2); +ORT_RUNTIME_CLASS(NvTensorRtRtxProviderOptions); +ORT_RUNTIME_CLASS(CUDAProviderOptionsV2); +ORT_RUNTIME_CLASS(CANNProviderOptions); +ORT_RUNTIME_CLASS(DnnlProviderOptions); +ORT_RUNTIME_CLASS(Op); +ORT_RUNTIME_CLASS(OpAttr); +ORT_RUNTIME_CLASS(Logger); +ORT_RUNTIME_CLASS(ShapeInferContext); +ORT_RUNTIME_CLASS(LoraAdapter); +ORT_RUNTIME_CLASS(ValueInfo); +ORT_RUNTIME_CLASS(Node); +ORT_RUNTIME_CLASS(Graph); +ORT_RUNTIME_CLASS(Model); +ORT_RUNTIME_CLASS(ModelCompilationOptions); +ORT_RUNTIME_CLASS(HardwareDevice); +ORT_RUNTIME_CLASS(EpDevice); +ORT_RUNTIME_CLASS(KeyValuePairs); + +#ifdef _MSC_VER +typedef _Return_type_success_(return == 0) OrtStatus* OrtStatusPtr; +#else +typedef OrtStatus* OrtStatusPtr; +#endif + +/** \brief Memory allocation interface + * + * Structure of function pointers that defines a memory allocator. This can be created and filled in by the user for custom allocators. + * + * When an allocator is passed to any function, be sure that the allocator object is not destroyed until the last allocated object using it is freed. + */ +typedef struct OrtAllocator { + uint32_t version; ///< Must be initialized to ORT_API_VERSION + void*(ORT_API_CALL* Alloc)(struct OrtAllocator* this_, size_t size); ///< Returns a pointer to an allocated block of `size` bytes + void(ORT_API_CALL* Free)(struct OrtAllocator* this_, void* p); ///< Free a block of memory previously allocated with OrtAllocator::Alloc + const struct OrtMemoryInfo*(ORT_API_CALL* Info)(const struct OrtAllocator* this_); ///< Return a pointer to an ::OrtMemoryInfo that describes this allocator + /** + * @brief Optional allocation function to use for memory allocations made during session initialization. + * Use this function if you want to separate allocations made by ORT during Run() calls from + * those made during session initialization. This allows for separate memory management strategies for these allocations. + */ + void*(ORT_API_CALL* Reserve)(struct OrtAllocator* this_, size_t size); ///< Returns a pointer to an allocated block of `size` bytes +} OrtAllocator; + +typedef void(ORT_API_CALL* OrtLoggingFunction)( + void* param, OrtLoggingLevel severity, const char* category, const char* logid, const char* code_location, + const char* message); + +/** \brief Graph optimization level + * + * Refer to https://www.onnxruntime.ai/docs/performance/graph-optimizations.html#graph-optimization-levels + * for an in-depth understanding of the Graph Optimization Levels. + */ +typedef enum GraphOptimizationLevel { + ORT_DISABLE_ALL = 0, + ORT_ENABLE_BASIC = 1, + ORT_ENABLE_EXTENDED = 2, + ORT_ENABLE_ALL = 99 +} GraphOptimizationLevel; + +typedef enum ExecutionMode { + ORT_SEQUENTIAL = 0, + ORT_PARALLEL = 1, +} ExecutionMode; + +/** \brief Language projection identifiers + * /see OrtApi::SetLanguageProjection + */ +typedef enum OrtLanguageProjection { + ORT_PROJECTION_C = 0, + ORT_PROJECTION_CPLUSPLUS = 1, + ORT_PROJECTION_CSHARP = 2, + ORT_PROJECTION_PYTHON = 3, + ORT_PROJECTION_JAVA = 4, + ORT_PROJECTION_WINML = 5, + ORT_PROJECTION_NODEJS = 6, +} OrtLanguageProjection; + +struct OrtKernelInfo; +typedef struct OrtKernelInfo OrtKernelInfo; +struct OrtKernelContext; +typedef struct OrtKernelContext OrtKernelContext; +struct OrtCustomOp; +typedef struct OrtCustomOp OrtCustomOp; + +typedef enum OrtAllocatorType { + OrtInvalidAllocator = -1, + OrtDeviceAllocator = 0, + OrtArenaAllocator = 1 +} OrtAllocatorType; + +/** \brief Memory types for allocated memory, execution provider specific types should be extended in each provider. + */ +// Whenever this struct is updated, please also update the MakeKey function in onnxruntime / core / framework / execution_provider.cc +typedef enum OrtMemType { + OrtMemTypeCPUInput = -2, ///< Any CPU memory used by non-CPU execution provider + OrtMemTypeCPUOutput = -1, ///< CPU accessible memory outputted by non-CPU execution provider, i.e. CUDA_PINNED + OrtMemTypeCPU = OrtMemTypeCPUOutput, ///< Temporary CPU accessible memory allocated by non-CPU execution provider, i.e. CUDA_PINNED + OrtMemTypeDefault = 0, ///< The default allocator for execution provider +} OrtMemType; + +/** \brief This mimics OrtDevice type constants so they can be returned in the API + */ +typedef enum OrtMemoryInfoDeviceType { + OrtMemoryInfoDeviceType_CPU = 0, + OrtMemoryInfoDeviceType_GPU = 1, + OrtMemoryInfoDeviceType_FPGA = 2 +} OrtMemoryInfoDeviceType; + +typedef enum OrtHardwareDeviceType { + OrtHardwareDeviceType_CPU, + OrtHardwareDeviceType_GPU, + OrtHardwareDeviceType_NPU +} OrtHardwareDeviceType; + +/** \brief These are the default EP selection policies used by ORT when doing automatic EP selection. + */ +typedef enum OrtExecutionProviderDevicePolicy { + OrtExecutionProviderDevicePolicy_DEFAULT, + OrtExecutionProviderDevicePolicy_PREFER_CPU, + OrtExecutionProviderDevicePolicy_PREFER_NPU, + OrtExecutionProviderDevicePolicy_PREFER_GPU, + OrtExecutionProviderDevicePolicy_MAX_PERFORMANCE, + OrtExecutionProviderDevicePolicy_MAX_EFFICIENCY, + OrtExecutionProviderDevicePolicy_MIN_OVERALL_POWER, +} OrtExecutionProviderDevicePolicy; + +/** \brief Delegate to allow providing custom OrtEpDevice selection logic + * + * This delegate is called by the EP selection code to allow the user to provide custom device selection logic. + * The user can use this to select OrtEpDevice instances from the list of available devices. + * + * \param ep_devices The list of available devices. + * \param num_devices The number of available devices. + * \param model_metadata The model metadata. + * \param runtime_metadata The runtime metadata. May be nullptr. + * \param selected Pre-allocated array to populate with selected OrtEpDevice pointers from ep_devices. + * \param max_ep_devices The maximum number of devices that can be selected in the pre-allocated array. + Currently the maximum is 8. + * \param num_ep_devices The number of selected devices. + * \param state Opaque pointer. Required to use the delegate from other languages like C# and python. + * + * \return OrtStatus* Selection status. Return nullptr on success. + * Use CreateStatus to provide error info. Use ORT_FAIL as the error code. + * ORT will release the OrtStatus* if not null. + */ +typedef OrtStatus* (*EpSelectionDelegate)(_In_ const OrtEpDevice** ep_devices, + _In_ size_t num_devices, + _In_ const OrtKeyValuePairs* model_metadata, + _In_opt_ const OrtKeyValuePairs* runtime_metadata, + _Inout_ const OrtEpDevice** selected, + _In_ size_t max_selected, + _Out_ size_t* num_selected, + _In_ void* state); + +/** \brief Algorithm to use for cuDNN Convolution Op + */ +typedef enum OrtCudnnConvAlgoSearch { + OrtCudnnConvAlgoSearchExhaustive, // expensive exhaustive benchmarking using cudnnFindConvolutionForwardAlgorithmEx + OrtCudnnConvAlgoSearchHeuristic, // lightweight heuristic based search using cudnnGetConvolutionForwardAlgorithm_v7 + OrtCudnnConvAlgoSearchDefault, // default algorithm using CUDNN_CONVOLUTION_FWD_ALGO_IMPLICIT_PRECOMP_GEMM +} OrtCudnnConvAlgoSearch; + +/** \brief CUDA Provider Options + * + * \see OrtApi::SessionOptionsAppendExecutionProvider_CUDA + */ +typedef struct OrtCUDAProviderOptions { +#ifdef __cplusplus + OrtCUDAProviderOptions() + : device_id{}, + cudnn_conv_algo_search{OrtCudnnConvAlgoSearchExhaustive}, + gpu_mem_limit{SIZE_MAX}, + arena_extend_strategy{}, + do_copy_in_default_stream{1}, + has_user_compute_stream{}, + user_compute_stream{}, + default_memory_arena_cfg{}, + tunable_op_enable{false}, + tunable_op_tuning_enable{false}, + tunable_op_max_tuning_duration_ms{} {} +#endif + + /** \brief CUDA device Id + * Defaults to 0. + */ + int device_id; + + /** \brief CUDA Convolution algorithm search configuration. + * See enum OrtCudnnConvAlgoSearch for more details. + * Defaults to OrtCudnnConvAlgoSearchExhaustive. + */ + OrtCudnnConvAlgoSearch cudnn_conv_algo_search; + + /** \brief CUDA memory limit (To use all possible memory pass in maximum size_t) + * Defaults to SIZE_MAX. + * \note If a ::OrtArenaCfg has been applied, it will override this field + */ + size_t gpu_mem_limit; + + /** \brief Strategy used to grow the memory arena + * 0 = kNextPowerOfTwo
+ * 1 = kSameAsRequested
+ * Defaults to 0. + * \note If a ::OrtArenaCfg has been applied, it will override this field + */ + int arena_extend_strategy; + + /** \brief Flag indicating if copying needs to take place on the same stream as the compute stream in the CUDA EP + * 0 = Use separate streams for copying and compute. + * 1 = Use the same stream for copying and compute. + * Defaults to 1. + * WARNING: Setting this to 0 may result in data races for some models. + * Please see issue #4829 for more details. + */ + int do_copy_in_default_stream; + + /** \brief Flag indicating if there is a user provided compute stream + * Defaults to 0. + */ + int has_user_compute_stream; + + /** \brief User provided compute stream. + * If provided, please set `has_user_compute_stream` to 1. + */ + void* user_compute_stream; + + /** \brief CUDA memory arena configuration parameters + */ + OrtArenaCfg* default_memory_arena_cfg; + + /** \brief Enable TunableOp for using. + * Set it to 1/0 to enable/disable TunableOp. Otherwise, it is disabled by default. + * This option can be overridden by environment variable ORT_CUDA_TUNABLE_OP_ENABLE. + */ + int tunable_op_enable; + + /** \brief Enable TunableOp for tuning. + * Set it to 1/0 to enable/disable TunableOp tuning. Otherwise, it is disabled by default. + * This option can be overridden by environment variable ORT_CUDA_TUNABLE_OP_TUNING_ENABLE. + */ + int tunable_op_tuning_enable; + + /** \brief Max tuning duration time limit for each instance of TunableOp. + * Defaults to 0 to disable the limit. + */ + int tunable_op_max_tuning_duration_ms; + +} OrtCUDAProviderOptions; + +/** \brief ROCM Provider Options + * + * \see OrtApi::SessionOptionsAppendExecutionProvider_ROCM + */ +typedef struct OrtROCMProviderOptions { +#ifdef __cplusplus + OrtROCMProviderOptions() + : device_id{}, + miopen_conv_exhaustive_search{0}, + gpu_mem_limit{SIZE_MAX}, + arena_extend_strategy{}, + do_copy_in_default_stream{1}, + has_user_compute_stream{}, + user_compute_stream{}, + default_memory_arena_cfg{}, + enable_hip_graph{false}, + tunable_op_enable{false}, + tunable_op_tuning_enable{false}, + tunable_op_max_tuning_duration_ms{} {} +#endif + + /** \brief ROCM device Id + * Defaults to 0. + */ + int device_id; + + /** \brief ROCM MIOpen Convolution algorithm exaustive search option. + * Defaults to 0 (false). + */ + int miopen_conv_exhaustive_search; + + /** \brief ROCM memory limit (To use all possible memory pass in maximum size_t) + * Defaults to SIZE_MAX. + * \note If a ::OrtArenaCfg has been applied, it will override this field + */ + size_t gpu_mem_limit; + + /** \brief Strategy used to grow the memory arena + * 0 = kNextPowerOfTwo
+ * 1 = kSameAsRequested
+ * Defaults to 0. + * \note If a ::OrtArenaCfg has been applied, it will override this field + */ + int arena_extend_strategy; + + /** \brief Flag indicating if copying needs to take place on the same stream as the compute stream in the ROCM EP + * 0 = Use separate streams for copying and compute. + * 1 = Use the same stream for copying and compute. + * Defaults to 1. + * WARNING: Setting this to 0 may result in data races for some models. + * Please see issue #4829 for more details. + */ + int do_copy_in_default_stream; + + /** \brief Flag indicating if there is a user provided compute stream + * Defaults to 0. + */ + int has_user_compute_stream; + + /** \brief User provided compute stream. + * If provided, please set `has_user_compute_stream` to 1. + */ + void* user_compute_stream; + + /** \brief ROCM memory arena configuration parameters + */ + OrtArenaCfg* default_memory_arena_cfg; + + int enable_hip_graph; + + /** \brief Enable TunableOp for using. + * Set it to 1/0 to enable/disable TunableOp. Otherwise, it is disabled by default. + * This option can be overridden by environment variable ORT_ROCM_TUNABLE_OP_ENABLE. + */ + int tunable_op_enable; + + /** \brief Enable TunableOp for tuning. + * Set it to 1/0 to enable/disable TunableOp tuning. Otherwise, it is disabled by default. + * This option can be overridden by environment variable ORT_ROCM_TUNABLE_OP_TUNING_ENABLE. + */ + int tunable_op_tuning_enable; + + /** \brief Max tuning duration time limit for each instance of TunableOp. + * Defaults to 0 to disable the limit. + */ + int tunable_op_max_tuning_duration_ms; + +} OrtROCMProviderOptions; + +/** \brief TensorRT Provider Options + * + * \see OrtApi::SessionOptionsAppendExecutionProvider_TensorRT + */ +typedef struct OrtTensorRTProviderOptions { + int device_id; ///< CUDA device id (0 = default device) + int has_user_compute_stream; // indicator of user specified CUDA compute stream. + void* user_compute_stream; // user specified CUDA compute stream. + int trt_max_partition_iterations; // maximum iterations for TensorRT parser to get capability + int trt_min_subgraph_size; // minimum size of TensorRT subgraphs + size_t trt_max_workspace_size; // maximum workspace size for TensorRT. + int trt_fp16_enable; // enable TensorRT FP16 precision. Default 0 = false, nonzero = true + int trt_int8_enable; // enable TensorRT INT8 precision. Default 0 = false, nonzero = true + const char* trt_int8_calibration_table_name; // TensorRT INT8 calibration table name. + int trt_int8_use_native_calibration_table; // use native TensorRT generated calibration table. Default 0 = false, nonzero = true + int trt_dla_enable; // enable DLA. Default 0 = false, nonzero = true + int trt_dla_core; // DLA core number. Default 0 + int trt_dump_subgraphs; // dump TRT subgraph. Default 0 = false, nonzero = true + int trt_engine_cache_enable; // enable engine caching. Default 0 = false, nonzero = true + const char* trt_engine_cache_path; // specify engine cache path + int trt_engine_decryption_enable; // enable engine decryption. Default 0 = false, nonzero = true + const char* trt_engine_decryption_lib_path; // specify engine decryption library path + int trt_force_sequential_engine_build; // force building TensorRT engine sequentially. Default 0 = false, nonzero = true + // This is the legacy struct and don't add new fields here. + // For new field that can be represented by string, please add it in include/onnxruntime/core/providers/tensorrt/tensorrt_provider_options.h + // For non-string field, need to create a new separate api to handle it. +} OrtTensorRTProviderOptions; + +/** \brief MIGraphX Provider Options + * + * \see OrtApi::SessionOptionsAppendExecutionProvider_MIGraphX + */ +typedef struct OrtMIGraphXProviderOptions { + int device_id; // hip device id. + int migraphx_fp16_enable; // MIGraphX FP16 precision. Default 0 = false, nonzero = true + int migraphx_int8_enable; // MIGraphX INT8 precision. Default 0 = false, nonzero = true + int migraphx_use_native_calibration_table; // MIGraphx INT8 cal table. Default 0 = false, noznero = true + const char* migraphx_int8_calibration_table_name; // MIGraphx INT8 calibration table name + int migraphx_save_compiled_model; // migraphx save compiled model. Default 0 = false, noznero = true + const char* migraphx_save_model_path; // migraphx model path name + int migraphx_load_compiled_model; // migraphx int8 cal table. Default 0 = false, noznero = true + const char* migraphx_load_model_path; // migraphx model path name + bool migraphx_exhaustive_tune; // migraphx tuned compile Default = false +} OrtMIGraphXProviderOptions; + +/** \brief OpenVINO Provider Options + * \brief This Struct is frozen since ORT 1.13.0. Its maintained part of Legacy API for compatibility. + * \brief For latest OpenVINO Provider Options update to the ProviderOptions map. + * \brief Latest OpenVINO Provider Options are listed in the + * \htmlonly + * onnxruntime document. + * \endhtmlonly + * \see OrtApi::SessionOptionsAppendExecutionProvider() + */ +typedef struct OrtOpenVINOProviderOptions { +#ifdef __cplusplus + OrtOpenVINOProviderOptions() : device_type{}, + enable_npu_fast_compile{}, + device_id{}, + num_of_threads{}, + cache_dir{}, + context{}, + enable_opencl_throttling{}, + enable_dynamic_shapes{} {} +#endif + /** \brief Device type string + * + * Valid settings are one of: "CPU_FP32", "CPU_FP16", "GPU_FP32", "GPU_FP16" + */ + const char* device_type; + unsigned char enable_npu_fast_compile; ///< 0 = disabled, nonzero = enabled + const char* device_id; + size_t num_of_threads; ///< 0 = Use default number of threads + const char* cache_dir; // path is set to empty by default + void* context; + unsigned char enable_opencl_throttling; ///< 0 = disabled, nonzero = enabled + unsigned char enable_dynamic_shapes; ///< 0 = disabled, nonzero = enabled +} OrtOpenVINOProviderOptions; + +struct OrtApi; +typedef struct OrtApi OrtApi; + +struct OrtTrainingApi; +typedef struct OrtTrainingApi OrtTrainingApi; + +struct OrtModelEditorApi; +typedef struct OrtModelEditorApi OrtModelEditorApi; + +struct OrtCompileApi; +typedef struct OrtCompileApi OrtCompileApi; + +struct OrtEpApi; +typedef struct OrtEpApi OrtEpApi; + +/** \brief The helper interface to get the right version of OrtApi + * + * Get a pointer to this structure through ::OrtGetApiBase + */ +struct OrtApiBase { + /** \brief Get a pointer to the requested version of the ::OrtApi + * + * \param[in] version Must be ::ORT_API_VERSION + * \return The ::OrtApi for the version requested, nullptr will be returned if this version is unsupported, for example when using a runtime + * older than the version created with this header file. + * + * One can call GetVersionString() to get the version of the Onnxruntime library for logging + * and error reporting purposes. + */ + const OrtApi*(ORT_API_CALL* GetApi)(uint32_t version)NO_EXCEPTION; + + /** \brief Returns a null terminated string of the version of the Onnxruntime library (eg: "1.8.1") + * + * \return UTF-8 encoded version string. Do not deallocate the returned buffer. + */ + const char*(ORT_API_CALL* GetVersionString)(void)NO_EXCEPTION; +}; + +typedef struct OrtApiBase OrtApiBase; + +/** \brief The Onnxruntime library's entry point to access the C API + * + * Call this to get the a pointer to an ::OrtApiBase + */ +ORT_EXPORT const OrtApiBase* ORT_API_CALL OrtGetApiBase(void) NO_EXCEPTION; + +/** \brief Thread work loop function + * + * Onnxruntime will provide the working loop on custom thread creation + * Argument is an onnxruntime built-in type which will be provided when thread pool calls OrtCustomCreateThreadFn + */ +typedef void (*OrtThreadWorkerFn)(void* ort_worker_fn_param); + +typedef const struct OrtCustomHandleType { + char __place_holder; +}* OrtCustomThreadHandle; + +/** \brief Ort custom thread creation function + * + * The function should return a thread handle to be used in onnxruntime thread pools + * Onnxruntime will throw exception on return value of nullptr or 0, indicating that the function failed to create a thread + */ +typedef OrtCustomThreadHandle (*OrtCustomCreateThreadFn)(void* ort_custom_thread_creation_options, OrtThreadWorkerFn ort_thread_worker_fn, void* ort_worker_fn_param); + +/** \brief Custom thread join function + * + * Onnxruntime thread pool destructor will call the function to join a custom thread. + * Argument ort_custom_thread_handle is the value returned by OrtCustomCreateThreadFn + */ +typedef void (*OrtCustomJoinThreadFn)(OrtCustomThreadHandle ort_custom_thread_handle); + +typedef OrtStatus*(ORT_API_CALL* RegisterCustomOpsFn)(OrtSessionOptions* options, const OrtApiBase* api); + +/** \brief Callback function for RunAsync + * + * \param[in] user_data User specific data that passed back to the callback + * \param[out] outputs On succeed, outputs host inference results, on error, the value will be nullptr + * \param[out] num_outputs Number of outputs, on error, the value will be zero + * \param[out] status On error, status will provide details + */ +typedef void (*RunAsyncCallbackFn)(void* user_data, OrtValue** outputs, size_t num_outputs, OrtStatusPtr status); + +/** \brief The C API + * + * All C API functions are defined inside this structure as pointers to functions. + * Call OrtApiBase::GetApi to get a pointer to it + * + * \nosubgrouping + */ +struct OrtApi { + /// \name OrtStatus + /// @{ + + /** + * \brief Create an OrtStatus from a null terminated string + * + * \param[in] code + * \param[in] msg A null-terminated string. Its contents will be copied. + * \return A new OrtStatus object, must be destroyed with OrtApi::ReleaseStatus + */ + OrtStatus*(ORT_API_CALL* CreateStatus)(OrtErrorCode code, _In_ const char* msg)NO_EXCEPTION ORT_ALL_ARGS_NONNULL; + + /** \brief Get OrtErrorCode from OrtStatus + * + * \param[in] status + * \return OrtErrorCode that \p status was created with + */ + OrtErrorCode(ORT_API_CALL* GetErrorCode)(_In_ const OrtStatus* status) NO_EXCEPTION ORT_ALL_ARGS_NONNULL; + + /** \brief Get error string from OrtStatus + * + * \param[in] status + * \return The error message inside the `status`. Do not free the returned value. + */ + const char*(ORT_API_CALL* GetErrorMessage)(_In_ const OrtStatus* status)NO_EXCEPTION ORT_ALL_ARGS_NONNULL; + + /// @} + /// \name OrtEnv + /// @{ + + /** \brief Create an OrtEnv + * + * \note Invoking this function will return the same instance of the environment as that returned by a previous call + * to another env creation function; all arguments to this function will be ignored. + * \param[in] log_severity_level The log severity level. + * \param[in] logid The log identifier. + * \param[out] out Returned newly created OrtEnv. Must be freed with OrtApi::ReleaseEnv + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(CreateEnv, OrtLoggingLevel log_severity_level, _In_ const char* logid, _Outptr_ OrtEnv** out); + + /** \brief Create an OrtEnv + * + * \note Invoking this function will return the same instance of the environment as that returned by a previous call + * to another env creation function; all arguments to this function will be ignored. If you want to provide your + * own logging function, consider setting it using the SetUserLoggingFunction API instead. + * \param[in] logging_function A pointer to a logging function. + * \param[in] logger_param A pointer to arbitrary data passed as the ::OrtLoggingFunction `param` parameter to + * `logging_function`. This parameter is optional. + * \param[in] log_severity_level The log severity level. + * \param[in] logid The log identifier. + * \param[out] out Returned newly created OrtEnv. Must be freed with OrtApi::ReleaseEnv + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(CreateEnvWithCustomLogger, _In_ OrtLoggingFunction logging_function, _In_opt_ void* logger_param, + _In_ OrtLoggingLevel log_severity_level, _In_ const char* logid, _Outptr_ OrtEnv** out); + + /** \brief Enable Telemetry + * + * \note Telemetry events are on by default since they are lightweight + * \param[in] env + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(EnableTelemetryEvents, _In_ const OrtEnv* env); + /** \brief Disable Telemetry + * + * \see OrtApi::EnableTelemetryEvents + * \param[in] env + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(DisableTelemetryEvents, _In_ const OrtEnv* env); + + /// @} + /// \name OrtSession + /// @{ + + /** \brief Create an OrtSession from a model file + * + * \param[in] env + * \param[in] model_path + * \param[in] options + * \param[out] out Returned newly created OrtSession. Must be freed with OrtApi::ReleaseSession + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + // TODO: document the path separator convention? '/' vs '\' + // TODO: should specify the access characteristics of model_path. Is this read only during the + // execution of CreateSession, or does the OrtSession retain a handle to the file/directory + // and continue to access throughout the OrtSession lifetime? + // What sort of access is needed to model_path : read or read/write? + ORT_API2_STATUS(CreateSession, _In_ const OrtEnv* env, _In_ const ORTCHAR_T* model_path, + _In_ const OrtSessionOptions* options, _Outptr_ OrtSession** out); + + /** \brief Create an OrtSession from memory + * + * \param[in] env + * \param[in] model_data + * \param[in] model_data_length + * \param[in] options + * \param[out] out Returned newly created OrtSession. Must be freed with OrtApi::ReleaseSession + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(CreateSessionFromArray, _In_ const OrtEnv* env, + _In_ const void* model_data, size_t model_data_length, + _In_ const OrtSessionOptions* options, _Outptr_ OrtSession** out); + + /** \brief Run the model in an ::OrtSession + * + * Will not return until the model run has completed. Multiple threads might be used to run the model based on + * the options in the ::OrtSession and settings used when creating the ::OrtEnv + * + * \param[in] session + * \param[in] run_options If nullptr, will use a default ::OrtRunOptions + * \param[in] input_names Array of null terminated UTF8 encoded strings of the input names + * \param[in] inputs Array of ::OrtValue%s of the input values + * \param[in] input_len Number of elements in the input_names and inputs arrays + * \param[in] output_names Array of null terminated UTF8 encoded strings of the output names + * \param[in] output_names_len Number of elements in the output_names and outputs array + * \param[out] outputs Array of ::OrtValue%s that the outputs are stored in. This can also be + * an array of nullptr values, in this case ::OrtValue objects will be allocated and pointers + * to them will be set into the `outputs` array. + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(Run, _Inout_ OrtSession* session, _In_opt_ const OrtRunOptions* run_options, + _In_reads_(input_len) const char* const* input_names, + _In_reads_(input_len) const OrtValue* const* inputs, size_t input_len, + _In_reads_(output_names_len) const char* const* output_names, size_t output_names_len, + _Inout_updates_all_(output_names_len) OrtValue** outputs); + + /// @} + /// \name OrtSessionOptions + /// @{ + + /** \brief Create an ::OrtSessionOptions object + * + * To use additional providers, you must build ORT with the extra providers enabled. Then call one of these + * functions to enable them in the session:
+ * OrtSessionOptionsAppendExecutionProvider_CPU
+ * OrtSessionOptionsAppendExecutionProvider_CUDA
+ * OrtSessionOptionsAppendExecutionProvider_(remaining providers...)
+ * The order they are called indicates the preference order as well. In other words call this method + * on your most preferred execution provider first followed by the less preferred ones. + * If none are called Ort will use its internal CPU execution provider. + * + * \param[out] options The newly created OrtSessionOptions. Must be freed with OrtApi::ReleaseSessionOptions + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(CreateSessionOptions, _Outptr_ OrtSessionOptions** options); + + /** \brief Set filepath to save optimized model after graph level transformations + * + * \param[in] options + * \param[in] optimized_model_filepath + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(SetOptimizedModelFilePath, _Inout_ OrtSessionOptions* options, + _In_ const ORTCHAR_T* optimized_model_filepath); + + /** \brief Create a copy of an existing ::OrtSessionOptions + * + * \param[in] in_options OrtSessionOptions to copy + * \param[out] out_options Returned newly created ::OrtSessionOptions. Must be freed with OrtApi::ReleaseSessionOptions + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(CloneSessionOptions, _In_ const OrtSessionOptions* in_options, + _Outptr_ OrtSessionOptions** out_options); + + /** \brief Set execution mode + * + * Controls whether you want to execute operators in your graph sequentially or in parallel. Usually when the model + * has many branches, setting this option to ExecutionMode.ORT_PARALLEL will give you better performance. + * See [docs/ONNX_Runtime_Perf_Tuning.md] for more details. + * + * \param[in] options + * \param[in] execution_mode + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(SetSessionExecutionMode, _Inout_ OrtSessionOptions* options, ExecutionMode execution_mode); + + /** \brief Enable profiling for a session + * + * \param[in] options + * \param[in] profile_file_prefix + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(EnableProfiling, _Inout_ OrtSessionOptions* options, _In_ const ORTCHAR_T* profile_file_prefix); + + /** \brief Disable profiling for a session + * + * \param[in] options + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(DisableProfiling, _Inout_ OrtSessionOptions* options); + + /** \brief Enable the memory pattern optimization + * + * The idea is if the input shapes are the same, we could trace the internal memory allocation + * and generate a memory pattern for future request. So next time we could just do one allocation + * with a big chunk for all the internal memory allocation. + * \note Memory pattern optimization is only available when Sequential Execution mode is enabled (see OrtApi::SetSessionExecutionMode) + * + * \see OrtApi::DisableMemPattern + * + * \param[in] options + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(EnableMemPattern, _Inout_ OrtSessionOptions* options); + + /** \brief Disable the memory pattern optimization + * + * \see OrtApi::EnableMemPattern + * + * \param[in] options + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(DisableMemPattern, _Inout_ OrtSessionOptions* options); + + /** \brief Enable the memory arena on CPU + * + * Arena may pre-allocate memory for future usage. + * + * \param[in] options + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(EnableCpuMemArena, _Inout_ OrtSessionOptions* options); + + /** \brief Disable the memory arena on CPU + * + * \param[in] options + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(DisableCpuMemArena, _Inout_ OrtSessionOptions* options); + + /** \brief Set session log id + * + * \param[in] options + * \param[in] logid The log identifier. + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(SetSessionLogId, _Inout_ OrtSessionOptions* options, const char* logid); + + /** \brief Set session log verbosity level + * + * Applies to session load, initialization, etc + * + * \param[in] options + * \param[in] session_log_verbosity_level \snippet{doc} snippets.dox Log Verbosity Level + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(SetSessionLogVerbosityLevel, _Inout_ OrtSessionOptions* options, int session_log_verbosity_level); + + /** \brief Set session log severity level + * + * \param[in] options + * \param[in] session_log_severity_level The log severity level (refer to ::OrtLoggingLevel for possible values). + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(SetSessionLogSeverityLevel, _Inout_ OrtSessionOptions* options, int session_log_severity_level); + + /** \brief Set the optimization level to apply when loading a graph + * + * Please see https://onnxruntime.ai/docs/performance/model-optimizations/graph-optimizations.html for an in-depth explanation + * \param[in,out] options The session options object + * \param[in] graph_optimization_level The optimization level + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(SetSessionGraphOptimizationLevel, _Inout_ OrtSessionOptions* options, + GraphOptimizationLevel graph_optimization_level); + + /** \brief Sets the number of threads used to parallelize the execution within nodes + * + * When running a single node operation, ex. add, this sets the maximum number of threads to use. + * + * \note If built with OpenMP, this has no effect on the number of threads used. In this case + * use the OpenMP env variables to configure the number of intra op num threads. + * + * \param[in] options + * \param[in] intra_op_num_threads Number of threads to use
+ * A value of 0 will use the default number of threads
+ * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(SetIntraOpNumThreads, _Inout_ OrtSessionOptions* options, int intra_op_num_threads); + + /** \brief Sets the number of threads used to parallelize the execution of the graph + * + * If nodes can be run in parallel, this sets the maximum number of threads to use to run them in parallel. + * + * \note If sequential execution is enabled this value is ignored, it acts as if it was set to 1. + * + * \param[in] options + * \param[in] inter_op_num_threads Number of threads to use
+ * A value of 0 will use the default number of threads
+ * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(SetInterOpNumThreads, _Inout_ OrtSessionOptions* options, int inter_op_num_threads); + + /// @} + /// \name OrtCustomOpDomain + /// @{ + + /** \brief Create a custom op domain + * + * \param[in] domain + * \param[out] out Newly created domain. Must be freed with OrtApi::ReleaseCustomOpDomain + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(CreateCustomOpDomain, _In_ const char* domain, _Outptr_ OrtCustomOpDomain** out); + + /** \brief Add a custom op to a custom op domain + * + * \note The OrtCustomOp* pointer must remain valid until the ::OrtCustomOpDomain using it is released + * + * \param[in] custom_op_domain + * \param[in] op + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(CustomOpDomain_Add, _Inout_ OrtCustomOpDomain* custom_op_domain, _In_ const OrtCustomOp* op); + + /// @} + /// \name OrtSessionOptions + /// @{ + + /** \brief Add custom op domain to a session options + * + * \note The OrtCustomOpDomain* must not be deleted until all sessions using it are released + * + * \param[in] options + * \param[in] custom_op_domain + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(AddCustomOpDomain, _Inout_ OrtSessionOptions* options, _In_ OrtCustomOpDomain* custom_op_domain); + + /** \deprecated Use OrtApi::RegisterCustomOpsLibrary_V2. + * + * Registers custom ops from a shared library. + * + * Loads a shared library (dll on windows, so on linux, etc) named 'library_path' and looks for this entry point: + * OrtStatus* RegisterCustomOps(OrtSessionOptions * options, const OrtApiBase* api); + * It then passes in the provided session options to this function along with the api base. + * The handle to the loaded library is returned in library_handle. It can be freed by the caller after all sessions using the passed in + * session options are destroyed, or if an error occurs and it is non null. + * + * \param[in] options + * \param[in] library_path + * \param[out] library_handle OS specific handle to the loaded library (Use FreeLibrary on Windows, dlclose on Linux, etc.. to unload) + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(RegisterCustomOpsLibrary, _Inout_ OrtSessionOptions* options, _In_ const char* library_path, _Outptr_ void** library_handle); + + /// @} + /// \name OrtSession + /// @{ + + /** \brief Get input count for a session + * + * This number must also match the number of inputs passed to OrtApi::Run + * + * \see OrtApi::SessionGetInputTypeInfo, OrtApi::SessionGetInputName, OrtApi::Session + * + * \param[in] session + * \param[out] out Number of inputs + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(SessionGetInputCount, _In_ const OrtSession* session, _Out_ size_t* out); + + /** \brief Get output count for a session + * + * This number must also match the number of outputs returned by OrtApi::Run + * + * \see OrtApi::SessionGetOutputTypeInfo, OrtApi::SessionGetOutputName, OrtApi::Session + * + * \param[in] session + * \param[out] out Number of outputs + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(SessionGetOutputCount, _In_ const OrtSession* session, _Out_ size_t* out); + + /** \brief Get overridable initializer count + * + * \see OrtApi::SessionGetOverridableInitializerTypeInfo, OrtApi::SessionGetOverridableInitializerName + * + * \param[in] session + * \param[in] out + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(SessionGetOverridableInitializerCount, _In_ const OrtSession* session, _Out_ size_t* out); + + /** \brief Get input type information + * + * \param[in] session + * \param[in] index Must be between 0 (inclusive) and what OrtApi::SessionGetInputCount returns (exclusive) + * \param[out] type_info Must be freed with OrtApi::ReleaseTypeInfo + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(SessionGetInputTypeInfo, _In_ const OrtSession* session, size_t index, _Outptr_ OrtTypeInfo** type_info); + + /** \brief Get output type information + * + * \param[in] session + * \param[in] index Must be between 0 (inclusive) and what OrtApi::SessionGetOutputCount returns (exclusive) + * \param[out] type_info Must be freed with OrtApi::ReleaseTypeInfo + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(SessionGetOutputTypeInfo, _In_ const OrtSession* session, size_t index, _Outptr_ OrtTypeInfo** type_info); + + /** \brief Get overridable initializer type information + * + * \param[in] session + * \param[in] index Must be between 0 (inclusive) and what OrtApi::SessionGetOverridableInitializerCount returns (exclusive) + * \param[out] type_info Must be freed with OrtApi::ReleaseTypeInfo + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(SessionGetOverridableInitializerTypeInfo, _In_ const OrtSession* session, size_t index, _Outptr_ OrtTypeInfo** type_info); + + /** \brief Get input name + * + * \param[in] session + * \param[in] index Must be between 0 (inclusive) and what OrtApi::SessionGetInputCount returns (exclusive) + * \param[in] allocator + * \param[out] value Set to a null terminated UTF-8 encoded string allocated using `allocator`. Must be freed using `allocator`. + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(SessionGetInputName, _In_ const OrtSession* session, size_t index, _Inout_ OrtAllocator* allocator, _Outptr_ char** value); + + /** \brief Get output name + * + * \param[in] session + * \param[in] index Must be between 0 (inclusive) and what OrtApi::SessionGetOutputCount returns (exclusive) + * \param[in] allocator + * \param[out] value Set to a null terminated UTF-8 encoded string allocated using `allocator`. Must be freed using `allocator`. + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(SessionGetOutputName, _In_ const OrtSession* session, size_t index, _Inout_ OrtAllocator* allocator, _Outptr_ char** value); + + /** \brief Get overridable initializer name + * + * \param[in] session + * \param[in] index Must be between 0 (inclusive) and what OrtApi::SessionGetOverridableInitializerCount returns (exclusive) + * \param[in] allocator + * \param[out] value Set to a null terminated UTF-8 encoded string allocated using `allocator`. Must be freed using `allocator`. + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(SessionGetOverridableInitializerName, _In_ const OrtSession* session, size_t index, + _Inout_ OrtAllocator* allocator, _Outptr_ char** value); + + /// @} + /// \name OrtRunOptions + /// @{ + + /** \brief Create an OrtRunOptions + * + * \param[out] out Returned newly created ::OrtRunOptions. Must be freed with OrtApi::ReleaseRunOptions + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(CreateRunOptions, _Outptr_ OrtRunOptions** out); + + /** \brief Set per-run log verbosity level + * + * \see OrtApi::RunOptionsGetRunLogVerbosityLevel + * + * \param[in] options + * \param[in] log_verbosity_level \snippet{doc} snippets.dox Log Verbosity Level + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(RunOptionsSetRunLogVerbosityLevel, _Inout_ OrtRunOptions* options, int log_verbosity_level); + + /** \brief Set per-run log severity level + * + * \see OrtApi::RunOptionsGetRunLogSeverityLevel + * + * \param[in] options + * \param[in] log_severity_level The log severity level (refer to ::OrtLoggingLevel for possible values). + */ + ORT_API2_STATUS(RunOptionsSetRunLogSeverityLevel, _Inout_ OrtRunOptions* options, int log_severity_level); + + /** \brief Set per-run tag + * + * This is used in a per-run log identifier. + * + * \see OrtApi::RunOptionsGetRunTag + * + * \param[in] options + * \param[in] run_tag The run tag. + */ + ORT_API2_STATUS(RunOptionsSetRunTag, _Inout_ OrtRunOptions* options, _In_ const char* run_tag); + + /** \brief Get per-run log verbosity level + * + * \see OrtApi::RunOptionsSetRunLogVerbosityLevel + * + * \param[in] options + * \param[out] log_verbosity_level \snippet{doc} snippets.dox Log Verbosity Level + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(RunOptionsGetRunLogVerbosityLevel, _In_ const OrtRunOptions* options, + _Out_ int* log_verbosity_level); + + /** \brief Get per-run log severity level + * + * \see OrtApi::RunOptionsSetRunLogSeverityLevel + * + * \param[in] options + * \param[out] log_severity_level The log severity level (refer to ::OrtLoggingLevel for possible values). + */ + ORT_API2_STATUS(RunOptionsGetRunLogSeverityLevel, _In_ const OrtRunOptions* options, _Out_ int* log_severity_level); + + /** \brief Get per-run tag + * + * This is used in a per-run log identifier. + * + * \see OrtApi::RunOptionsSetRunTag + * + * \param[in] options + * \param[out] run_tag The run tag. + * Do not free this value, it is owned by `options`. It will be invalidated if the run tag + * changes (i.e., with OrtApi::RunOptionsSetRunTag) or `options` is freed. + */ + ORT_API2_STATUS(RunOptionsGetRunTag, _In_ const OrtRunOptions* options, _Out_ const char** run_tag); + + /** \brief Set terminate flag + * + * If a currently executing session needs to be force terminated, this can be called from another thread to force it to fail with an error. + * + * \param[in] options + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(RunOptionsSetTerminate, _Inout_ OrtRunOptions* options); + + /** \brief Clears the terminate flag + * + * Used so the OrtRunOptions instance can be used in a new OrtApi::Run call without it instantly terminating + * + * \param[in] options + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(RunOptionsUnsetTerminate, _Inout_ OrtRunOptions* options); + + /// @} + /// \name OrtValue + /// @{ + + /** \brief Create a tensor + * + * Create a tensor using a supplied ::OrtAllocator + * + * \param[in] allocator + * \param[in] shape Pointer to the tensor shape dimensions. + * \param[in] shape_len The number of tensor shape dimensions. + * \param[in] type + * \param[out] out Returns newly created ::OrtValue. Must be freed with OrtApi::ReleaseValue + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(CreateTensorAsOrtValue, _Inout_ OrtAllocator* allocator, _In_ const int64_t* shape, size_t shape_len, + ONNXTensorElementDataType type, _Outptr_ OrtValue** out); + + /** \brief Create a tensor backed by a user supplied buffer + * + * Create a tensor with user's buffer. You can fill the buffer either before calling this function or after. + * p_data is owned by caller. ReleaseValue won't release p_data. + * + * If you wish to transfer ownership of p_data to ORT use CreateTensorWithDataAndDeleterAsOrtValue. + * + * \param[in] info Memory description of where the p_data buffer resides (CPU vs GPU etc). + * \param[in] p_data Pointer to the data buffer. + * \param[in] p_data_len The number of bytes in the data buffer. + * \param[in] shape Pointer to the tensor shape dimensions. + * \param[in] shape_len The number of tensor shape dimensions. + * \param[in] type The data type. + * \param[out] out Returns newly created ::OrtValue. Must be freed with OrtApi::ReleaseValue + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(CreateTensorWithDataAsOrtValue, _In_ const OrtMemoryInfo* info, _Inout_ void* p_data, + size_t p_data_len, _In_ const int64_t* shape, size_t shape_len, ONNXTensorElementDataType type, + _Outptr_ OrtValue** out); + + /** \brief Return if an ::OrtValue is a tensor type + * + * \param[in] value A tensor type (string tensors are not supported) + * \param[out] out Set to 1 iff ::OrtValue is a tensor, 0 otherwise + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(IsTensor, _In_ const OrtValue* value, _Out_ int* out); + + /** \brief Get a pointer to the raw data inside a tensor + * + * Used to read/write/modify the internal tensor data directly. + * \note The returned pointer is valid until the \p value is destroyed. + * + * \param[in] value A tensor type (string tensors are not supported) + * \param[out] out Filled in with a pointer to the internal storage + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(GetTensorMutableData, _In_ OrtValue* value, _Outptr_ void** out); + + /** \brief Set all strings at once in a string tensor + * + * \param[in,out] value A tensor of type ONNX_TENSOR_ELEMENT_DATA_TYPE_STRING + * \param[in] s An array of strings. Each string in this array must be null terminated. + * \param[in] s_len Count of strings in s (Must match the size of \p value's tensor shape) + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(FillStringTensor, _Inout_ OrtValue* value, _In_ const char* const* s, size_t s_len); + + /** \brief Get total byte length for all strings in a string tensor + * + * Typically used with OrtApi::GetStringTensorContent + * + * \param[in] value A tensor of type ONNX_TENSOR_ELEMENT_DATA_TYPE_STRING + * \param[out] len Total byte length of all strings (does not include trailing nulls) + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(GetStringTensorDataLength, _In_ const OrtValue* value, _Out_ size_t* len); + + /** \brief Get all strings from a string tensor + * + * An example of the results:
+ * Given \p value is a string tensor with the strings { "This" "is" "a" "test" }
+ * \p s must have a size of 11 bytes
+ * \p offsets must have 4 elements
+ * After the call, these values will be filled in:
+ * \p s will contain "Thisisatest"
+ * \p offsets will contain { 0, 4, 6, 7 }
+ * The length of the last string is just s_len - offsets[last] + * + * \param[in] value A tensor of type ONNX_TENSOR_ELEMENT_DATA_TYPE_STRING + * \param[in] s Buffer to sequentially write all tensor strings to. Each string is NOT null-terminated. + * \param[in] s_len Number of bytes of buffer pointed to by \p s (Get it from OrtApi::GetStringTensorDataLength) + * \param[out] offsets Array of start offsets into the strings written to \p s + * \param[in] offsets_len Number of elements in offsets + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(GetStringTensorContent, _In_ const OrtValue* value, _Out_writes_bytes_all_(s_len) void* s, + size_t s_len, _Out_writes_all_(offsets_len) size_t* offsets, size_t offsets_len); + + /// @} + /// \name OrtTypeInfo + /// @{ + + /** \brief Get ::OrtTensorTypeAndShapeInfo from an ::OrtTypeInfo + * + * \param[in] type_info + * \param[out] out Do not free this value, it will be valid until type_info is freed. + * If type_info does not represent tensor, this value will be set to nullptr. + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(CastTypeInfoToTensorInfo, _In_ const OrtTypeInfo* type_info, + _Outptr_result_maybenull_ const OrtTensorTypeAndShapeInfo** out); + + /** \brief Get ::ONNXType from ::OrtTypeInfo + * + * \param[in] type_info + * \param[out] out + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(GetOnnxTypeFromTypeInfo, _In_ const OrtTypeInfo* type_info, _Out_ enum ONNXType* out); + + /// @} + /// \name OrtTensorTypeAndShapeInfo + /// @{ + + /** \brief Create an ::OrtTensorTypeAndShapeInfo object + * + * \param[out] out Returns newly created ::OrtTensorTypeAndShapeInfo. Must be freed with OrtApi::ReleaseTensorTypeAndShapeInfo + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(CreateTensorTypeAndShapeInfo, _Outptr_ OrtTensorTypeAndShapeInfo** out); + + /** \brief Set element type in ::OrtTensorTypeAndShapeInfo + * + * \param[in] info + * \param[in] type + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(SetTensorElementType, _Inout_ OrtTensorTypeAndShapeInfo* info, enum ONNXTensorElementDataType type); + + /** \brief Set shape information in ::OrtTensorTypeAndShapeInfo + * + * \param[in] info + * \param[in] dim_values Array with `dim_count` elements. Can contain negative values. + * \param[in] dim_count Number of elements in `dim_values` + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(SetDimensions, OrtTensorTypeAndShapeInfo* info, _In_ const int64_t* dim_values, size_t dim_count); + + /** \brief Get element type in ::OrtTensorTypeAndShapeInfo + * + * \see OrtApi::SetTensorElementType + * + * \param[in] info + * \param[out] out + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(GetTensorElementType, _In_ const OrtTensorTypeAndShapeInfo* info, + _Out_ enum ONNXTensorElementDataType* out); + + /** \brief Get dimension count in ::OrtTensorTypeAndShapeInfo + * + * \see OrtApi::GetDimensions + * + * \param[in] info + * \param[out] out + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(GetDimensionsCount, _In_ const OrtTensorTypeAndShapeInfo* info, _Out_ size_t* out); + + /** \brief Get dimensions in ::OrtTensorTypeAndShapeInfo + * + * \param[in] info + * \param[out] dim_values Array with `dim_values_length` elements. On return, filled with the dimensions stored in the ::OrtTensorTypeAndShapeInfo + * \param[in] dim_values_length Number of elements in `dim_values`. Use OrtApi::GetDimensionsCount to get this value + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(GetDimensions, _In_ const OrtTensorTypeAndShapeInfo* info, _Out_ int64_t* dim_values, + size_t dim_values_length); + + /** \brief Get symbolic dimension names in ::OrtTensorTypeAndShapeInfo + * + * \param[in] info + * \param[in] dim_params Array with `dim_params_length` elements. On return filled with pointers to null terminated strings of the dimension names + * \param[in] dim_params_length Number of elements in `dim_params`. Use OrtApi::GetDimensionsCount to get this value + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(GetSymbolicDimensions, _In_ const OrtTensorTypeAndShapeInfo* info, + _Out_writes_all_(dim_params_length) const char* dim_params[], size_t dim_params_length); + + /** \brief Get total number of elements in a tensor shape from an ::OrtTensorTypeAndShapeInfo + * + * Return the number of elements specified by the tensor shape (all dimensions multiplied by each other). + * For 0 dimensions, 1 is returned. If any dimension is less than 0, the result is always -1. + * + * Examples:
+ * [] = 1
+ * [1,3,4] = 12
+ * [2,0,4] = 0
+ * [-1,3,4] = -1
+ * + * \param[in] info + * \param[out] out Number of elements + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(GetTensorShapeElementCount, _In_ const OrtTensorTypeAndShapeInfo* info, _Out_ size_t* out); + + /// @} + /// \name OrtValue + /// @{ + + /** \brief Get type and shape information from a tensor ::OrtValue + * + * \param[in] value Must be a tensor (not a map/sequence/etc) or will return failure + * \param[out] out Newly created ::OrtTensorTypeAndShapeInfo. Must be freed with OrtApi::ReleaseTensorTypeAndShapeInfo + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(GetTensorTypeAndShape, _In_ const OrtValue* value, _Outptr_ OrtTensorTypeAndShapeInfo** out); + + /** \brief Get type information of an OrtValue + * + * \param[in] value + * \param[out] out Newly created ::OrtTypeInfo. Must be freed with OrtApi::ReleaseTypeInfo + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(GetTypeInfo, _In_ const OrtValue* value, _Outptr_result_maybenull_ OrtTypeInfo** out); + + /** \brief Get ONNXType of an ::OrtValue + * + * \param[in] value + * \param[out] out + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(GetValueType, _In_ const OrtValue* value, _Out_ enum ONNXType* out); + + /// @} + /// \name OrtMemoryInfo + /// @{ + + /** \brief Create an ::OrtMemoryInfo + * + * \param[in] name + * \param[in] type + * \param[in] id + * \param[in] mem_type + * \param[out] out Newly created ::OrtMemoryInfo. Must be freed with OrtAPi::ReleaseMemoryInfo + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(CreateMemoryInfo, _In_ const char* name, enum OrtAllocatorType type, int id, + enum OrtMemType mem_type, _Outptr_ OrtMemoryInfo** out); + + /** \brief Create an ::OrtMemoryInfo for CPU memory + * + * Special case version of OrtApi::CreateMemoryInfo for CPU based memory. Same as using OrtApi::CreateMemoryInfo with name = "Cpu" and id = 0. + * + * \param[in] type + * \param[in] mem_type + * \param[out] out + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(CreateCpuMemoryInfo, enum OrtAllocatorType type, enum OrtMemType mem_type, + _Outptr_ OrtMemoryInfo** out); + + /** \brief Compare ::OrtMemoryInfo objects for equality + * + * Compares all settings of each ::OrtMemoryInfo for equality + * + * \param[in] info1 + * \param[in] info2 + * \param[out] out Set to 0 if equal, -1 if not equal + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(CompareMemoryInfo, _In_ const OrtMemoryInfo* info1, _In_ const OrtMemoryInfo* info2, _Out_ int* out); + + /** \brief Get name from ::OrtMemoryInfo + * + * \param[in] ptr + * \param[out] out Writes null terminated string to this pointer. Do NOT free the returned pointer. It is valid for the lifetime of the ::OrtMemoryInfo + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(MemoryInfoGetName, _In_ const OrtMemoryInfo* ptr, _Out_ const char** out); + + /** \brief Get the id from ::OrtMemoryInfo + */ + ORT_API2_STATUS(MemoryInfoGetId, _In_ const OrtMemoryInfo* ptr, _Out_ int* out); + + /** \brief Get the ::OrtMemType from ::OrtMemoryInfo + */ + ORT_API2_STATUS(MemoryInfoGetMemType, _In_ const OrtMemoryInfo* ptr, _Out_ OrtMemType* out); + + /** \brief Get the ::OrtAllocatorType from ::OrtMemoryInfo + */ + ORT_API2_STATUS(MemoryInfoGetType, _In_ const OrtMemoryInfo* ptr, _Out_ OrtAllocatorType* out); + + /// @} + /// \name OrtAllocator + /// @{ + + /// \brief Calls OrtAllocator::Alloc function + ORT_API2_STATUS(AllocatorAlloc, _Inout_ OrtAllocator* ort_allocator, size_t size, _Outptr_ void** out); + /// \brief Calls OrtAllocator::Free function + ORT_API2_STATUS(AllocatorFree, _Inout_ OrtAllocator* ort_allocator, void* p); + /// \brief Calls OrtAllocator::Info function + ORT_API2_STATUS(AllocatorGetInfo, _In_ const OrtAllocator* ort_allocator, _Outptr_ const struct OrtMemoryInfo** out); + + /** \brief Get the default allocator + * + * The default allocator is a CPU based, non-arena. Always returns the same pointer to the same default allocator. + * + * \param[out] out Returned value should NOT be freed + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(GetAllocatorWithDefaultOptions, _Outptr_ OrtAllocator** out); + + /// @} + /// \name OrtSessionOptions + /// @{ + + /** \brief Override session symbolic dimensions + * + * Override symbolic dimensions (by specific denotation strings) with actual values if known at session initialization time to enable + * optimizations that can take advantage of fixed values (such as memory planning, etc) + * + * \param[in] options + * \param[in] dim_denotation + * \param[in] dim_value + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(AddFreeDimensionOverride, _Inout_ OrtSessionOptions* options, _In_ const char* dim_denotation, + _In_ int64_t dim_value); + + /// @} + /// \name OrtValue + /// @{ + + /* Internal information (not seen in Doxygen) + * + * APIs to support non-tensor types - map and sequence. + * Currently only the following types are supported + * Note: the following types should be kept in sync with data_types.h + * Map types + * ========= + * std::map + * std::map + * std::map + * std::map + * std::map + * std::map + * std::map + * std::map + * + * Sequence types + * ============== + * std::vector + * std::vector + * std::vector + * std::vector + * std::vector> + * std::vector + */ + + /** \brief Get non tensor data from an ::OrtValue + * + * If `value` is of type ONNX_TYPE_MAP, you need to retrieve the keys and values + * separately. Use index=0 to retrieve keys and index=1 to retrieve values. + * If `value` is of type ONNX_TYPE_SEQUENCE, use index to retrieve the index'th element + * of the sequence. + * + * \param[in] value + * \param[in] index See above for usage based on `value` type + * \param[in] allocator Allocator used to allocate ::OrtValue + * \param[out] out Created ::OrtValue that holds the element requested. Must be freed with OrtApi::ReleaseValue + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(GetValue, _In_ const OrtValue* value, int index, _Inout_ OrtAllocator* allocator, + _Outptr_ OrtValue** out); + + /** \brief Get non tensor value count from an ::OrtValue + * + * If `value` is of type ONNX_TYPE_MAP 2 will always be returned. For ONNX_TYPE_SEQUENCE + * the number of elements in the sequence will be returned + * + * \param[in] value + * \param[out] out + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(GetValueCount, _In_ const OrtValue* value, _Out_ size_t* out); + + /** \brief Create a map or sequence ::OrtValue + * + * To construct a map (ONNX_TYPE_MAP), use num_values = 2 and `in` should be an array of 2 ::OrtValue%s + * representing keys and values.
+ * + * To construct a sequence (ONNX_TYPE_SEQUENCE), use num_values = N where N is the number of the elements in the + * sequence. 'in' should be an array of N ::OrtValue%s. + * + * \param[in] in See above for details + * \param[in] num_values + * \param[in] value_type Must be either ONNX_TYPE_MAP or ONNX_TYPE_SEQUENCE + * \param[out] out Newly created ::OrtValue. Must be freed with OrtApi::ReleaseValue + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(CreateValue, _In_reads_(num_values) const OrtValue* const* in, size_t num_values, + enum ONNXType value_type, _Outptr_ OrtValue** out); + + /** \brief Create an opaque (custom user defined type) ::OrtValue + * + * Constructs an ::OrtValue that contains a value of non-standard type created for + * experiments or while awaiting standardization. ::OrtValue in this case would contain + * an internal representation of the Opaque type. Opaque types are distinguished from + * each other by two strings 1) domain and 2) type name. The combination of the two + * must be unique, so the type representation is properly identified internally. The combination + * must be properly registered from within ORT at both compile/run time or by another API. + * + * To construct the ::OrtValue pass domain and type names, also a pointer to a data container + * the type of which must be known to both ORT and the client program. That data container may or may + * not match the internal representation of the Opaque type. The sizeof(data_container) is passed for + * verification purposes. + * + * \param[in] domain_name Null terminated string of the domain name + * \param[in] type_name Null terminated string of the type name + * \param[in] data_container User pointer Data to populate ::OrtValue + * \param[in] data_container_size Size in bytes of what `data_container` points to + * \param[out] out Newly created ::OrtValue. Must be freed with OrtApi::ReleaseValue + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(CreateOpaqueValue, _In_z_ const char* domain_name, _In_z_ const char* type_name, + _In_ const void* data_container, size_t data_container_size, _Outptr_ OrtValue** out); + + /** \brief Get internal data from an opaque (custom user defined type) ::OrtValue + * + * Copies internal data from an opaque value into a user provided buffer + * + * \see OrtApi::CreateOpaqueValue + * + * \param[in] domain_name Null terminated string of the domain name + * \param[in] type_name Null terminated string of the type name + * \param[in] in The opaque ::OrtValue + * \param[out] data_container Buffer to copy data into + * \param[out] data_container_size Size in bytes of the buffer pointed to by data_container. Must match the size of the internal buffer. + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(GetOpaqueValue, _In_ const char* domain_name, _In_ const char* type_name, _In_ const OrtValue* in, + _Out_ void* data_container, size_t data_container_size); + + /// @} + /// \name OrtKernelInfo + /// Custom operator APIs. + /// @{ + + /** \brief Get a float stored as an attribute in the graph node + * + * \param[in] info ::OrtKernelInfo instance + * \param[in] name Null terminated string of the name of the attribute + * \param[out] out Pointer to memory where the attribute will be stored + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(KernelInfoGetAttribute_float, _In_ const OrtKernelInfo* info, _In_ const char* name, + _Out_ float* out); + + /** \brief Fetch a 64-bit int stored as an attribute in the graph node + * + * \param[in] info ::OrtKernelInfo instance + * \param[in] name Null terminated string of the name of the attribute + * \param[out] out Pointer to memory where the attribute will be stored + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(KernelInfoGetAttribute_int64, _In_ const OrtKernelInfo* info, _In_ const char* name, + _Out_ int64_t* out); + + /** \brief Fetch a string stored as an attribute in the graph node + * + * If `out` is nullptr, the value of `size` is set to the true size of the string + * attribute, and a success status is returned. + * + * If the `size` parameter is greater than or equal to the actual string attribute's size, + * the value of `size` is set to the true size of the string attribute, the provided memory + * is filled with the attribute's contents, and a success status is returned. + * + * If the `size` parameter is less than the actual string attribute's size and `out` + * is not nullptr, the value of `size` is set to the true size of the string attribute + * and a failure status is returned.) + * + * \param[in] info ::OrtKernelInfo instance + * \param[in] name Null terminated string of the name of the attribute + * \param[out] out Pointer to memory where the attribute will be stored + * \param[in,out] size See above comments for details + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(KernelInfoGetAttribute_string, _In_ const OrtKernelInfo* info, _In_ const char* name, _Out_ char* out, + _Inout_ size_t* size); + + /// @} + /// \name OrtKernelContext + /// Custom operator APIs. + /// @{ + + /** \brief Used for custom operators, get the input count of a kernel + * + * \see ::OrtCustomOp + */ + ORT_API2_STATUS(KernelContext_GetInputCount, _In_ const OrtKernelContext* context, _Out_ size_t* out); + + /** \brief Used for custom operators, get the output count of a kernel + * + * \see ::OrtCustomOp + */ + ORT_API2_STATUS(KernelContext_GetOutputCount, _In_ const OrtKernelContext* context, _Out_ size_t* out); + + /** \brief Used for custom operators, get an input of a kernel + * + * The function attempts fetches the input of the kernel. If the input is optional + * and not present, the function returns success and out is set to nullptr. + * + * \param[in] context ::OrtKernelContext instance + * \param[in] index See KernelContext_GetInputCount for boundaries check. + * \param[out] out OrtValue if the input is present otherwise is set nullptr + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(KernelContext_GetInput, _In_ const OrtKernelContext* context, _In_ size_t index, + _Out_ const OrtValue** out); + + /** \brief Used for custom operators, get an output of a kernel + * + * The function attempts fetches the output of the kernel. If the output is optional + * and not present, the function returns success and out is set to nullptr. + * + * \param[in] context ::OrtKernelContext instance + * \param[in] index See KernelContext_GetOutputCount for boundaries check. + * \param[in] dim_values output dimensions + * \param[in] dim_count number of dimensions + * \param[out] out a ptr to OrtValue to output otherwise set to nullptr + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(KernelContext_GetOutput, _Inout_ OrtKernelContext* context, _In_ size_t index, + _In_ const int64_t* dim_values, size_t dim_count, _Outptr_ OrtValue** out); + + /// @} + /// \name OrtEnv + /// @{ + ORT_CLASS_RELEASE(Env); + /// @} + /// \name OrtStatus + /// @{ + ORT_CLASS_RELEASE(Status); + /// @} + /// \name OrtMemoryInfo + /// @{ + ORT_CLASS_RELEASE(MemoryInfo); + /// @} + /// \name OrtSession + /// @{ + ORT_CLASS_RELEASE(Session); // Don't call ReleaseSession from Dllmain (because session owns a thread pool) + /// @} + /// \name OrtValue + /// @{ + ORT_CLASS_RELEASE(Value); + /// @} + /// \name OrtRunOptions + /// @{ + ORT_CLASS_RELEASE(RunOptions); + /// @} + /// \name OrtTypeInfo + /// @{ + ORT_CLASS_RELEASE(TypeInfo); + /// @} + /// \name OrtTensorTypeAndShapeInfo + /// @{ + ORT_CLASS_RELEASE(TensorTypeAndShapeInfo); + /// @} + /// \name OrtSessionOptions + /// @{ + ORT_CLASS_RELEASE(SessionOptions); + /// @} + /// \name OrtCustomOpDomain + /// @{ + ORT_CLASS_RELEASE(CustomOpDomain); + + /// @} + /// \name OrtTypeInfo + /// @{ + + /** \brief Get denotation from type information + * + * Augments ::OrtTypeInfo to return denotations on the type. + * + * This is used by WinML to determine if an input/output is intended to be an Image or a Tensor. + * + * \param[in] type_info + * \param[out] denotation Pointer to the null terminated denotation string is written to this pointer. This pointer is valid until the object is destroyed or the name is changed, do not free. + * \param[out] len Length in bytes of the string returned in `denotation` + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(GetDenotationFromTypeInfo, _In_ const OrtTypeInfo* type_info, _Out_ const char** const denotation, + _Out_ size_t* len); + + /** \brief Get detailed map information from an ::OrtTypeInfo + * + * This augments ::OrtTypeInfo to return an ::OrtMapTypeInfo when the type is a map. + * The OrtMapTypeInfo has additional information about the map's key type and value type. + * + * This is used by WinML to support model reflection APIs. + * + * \param[out] type_info + * \param[out] out A pointer to the ::OrtMapTypeInfo. Do not free this value. If type_info + * does not contain a map, this value will be set to nullptr. + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(CastTypeInfoToMapTypeInfo, _In_ const OrtTypeInfo* type_info, + _Outptr_result_maybenull_ const OrtMapTypeInfo** out); + + /** \brief Cast ::OrtTypeInfo to an ::OrtSequenceTypeInfo + * + * This api augments ::OrtTypeInfo to return an ::OrtSequenceTypeInfo when the type is a sequence. + * The ::OrtSequenceTypeInfo has additional information about the sequence's element type. + * + * This is used by WinML to support model reflection APIs. + * + * \param[in] type_info + * \param[out] out A pointer to the OrtSequenceTypeInfo. Do not free this value. If type_info + * doesn not contain a sequence, this value will be set to nullptr. + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(CastTypeInfoToSequenceTypeInfo, _In_ const OrtTypeInfo* type_info, + _Outptr_result_maybenull_ const OrtSequenceTypeInfo** out); + + /// @} + /// \name OrtMapTypeInfo + /// @{ + + /** \brief Get key type from an ::OrtMapTypeInfo + * + * Key types are restricted to being scalar types. + * + * This is used by WinML to support model reflection APIs. + * + * \param[in] map_type_info + * \param[out] out + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(GetMapKeyType, _In_ const OrtMapTypeInfo* map_type_info, _Out_ enum ONNXTensorElementDataType* out); + + /** \brief Get the value type from an ::OrtMapTypeInfo + * + * \param[in] map_type_info + * \param[out] type_info A copy of the OrtTypeInfo for the map value type. + * The user must free this value with ReleaseTypeInfo. + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(GetMapValueType, _In_ const OrtMapTypeInfo* map_type_info, _Outptr_ OrtTypeInfo** type_info); + + /// @} + /// \name OrtSequenceTypeInfo + /// @{ + + /** \brief Get element type from an ::OrtSequenceTypeInfo + * + * This is used by WinML to support model reflection APIs. + * + * \param[in] sequence_type_info + * \param[out] type_info A copy of the OrtTypeInfo for the sequence element type. + * The user must free this value with ReleaseTypeInfo. + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(GetSequenceElementType, _In_ const OrtSequenceTypeInfo* sequence_type_info, + _Outptr_ OrtTypeInfo** type_info); + + /// @} + /// \name OrtMapTypeInfo + /// @{ + ORT_CLASS_RELEASE(MapTypeInfo); + /// @} + /// \name OrtSequenceTypeInfo + /// @{ + ORT_CLASS_RELEASE(SequenceTypeInfo); + + /// @} + /// \name OrtSession + /// @{ + + /** \brief End profiling and return filename of the profile data + * + * Profiling is turned on through OrtApi::EnableProfiling + * + * \param[in] session + * \param[in] allocator + * \param[out] out Null terminated string of the filename, allocated using `allocator`. Must be freed using `allocator` + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(SessionEndProfiling, _In_ OrtSession* session, _Inout_ OrtAllocator* allocator, _Outptr_ char** out); + + /** \brief Get ::OrtModelMetadata from an ::OrtSession + * + * \param[in] session + * \param[out] out Newly created ::OrtModelMetadata. Must be freed using OrtApi::ReleaseModelMetadata + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(SessionGetModelMetadata, _In_ const OrtSession* session, _Outptr_ OrtModelMetadata** out); + + /// @} + /// \name OrtModelMetadata + /// @{ + + /** \brief Get `producer name` from an ::OrtModelMetadata + * + * \param[in] model_metadata + * \param[in] allocator + * \param[out] value Set to a null terminated string allocated using `allocator`. Must be freed using `allocator` + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(ModelMetadataGetProducerName, _In_ const OrtModelMetadata* model_metadata, + _Inout_ OrtAllocator* allocator, _Outptr_ char** value); + + /** \brief Get `graph name` from an ::OrtModelMetadata + * + * \param[in] model_metadata + * \param[in] allocator + * \param[out] value Set to a null terminated string allocated using `allocator`. Must be freed using `allocator` + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(ModelMetadataGetGraphName, _In_ const OrtModelMetadata* model_metadata, + _Inout_ OrtAllocator* allocator, _Outptr_ char** value); + + /** \brief Get `domain` from an ::OrtModelMetadata + * + * \param[in] model_metadata + * \param[in] allocator + * \param[out] value Set to a null terminated string allocated using `allocator`. Must be freed using `allocator` + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(ModelMetadataGetDomain, _In_ const OrtModelMetadata* model_metadata, _Inout_ OrtAllocator* allocator, + _Outptr_ char** value); + + /** \brief Get `description` from an ::OrtModelMetadata + * + * \param[in] model_metadata + * \param[in] allocator + * \param[out] value Set to a null terminated string allocated using `allocator`. Must be freed using `allocator` + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(ModelMetadataGetDescription, _In_ const OrtModelMetadata* model_metadata, + _Inout_ OrtAllocator* allocator, _Outptr_ char** value); + + /** \brief Return data for a key in the custom metadata map in an ::OrtModelMetadata + * + * \param[in] model_metadata + * \param[in] allocator + * \param[in] key Null terminated string + * \param[out] value Set to a null terminated string allocated using `allocator`. Must be freed using `allocator` + * `value` will be set to nullptr if the given key is not found in the custom metadata map. + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(ModelMetadataLookupCustomMetadataMap, _In_ const OrtModelMetadata* model_metadata, + _Inout_ OrtAllocator* allocator, _In_ const char* key, _Outptr_result_maybenull_ char** value); + + /** \brief Get version number from an ::OrtModelMetadata + * + * \param[in] model_metadata + * \param[out] value Set to the version number + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(ModelMetadataGetVersion, _In_ const OrtModelMetadata* model_metadata, _Out_ int64_t* value); + + ORT_CLASS_RELEASE(ModelMetadata); + + /// @} + /// \name OrtEnv + /// @{ + + /** \brief Create an OrtEnv + * + * Create an environment with global threadpools that will be shared across sessions. + * Use this in conjunction with OrtApi::DisablePerSessionThreads or else the session will use + * its own thread pools. + * + * \param[in] log_severity_level The log severity level. + * \param[in] logid The log identifier. + * \param[in] tp_options + * \param[out] out Returned newly created OrtEnv. Must be freed with OrtApi::ReleaseEnv + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(CreateEnvWithGlobalThreadPools, OrtLoggingLevel log_severity_level, _In_ const char* logid, + _In_ const OrtThreadingOptions* tp_options, _Outptr_ OrtEnv** out); + + /// @} + /// \name OrtSessionOptions + /// @{ + + /** \brief Use global thread pool on a session + * + * Disable using per session thread pool and use the shared global threadpool. + * This should be used in conjunction with OrtApi::CreateEnvWithGlobalThreadPools. + * + * \param[in] options + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(DisablePerSessionThreads, _Inout_ OrtSessionOptions* options); + + /// @} + /// \name OrtThreadingOptions + /// @{ + + /** \brief Create an ::OrtThreadingOptions + * + * \param[out] out Newly created ::OrtThreadingOptions. Must be freed with OrtApi::ReleaseThreadingOptions + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(CreateThreadingOptions, _Outptr_ OrtThreadingOptions** out); + + ORT_CLASS_RELEASE(ThreadingOptions); + + /// @} + /// \name OrtModelMetadata + /// @{ + + /** + * + * \param[in] model_metadata + * \param[in] allocator + * \param[out] keys Array of null terminated strings (array count = num_keys) allocated using `allocator`. + * The strings and the pointer array must be freed using `allocator` + * `keys` will be set to nullptr if the custom metadata map is empty. + * \param[out] num_keys Set to the number of elements in the `keys` array + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(ModelMetadataGetCustomMetadataMapKeys, _In_ const OrtModelMetadata* model_metadata, + _Inout_ OrtAllocator* allocator, _Outptr_result_buffer_maybenull_(*num_keys) char*** keys, _Out_ int64_t* num_keys); + + /// @} + /// \name OrtSessionOptions + /// @{ + + /** + * + * Override symbolic dimensions (by specific name strings) with actual values + * if known at session initialization time to enable optimizations that can + * take advantage of fixed values (such as memory planning, etc) + * + */ + ORT_API2_STATUS(AddFreeDimensionOverrideByName, + _Inout_ OrtSessionOptions* options, _In_ const char* dim_name, + _In_ int64_t dim_value); + + /// @} + /// \name Misc + /// @{ + + /** \brief Get the names of all available providers + * + * \note The providers in the list are not guaranteed to be usable. They may fail to load due to missing system dependencies. + * For example, if the CUDA/cuDNN libraries are not installed, the CUDA provider will report an error when it is added to the session options. + * + * \param[out] out_ptr Set to a pointer to an array of null terminated strings of the available providers. The entries and the + * array itself must be freed using OrtApi::ReleaseAvailableProviders + * \param[out] provider_length Set to the number of entries in the `out_ptr` array + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(GetAvailableProviders, _Outptr_ char*** out_ptr, _Out_ int* provider_length); + + /** \brief Release data from OrtApi::GetAvailableProviders. This API will never fail + * so you can rely on it in a noexcept code. + * + * \param[in] ptr The `out_ptr` result from OrtApi::GetAvailableProviders. + * \param[in] providers_length The `provider_length` result from OrtApi::GetAvailableProviders + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(ReleaseAvailableProviders, _In_ char** ptr, + _In_ int providers_length); + + /// @} + /// \name OrtValue + /// @{ + + /** \brief Get the length of a single string in a string tensor + * + * \param[in] value A string tensor + * \param[in] index Index of the string in the tensor + * \param[out] out Set to number of bytes of the string element + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(GetStringTensorElementLength, _In_ const OrtValue* value, size_t index, _Out_ size_t* out); + + /** \brief Get a single string from a string tensor + * + * \param[in] value A string tensor + * \param[in] s_len Number of bytes in the `s` buffer. Must match the value returned by OrtApi::GetStringTensorElementLength. + * \param[in] index Index of the string in the tensor + * \param[out] s The string element contents in UTF-8 encoding. The string is NOT null-terminated. + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(GetStringTensorElement, _In_ const OrtValue* value, size_t s_len, size_t index, _Out_writes_bytes_all_(s_len) void* s); + + /** \brief Set a single string in a string tensor + * + * \param[in] value A string tensor + * \param[in] s A null terminated UTF-8 encoded string + * \param[in] index Index of the string in the tensor to set + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(FillStringTensorElement, _Inout_ OrtValue* value, _In_ const char* s, size_t index); + + /// @} + /// \name OrtSessionOptions + /// @{ + + /** \brief Set a session configuration entry as a pair of strings + * + * If a configuration with same key exists, this will overwrite the configuration with the given config_value. + * + * The config_key and the format of config_value are defined in onnxruntime_session_options_config_keys.h + * + * \param[in] options + * \param[in] config_key A null terminated string representation of the config key + * \param[in] config_value A null terminated string representation of the config value + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(AddSessionConfigEntry, _Inout_ OrtSessionOptions* options, + _In_z_ const char* config_key, _In_z_ const char* config_value); + + /// @} + /// \name OrtAllocator + /// @{ + + /** \brief Create an allocator for an ::OrtSession following an ::OrtMemoryInfo + * + * \param[in] session + * \param[in] mem_info valid ::OrtMemoryInfo instance + * \param[out] out Newly created ::OrtAllocator. Must be freed with OrtApi::ReleaseAllocator + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(CreateAllocator, _In_ const OrtSession* session, _In_ const OrtMemoryInfo* mem_info, + _Outptr_ OrtAllocator** out); + + /** \brief Release an ::OrtAllocator obtained from OrtApi::CreateAllocator + */ + ORT_CLASS_RELEASE(Allocator); + + /// @} + /// \name OrtSession + /// @{ + + /** \brief Run a model using Io Bindings for the inputs & outputs + * + * \see OrtApi::Run + * + * \param[in] session + * \param[in] run_options + * \param[in] binding_ptr + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(RunWithBinding, _Inout_ OrtSession* session, _In_ const OrtRunOptions* run_options, _In_ const OrtIoBinding* binding_ptr); + + /** \brief Create an ::OrtIoBinding instance + * + * An IoBinding object allows one to bind pre-allocated ::OrtValue%s to input names. + * Thus if you want to use a raw on device buffer as input or output you can avoid + * extra copy during runtime. + * + * \param[in] session + * \param[out] out Newly created ::OrtIoBinding. Must be freed with OrtApi::ReleaseIoBinding + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(CreateIoBinding, _Inout_ OrtSession* session, _Outptr_ OrtIoBinding** out); + + /// @} + /// \name OrtIoBinding + /// @{ + + /** \brief Release an ::OrtIoBinding obtained from OrtApi::CreateIoBinding + */ + ORT_CLASS_RELEASE(IoBinding); + + /** \brief Bind an ::OrtValue to an ::OrtIoBinding input + * + * When using OrtApi::RunWithBinding this value is used for the named input + * + * \param[in] binding_ptr + * \param[in] name Name for the model input + * \param[in] val_ptr ::OrtValue of Tensor type. + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(BindInput, _Inout_ OrtIoBinding* binding_ptr, _In_ const char* name, _In_ const OrtValue* val_ptr); + + /** \brief Bind an ::OrtValue to an ::OrtIoBinding output + * + * When using OrtApi::RunWithBinding this value is used for the named output + * + * \param[in] binding_ptr + * \param[in] name Null terminated string of the model output name + * \param[in] val_ptr ::OrtValue of Tensor type. + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(BindOutput, _Inout_ OrtIoBinding* binding_ptr, _In_ const char* name, _In_ const OrtValue* val_ptr); + + /** \brief Bind an ::OrtIoBinding output to a device + * + * Binds the ::OrtValue to a device which is specified by ::OrtMemoryInfo. + * You can either create an instance of ::OrtMemoryInfo with a device id or obtain one from the allocator that you have created/are using + * This is useful when one or more outputs have dynamic shapes and, it is hard to pre-allocate and bind a chunk of + * memory within ::OrtValue ahead of time. + * + * \see OrtApi::RunWithBinding + * + * \param[in] binding_ptr + * \param[in] name Null terminated string of the device name + * \param[in] mem_info_ptr + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(BindOutputToDevice, _Inout_ OrtIoBinding* binding_ptr, _In_ const char* name, _In_ const OrtMemoryInfo* mem_info_ptr); + + /** \brief Get the names of an ::OrtIoBinding's outputs + * + * Returns the names of the outputs in the order they were bound. This is useful after running the model + * with bound outputs because the returned names are in order in which output ::OrtValue are returned. This is useful if + * the order of outputs and their names is not known. + * + * \param[in] binding_ptr + * \param[in] allocator Allocator used to allocate continuous buffers for output strings and lengths. + * \param[out] buffer Returns an array of non-null terminated UTF-8 strings. The number of strings stored is returned in the count parameter. + * This buffer is allocated using `allocator` and must be freed using it. + * \param[out] lengths Returns an array of `count` lengths of the strings returned in `buffer` + * This buffer is allocated using `allocator` and must be freed using it. + * \param[out] count Number of strings returned. If `binding_ptr` has no bound outputs, zero is returned, + * no memory allocation is performed and buffer and lengths are set to nullptr. + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(GetBoundOutputNames, _In_ const OrtIoBinding* binding_ptr, _In_ OrtAllocator* allocator, + _Out_ char** buffer, _Out_writes_all_(count) size_t** lengths, _Out_ size_t* count); + + /** \brief Get the output ::OrtValue objects from an ::OrtIoBinding + * + * Returns an array of pointers to individually allocated ::OrtValue%s that contain results of a model execution with OrtApi::RunWithBinding + * The array contains the same number of ::OrtValue%s and they are in the same order as they were bound with OrtApi::BindOutput + * or OrtApi::BindOutputToDevice. + * + * The returned ::OrtValue%s must be released using OrtApi::ReleaseValue after they are no longer needed. + * The array is allocated using the specified instance of the allocator and must be freed using the same allocator after + * all the ::OrtValue%s contained therein are individually released. + * + * \param[in] binding_ptr + * \param[in] allocator Allocator used to allocate output array + * \param[out] output Set to the allocated array of allocated ::OrtValue outputs. Set to nullptr if there are 0 outputs. + * \param[out] output_count Set to number of ::OrtValue%s returned + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(GetBoundOutputValues, _In_ const OrtIoBinding* binding_ptr, _In_ OrtAllocator* allocator, + _Out_writes_all_(output_count) OrtValue*** output, _Out_ size_t* output_count); + + /** \brief Clears any previously set Inputs for an ::OrtIoBinding + */ + void(ORT_API_CALL* ClearBoundInputs)(_Inout_ OrtIoBinding* binding_ptr) NO_EXCEPTION ORT_ALL_ARGS_NONNULL; + + /** \brief Clears any previously set Outputs for an ::OrtIoBinding + */ + void(ORT_API_CALL* ClearBoundOutputs)(_Inout_ OrtIoBinding* binding_ptr) NO_EXCEPTION ORT_ALL_ARGS_NONNULL; + + /// @} + /// \name OrtValue + /// @{ + + /** \brief Direct memory access to a specified tensor element + * + * For example, given a tensor with shape of [3,224,224], a pointer to the element at location [2,150,128] can be retrieved + * + * This function only works for numeric type tensors (No strings, etc). + * This is a no-copy method whose returned pointer is valid until the passed in ::OrtValue is free'd. + * + * \param[in] value + * \param[in] location_values Pointer to an array of index values that specify an element's location relative to its shape + * \param[in] location_values_count Number of elements in location_values. Must match the number of elements in the tensor's shape. + * \param[out] out Set to a pointer to the element specified + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(TensorAt, _Inout_ OrtValue* value, const int64_t* location_values, size_t location_values_count, _Outptr_ void** out); + + /// @} + /// \name OrtEnv + /// @{ + + /** \brief Create an allocator and register it with the ::OrtEnv + * + * Enables sharing the allocator between multiple sessions that use the same env instance. + * Lifetime of the created allocator will be valid for the duration of the environment. + * Returns an error if an allocator with the same ::OrtMemoryInfo is already registered. + * + * See https://onnxruntime.ai/docs/get-started/with-c.html for details. + * + * \param[in] env ::OrtEnv instance + * \param[in] mem_info + * \param[in] arena_cfg Pass nullptr for defaults + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(CreateAndRegisterAllocator, _Inout_ OrtEnv* env, _In_ const OrtMemoryInfo* mem_info, + _In_ const OrtArenaCfg* arena_cfg); + + /** \brief Set language projection + * + * Set the language projection for collecting telemetry data when Env is created. + * + * The default is ORT_PROJECTION_C, which means it will classify the language not in the list to C also. + * + * \param[in] ort_env + * \param[in] projection + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(SetLanguageProjection, _In_ const OrtEnv* ort_env, _In_ OrtLanguageProjection projection); + + /// @} + /// \name OrtSession + /// @{ + + /** \brief Return the time that profiling was started + * + * \note The timer precision varies per platform. On Windows and MacOS, the precision will be ~100ns + * + * \param[in] session + * \param[out] out nanoseconds of profiling's start time + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(SessionGetProfilingStartTimeNs, _In_ const OrtSession* session, _Outptr_ uint64_t* out); + + /// @} + /// \name OrtThreadingOptions + /// @{ + + /** \brief Set global intra-op thread count + * + * This configures the global thread pool options to be used in the call to OrtApi::CreateEnvWithGlobalThreadPools + * + * \param[in] tp_options + * \param[in] intra_op_num_threads Number of threads, special values:
+ * 0 = Use default thread count
+ * 1 = The invoking thread will be used; no threads will be created in the thread pool. + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(SetGlobalIntraOpNumThreads, _Inout_ OrtThreadingOptions* tp_options, int intra_op_num_threads); + + /** \brief Set global inter-op thread count + * + * This configures the global thread pool options to be used in the call to OrtApi::CreateEnvWithGlobalThreadPools + * + * \param[in] tp_options + * \param[in] inter_op_num_threads Number of threads, special values:
+ * 0 = Use default thread count
+ * 1 = The invoking thread will be used; no threads will be created in the thread pool. + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(SetGlobalInterOpNumThreads, _Inout_ OrtThreadingOptions* tp_options, int inter_op_num_threads); + + /** \brief Set global spin control options + * + * This will configure the global thread pool options to be used in the call to OrtApi::CreateEnvWithGlobalThreadPools. + * Allow spinning of thread pools when their queues are empty. This will set the value for both + * inter_op and intra_op threadpools. + * + * \param[in] tp_options + * \param[in] allow_spinning Valid values are 0 or 1.
+ * 0 = It won't spin (recommended if CPU usage is high)
+ * 1 = Threadpool will spin to wait for queue to become non-empty + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(SetGlobalSpinControl, _Inout_ OrtThreadingOptions* tp_options, int allow_spinning); + + /// @} + /// \name OrtSessionOptions + /// @{ + + /** \brief Add a pre-allocated initializer to a session + * + * If a model contains an initializer with a name that is same as the name passed to this call, + * ORT will use this initializer instance instead of deserializing one from the model file. This + * is useful when you want to share the same initializer across sessions. + * + * \param[in] options + * \param[in] name Null terminated string of the initializer name + * \param[in] val ::OrtValue containing the initializer. Its lifetime and the underlying initializer buffer must be + * managed by the user (created using the OrtApi::CreateTensorWithDataAsOrtValue) and it must outlive the session object + * to which it is added. + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(AddInitializer, _Inout_ OrtSessionOptions* options, _In_z_ const char* name, + _In_ const OrtValue* val); + + /// @} + /// \name OrtEnv + /// @{ + + /** + * Create a custom environment with global threadpools and logger that will be shared across sessions. + * Use this in conjunction with OrtApi::DisablePerSessionThreads or else the session will use + * its own thread pools. + * + * \param[in] logging_function A pointer to a logging function. + * \param[in] logger_param A pointer to arbitrary data passed as the ::OrtLoggingFunction `param` parameter to + * `logging_function`. + * \param[in] log_severity_level The log severity level. + * \param[in] logid The log identifier. + * \param[in] tp_options + * \param[out] out Newly created OrtEnv. Must be freed with OrtApi::ReleaseEnv + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(CreateEnvWithCustomLoggerAndGlobalThreadPools, OrtLoggingFunction logging_function, _In_opt_ void* logger_param, OrtLoggingLevel log_severity_level, + _In_ const char* logid, _In_ const struct OrtThreadingOptions* tp_options, _Outptr_ OrtEnv** out); + + /// @} + /// \name OrtSessionOptions + /// @{ + + /** \brief Append CUDA provider to session options + * + * If CUDA is not available (due to a non CUDA enabled build, or if CUDA is not installed on the system), this function will return failure. + * + * \param[in] options + * \param[in] cuda_options + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(SessionOptionsAppendExecutionProvider_CUDA, + _In_ OrtSessionOptions* options, _In_ const OrtCUDAProviderOptions* cuda_options); + + /** \brief Append ROCM execution provider to the session options + * + * If ROCM is not available (due to a non ROCM enabled build, or if ROCM is not installed on the system), this function will return failure. + * + * \param[in] options + * \param[in] rocm_options + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(SessionOptionsAppendExecutionProvider_ROCM, + _In_ OrtSessionOptions* options, _In_ const OrtROCMProviderOptions* rocm_options); + + /** \brief Append OpenVINO execution provider to the session options + * + * If OpenVINO is not available (due to a non OpenVINO enabled build, or if OpenVINO is not installed on the system), this function will fail. + * + * \param[in] options + * \param[in] provider_options + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(SessionOptionsAppendExecutionProvider_OpenVINO, + _In_ OrtSessionOptions* options, _In_ const OrtOpenVINOProviderOptions* provider_options); + + /// @} + /// \name OrtThreadingOptions + /// @{ + + /** \brief Set threading flush-to-zero and denormal-as-zero + * + * Sets global thread pool options to be used in the call to OrtApi::CreateEnvWithGlobalThreadPools. + * Flush-to-zero and denormal-as-zero are applied to threads in both intra and inter global thread pool. + * \note This option is not needed if the models used have no denormals. Having no denormals is recommended as this option may hurt model accuracy. + * + * \param[in] tp_options + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(SetGlobalDenormalAsZero, _Inout_ OrtThreadingOptions* tp_options); + + /// @} + /// \name OrtArenaCfg + /// @{ + + /** \deprecated Use OrtApi::CreateArenaCfgV2 + * + * This will create the configuration of an arena that can eventually be used to define an arena based allocator's behavior + * + * \param[in] max_mem Use 0 to allow ORT to choose the default + * \param[in] arena_extend_strategy Use -1 to allow ORT to choose the default, 0 = kNextPowerOfTwo, 1 = kSameAsRequested + * \param[in] initial_chunk_size_bytes Use -1 to allow ORT to choose the default + * \param[in] max_dead_bytes_per_chunk Use -1 to allow ORT to choose the default + * \param[in] out A pointer to an OrtArenaCfg instance + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(CreateArenaCfg, _In_ size_t max_mem, int arena_extend_strategy, int initial_chunk_size_bytes, + int max_dead_bytes_per_chunk, _Outptr_ OrtArenaCfg** out); + + ORT_CLASS_RELEASE(ArenaCfg); + + /// @} + /// \name OrtModelMetadata + /// @{ + + /** + * Use this to obtain the description of the graph present in the model + * (doc_string field of the GraphProto message within the ModelProto message). + * If it doesn't exist, an empty string will be returned. + * + * \param[in] model_metadata An instance of ::OrtModelMetadata + * \param[in] allocator Allocator used to allocate the string that will be returned back + * \param[out] value Set to a null terminated string allocated using `allocator`. The caller is responsible for freeing it using `allocator` + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(ModelMetadataGetGraphDescription, _In_ const OrtModelMetadata* model_metadata, + _Inout_ OrtAllocator* allocator, _Outptr_ char** value); + + /// @} + /// \name OrtSessionOptions + /// @{ + + /** \brief Append TensorRT provider to session options + * + * If TensorRT is not available (due to a non TensorRT enabled build, or if TensorRT is not installed on the system), this function will return failure. + * + * \param[in] options + * \param[in] tensorrt_options + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(SessionOptionsAppendExecutionProvider_TensorRT, + _In_ OrtSessionOptions* options, _In_ const OrtTensorRTProviderOptions* tensorrt_options); + + /// @} + /// \name Misc + /// @{ + + /** \brief Set current GPU device ID + * + * Set the current device id of the GPU execution provider (CUDA/tensorrt/rocm). The device id should be less + * than the total number of devices available. This is only useful when multiple-GPUs are installed and it is + * required to restrict execution to a single GPU. + * + * \param[in] device_id + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(SetCurrentGpuDeviceId, _In_ int device_id); + + /** \brief Get current GPU device ID + * + * Get the current device id of the GPU execution provider (CUDA/tensorrt/rocm). + * + * \see OrtApi::SetCurrentGpuDeviceId + * + * \param[out] device_id + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(GetCurrentGpuDeviceId, _In_ int* device_id); + + /// @} + /// \name OrtKernelInfo + /// Custom operator APIs. + /// @{ + + /** \brief Fetch an array of int64_t values stored as an attribute in the graph node + * + * + * If `out` is nullptr, the value of `size` is set to the true size of the attribute + * array's size, and a success status is returned. + * + * If the `size` parameter is greater than or equal to the actual attribute array's size, + * the value of `size` is set to the true size of the attribute array's size, + * the provided memory is filled with the attribute's contents, + * and a success status is returned. + * + * If the `size` parameter is less than the actual attribute array's size and `out` + * is not nullptr, the value of `size` is set to the true size of the attribute array's size + * and a failure status is returned.) + * + * \param[in] info instance + * \param[in] name name of the attribute to be parsed + * \param[out] out pointer to memory where the attribute's contents are to be stored + * \param[in, out] size actual size of attribute array + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(KernelInfoGetAttributeArray_float, _In_ const OrtKernelInfo* info, _In_ const char* name, + _Out_ float* out, _Inout_ size_t* size); + + /** \brief Fetch an array of int64_t values stored as an attribute in the graph node + * + * If `out` is nullptr, the value of `size` is set to the true size of the attribute + * array's size, and a success status is returned. + * + * If the `size` parameter is greater than or equal to the actual attribute array's size, + * the value of `size` is set to the true size of the attribute array's size, + * the provided memory is filled with the attribute's contents, + * and a success status is returned. + * + * If the `size` parameter is less than the actual attribute array's size and `out` + * is not nullptr, the value of `size` is set to the true size of the attribute array's size + * and a failure status is returned.) + * + * \param[in] info instance + * \param[in] name name of the attribute to be parsed + * \param[out] out pointer to memory where the attribute's contents are to be stored + * \param[in, out] size actual size of attribute array + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(KernelInfoGetAttributeArray_int64, _In_ const OrtKernelInfo* info, _In_ const char* name, + _Out_ int64_t* out, _Inout_ size_t* size); + + /// @} + /// \name OrtArenaCfg + /// @{ + + /** \brief Create an ::OrtArenaCfg + * + * Create the configuration of an arena that can eventually be used to define an arena based allocator's behavior. + * + * Supported keys are (See https://onnxruntime.ai/docs/get-started/with-c.html for details on what the + * following parameters mean and how to choose these values.): + * "max_mem": Maximum memory that can be allocated by the arena based allocator. + * Use 0 for ORT to pick the best value. Default is 0. + * "arena_extend_strategy": 0 = kNextPowerOfTwo, 1 = kSameAsRequested. + * Use -1 to allow ORT to choose the default. + * "initial_chunk_size_bytes": (Possible) Size of the first allocation in the arena. + * Only relevant if arena strategy is `kNextPowerOfTwo`. Use -1 to allow ORT to choose the default. + * Ultimately, the first allocation size is determined by the allocation memory request. + * "max_dead_bytes_per_chunk": Threshold of unused memory in an allocated chunk of arena memory after + * crossing which the current chunk is chunked into 2. + * "initial_growth_chunk_size_bytes": (Possible) Size of the second allocation in the arena. + * Only relevant if arena strategy is `kNextPowerOfTwo`. Use -1 to allow ORT to choose the default. + * "max_power_of_two_extend_bytes": The maximum enxtend size if arena strategy is `kNextPowerOfTwo`. + * It is not an allocation limit, it is only a limit for extension when requested byte is less than the limit. + * When requested bytes is more than the limit, allocator will still return as requested. + * Use -1 to allow ORT to choose the default 1GB for max_power_of_two_extend_bytes. + * Ultimately, the allocation size is determined by the allocation memory request. + * Further allocation sizes are governed by the arena extend strategy. + * + * \param[in] arena_config_keys Keys to configure the arena + * \param[in] arena_config_values Values to configure the arena + * \param[in] num_keys Number of keys in `arena_config_keys` and `arena_config_values` + * \param[out] out Newly created ::OrtArenaCfg. Must be freed with OrtApi::ReleaseArenaCfg + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(CreateArenaCfgV2, _In_reads_(num_keys) const char* const* arena_config_keys, + _In_reads_(num_keys) const size_t* arena_config_values, _In_ size_t num_keys, + _Outptr_ OrtArenaCfg** out); + + /// @} + /// \name OrtRunOptions + /// @{ + + /** \brief Set a single run configuration entry as a pair of strings + * + * If a configuration with same key exists, this will overwrite the configuration with the given config_value + * + * The config_key and the format of config_value are defined in onnxruntime_run_options_config_keys.h + * + * \param[in] options + * \param[in] config_key A null terminated string representation of the config key + * \param[in] config_value A null terminated string representation of the config value + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(AddRunConfigEntry, _Inout_ OrtRunOptions* options, + _In_z_ const char* config_key, _In_z_ const char* config_value); + + /// @} + /// \name OrtPrepackedWeightsContainer + /// @{ + + /** \brief Create an ::OrtPrepackedWeightsContainer + * + * This container will hold pre-packed buffers of shared initializers for sharing between sessions + * (i.e.) if there are shared initializers that can be shared between sessions, the pre-packed buffers + * of these (if any) may possibly be shared to provide memory footprint savings. Pass this container + * to sessions that you would like to share pre-packed buffers of shared initializers at session + * creation time. + * + * \param[out] out Newly created ::OrtPrepackedWeightsContainer. Must be freed with OrtApi::ReleasePrepackedWeightsContainer + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(CreatePrepackedWeightsContainer, _Outptr_ OrtPrepackedWeightsContainer** out); + + /** \brief Release OrtPrepackedWeightsContainer instance + * + * \note instance must not be released until the sessions using it are released + */ + ORT_CLASS_RELEASE(PrepackedWeightsContainer); + + /// @} + /// \name OrtSession + /// @{ + + /** \brief Create session with prepacked weights container + * + * Same functionality offered by OrtApi::CreateSession except that a container that contains + * pre-packed weights' buffers is written into/read from by the created session. + * This is useful when used in conjunction with OrtApi::AddInitializer which injects + * shared initializer info into sessions. Wherever possible, the pre-packed versions of these + * shared initializers are cached in this container so that multiple sessions can just re-use + * these instead of duplicating these in memory. + * + * \param[in] env OrtEnv instance instance + * \param[in] model_path Null terminated string of the path (wchar on Windows, char otherwise) + * \param[in] options + * \param[in] prepacked_weights_container + * \param[out] out Newly created ::OrtSession. Must be freed with OrtApi::ReleaseSession + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(CreateSessionWithPrepackedWeightsContainer, _In_ const OrtEnv* env, _In_ const ORTCHAR_T* model_path, + _In_ const OrtSessionOptions* options, + _Inout_ OrtPrepackedWeightsContainer* prepacked_weights_container, + _Outptr_ OrtSession** out); + + /** \brief Create session from memory with prepacked weights container + * + * Same functionality offered by OrtApi::CreateSessionFromArray except that a container that contains + * pre-packed weights' buffers is written into/read from by the created session. + * This is useful when used in conjunction with OrtApi::AddInitializer which injects + * shared initializer info into sessions. Wherever possible, the pre-packed versions of these + * shared initializers are cached in this container so that multiple sessions can just re-use + * these instead of duplicating these in memory. + * + * \param[in] env + * \param[in] model_data Array of bytes holding the model + * \param[in] model_data_length Number of bytes in `model_data_model` + * \param[in] options + * \param[in] prepacked_weights_container + * \param[out] out Newly created ::OrtSession. Must be freed with OrtApi::ReleaseSession + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(CreateSessionFromArrayWithPrepackedWeightsContainer, _In_ const OrtEnv* env, + _In_ const void* model_data, size_t model_data_length, + _In_ const OrtSessionOptions* options, + _Inout_ OrtPrepackedWeightsContainer* prepacked_weights_container, + _Outptr_ OrtSession** out); + + /// @} + /// \name OrtSessionOptions + /// @{ + + /** \brief Append TensorRT execution provider to the session options + * + * If TensorRT is not available (due to a non TensorRT enabled build), this function will return failure. + * + * This is slightly different from OrtApi::SessionOptionsAppendExecutionProvider_TensorRT, it takes an + * ::OrtTensorRTProviderOptions which is publicly defined. This takes an opaque ::OrtTensorRTProviderOptionsV2 + * which must be created with OrtApi::CreateTensorRTProviderOptions. + * + * For OrtApi::SessionOptionsAppendExecutionProvider_TensorRT, the user needs to instantiate ::OrtTensorRTProviderOptions + * as well as allocate/release buffers for some members of ::OrtTensorRTProviderOptions. + * Here, OrtApi::CreateTensorRTProviderOptions and Ortapi::ReleaseTensorRTProviderOptions will do the memory management for you. + * + * \param[in] options + * \param[in] tensorrt_options + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(SessionOptionsAppendExecutionProvider_TensorRT_V2, + _In_ OrtSessionOptions* options, _In_ const OrtTensorRTProviderOptionsV2* tensorrt_options); + + /// @} + /// \name OrtTensorRTProviderOptionsV2 + /// @{ + + /** \brief Create an OrtTensorRTProviderOptionsV2 + * + * \param[out] out Newly created ::OrtTensorRTProviderOptionsV2. Must be released with OrtApi::ReleaseTensorRTProviderOptions + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(CreateTensorRTProviderOptions, _Outptr_ OrtTensorRTProviderOptionsV2** out); + + /** \brief Set options in a TensorRT Execution Provider. + * + * Please refer to https://onnxruntime.ai/docs/execution-providers/TensorRT-ExecutionProvider.html#cc + * to know the available keys and values. Key should be in null terminated string format of the member of ::OrtTensorRTProviderOptionsV2 + * and value should be its related range. Recreates the options and only sets the supplied values. + * + * For example, key="trt_max_workspace_size" and value="2147483648" + * + * \param[in] tensorrt_options + * \param[in] provider_options_keys Array of UTF-8 null-terminated string for provider options keys + * \param[in] provider_options_values Array of UTF-8 null-terminated string for provider options values + * \param[in] num_keys Number of elements in the `provider_option_keys` and `provider_options_values` arrays + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(UpdateTensorRTProviderOptions, _Inout_ OrtTensorRTProviderOptionsV2* tensorrt_options, + _In_reads_(num_keys) const char* const* provider_options_keys, + _In_reads_(num_keys) const char* const* provider_options_values, + _In_ size_t num_keys); + + /** \brief Get serialized TensorRT provider options string. + * + * For example, "trt_max_workspace_size=2147483648;trt_max_partition_iterations=10;trt_int8_enable=1;......" + * + * \param tensorrt_options - OrtTensorRTProviderOptionsV2 instance + * \param allocator - a ptr to an instance of OrtAllocator obtained with OrtApi::CreateAllocator or OrtApi::GetAllocatorWithDefaultOptions + * the specified allocator will be used to allocate continuous buffers for output strings and lengths. + * \param ptr - is a UTF-8 null terminated string allocated using 'allocator'. The caller is responsible for using the same allocator to free it. + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(GetTensorRTProviderOptionsAsString, _In_ const OrtTensorRTProviderOptionsV2* tensorrt_options, _Inout_ OrtAllocator* allocator, _Outptr_ char** ptr); + + /** \brief Release an ::OrtTensorRTProviderOptionsV2 + * + * \note This is an exception in the naming convention of other Release* functions, as the name of the method does not have the V2 suffix, but the type does + */ + void(ORT_API_CALL* ReleaseTensorRTProviderOptions)(_Frees_ptr_opt_ OrtTensorRTProviderOptionsV2* input); + + /// @} + /// \name OrtSessionOptions + /// @{ + + /** \brief Enable custom operators + * + * See onnxruntime-extensions: https://github.com/microsoft/onnxruntime-extensions.git + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(EnableOrtCustomOps, _Inout_ OrtSessionOptions* options); + + /// @} + /// \name OrtAllocator + /// @{ + + /** \brief Register a custom allocator + * + * Enables sharing between multiple sessions that use the same env instance. + * Returns an error if an allocator with the same ::OrtMemoryInfo is already registered. + * + * The behavior of this is exactly the same as OrtApi::CreateAndRegisterAllocator except + * instead of ORT creating an allocator based on provided info, in this case + * ORT uses the user-provided custom allocator. + * See https://onnxruntime.ai/docs/get-started/with-c.html for details. + * + * \param[in] env + * \param[in] allocator User provided allocator + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(RegisterAllocator, _Inout_ OrtEnv* env, _In_ OrtAllocator* allocator); + + /** \brief Unregister a custom allocator + * + * It is an error if you provide an ::OrtMemoryInfo not corresponding to any + * registered allocators for sharing. + * + * \param[in] env + * \param[in] mem_info + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(UnregisterAllocator, _Inout_ OrtEnv* env, + _In_ const OrtMemoryInfo* mem_info); + + /// @} + /// \name OrtValue + /// @{ + + /** \brief Sets *out to 1 iff an ::OrtValue is a SparseTensor, and 0 otherwise + * + * \param[in] value existing ::OrtValue + * \param[out] out unless an error occurs, contains 1 iff the value contains an instance + * of sparse tensor or 0 otherwise. + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(IsSparseTensor, _In_ const OrtValue* value, _Out_ int* out); + + /** \brief Create an ::OrtValue with a sparse tensor that is empty. + * + * Use FillSparseTensor() functions to populate sparse tensor with non-zero values and + * format specific indices data. + * Use ReleaseValue to destroy the sparse tensor, this will also release the buffer inside the output value + * if any was allocated. + * \param[in,out] allocator allocator to use when performing an allocation. Allocation will be performed + * by FillSparseTensor() APIs. The lifespan of the allocator instance must eclipse the lifespan + * this sparse tensor instance as the same allocator will be used to free memory. + * \param[in] dense_shape shape of the original dense tensor + * \param[in] dense_shape_len number of shape dimensions being passed + * \param[in] type must be one of TENSOR_ELEMENT_DATA_TYPE_xxxx + * \param[out] out Should be freed by calling ReleaseValue + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(CreateSparseTensorAsOrtValue, _Inout_ OrtAllocator* allocator, _In_ const int64_t* dense_shape, + size_t dense_shape_len, ONNXTensorElementDataType type, _Outptr_ OrtValue** out); + + /** + * This fills populates an empty tensor that was created using OrtApi::CreateSparseTensorAsOrtValue. + * This will allocate required memory and copy the supplied NNZ values and COO indices into that memory allocation. + * Memory allocation is performed using the allocator that was specified with OrtApi::CreateSparseTensorAsOrtValue. + * + * \param[in,out] ort_value ::OrtValue to populate with data + * \param[in] data_mem_info serves to identify the location of the data to be copied. If the allocator specified + * at the creation time has memory info that is not the same as mem_info argument to this function a X-device copy will be performed. + * String data is assumed to be on CPU and will only be copied into a CPU allocated buffer. + * \param[in] values_shape pointer to values shape array + * \param[in] values_shape_len length of the values_shape + * \param[in] values pointer to an array of values. For strings, pass const char**. + * \param[in] indices_data pointer to a location of COO indices + * \param[in] indices_num number of COO indices + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(FillSparseTensorCoo, _Inout_ OrtValue* ort_value, _In_ const OrtMemoryInfo* data_mem_info, + _In_ const int64_t* values_shape, size_t values_shape_len, _In_ const void* values, + _In_ const int64_t* indices_data, size_t indices_num); + + /** + * This fills populates an empty tensor that was created using OrtApi::CreateSparseTensorAsOrtValue. + * This will allocate required memory and copy the supplied NNZ values and CSR indices into that memory allocation. + * Memory allocation is performed using the allocator that was specified with OrtApi::CreateSparseTensorAsOrtValue. + * + * \param[in,out] ort_value ::OrtValue to populate with data + * \param[in] data_mem_info serves to identify the location of the data to be copied. If the allocator specified + * at the creation time has memory info that is not the same as mem_info argument to this function a X-device copy will be performed. + * String data is assumed to be on CPU and will only be copied into a CPU allocated buffer. + * \param[in] values_shape pointer to values shape array + * \param[in] values_shape_len length of the values_shape + * \param[in] values - pointer to an array of values. For strings, pass const char**. + * \param[in] inner_indices_data pointer to a location of CSR inner indices + * \param[in] inner_indices_num number of CSR inner indices + * \param[in] outer_indices_data pointer to a location of CSR outer indices + * \param[in] outer_indices_num number of CSR outer indices + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(FillSparseTensorCsr, _Inout_ OrtValue* ort_value, _In_ const OrtMemoryInfo* data_mem_info, + _In_ const int64_t* values_shape, size_t values_shape_len, _In_ const void* values, + _In_ const int64_t* inner_indices_data, size_t inner_indices_num, + _In_ const int64_t* outer_indices_data, size_t outer_indices_num); + + /** + * This fills populates an empty tensor that was created using OrtApi::CreateSparseTensorAsOrtValue. + * This will allocate required memory and copy the supplied NNZ values and BlockSparse indices into that memory allocation. + * Memory allocation is performed using the allocator that was specified with OrtApi::CreateSparseTensorAsOrtValue. + * + * \param[in,out] ort_value ::OrtValue to populate with data + * \param[in] data_mem_info serves to identify the location of the data to be copied. If the allocator specified + * at the creation time has memory info that is not the same as mem_info argument to this function a X-device copy will be performed. + * String data is assumed to be on CPU and will only be copied into a CPU allocated buffer. + * \param[in] values_shape + * \param[in] values_shape_len + * \param[in] values structure with values information + * \param[in] indices_shape_data pointer to a location of indices shape + * \param[in] indices_shape_len length of the block sparse indices shape + * \param[in] indices_data pointer to a location of indices data. Shape will determine the length of the indices data. + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(FillSparseTensorBlockSparse, _Inout_ OrtValue* ort_value, _In_ const OrtMemoryInfo* data_mem_info, + _In_ const int64_t* values_shape, size_t values_shape_len, _In_ const void* values, + _In_ const int64_t* indices_shape_data, size_t indices_shape_len, + _In_ const int32_t* indices_data); + + /** + * Create an ::OrtValue with a sparse tensor. This is the first step. + * Next, use UseIndices() functions to supply sparse tensor with + * format specific indices data and set its sparse format to a specific enum value. + * This will not perform memory allocations. It will + * use supplied user buffer which should outlive the created sparse tensor. + * Use OrtApi::ReleaseValue to destroy the sparse tensor. It would not release the supplied values buffer. + * This function can not be used to map strings from the user allocated memory. Strings must always be copied + * and have UTF-8 encoding. Therefore, use OrtApi::CreateSparseTensorAsOrtValue above and then fill it with data + * using appropriate Make*() function. + * + * \param[in] info memory info where sparse values reside. + * \param[in,out] p_data pointer to a user allocated buffer with values. To create a full sparse tensor with no non-zero + * values, pass nullptr + * \param[in] dense_shape shape of the original dense tensor + * \param[in] dense_shape_len number of shape dimensions being passed + * \param[in] values_shape shape of the values data. To create a fully sparse tensor with no non-zero values, + * pass {0} shape. + * \param[in] values_shape_len number of values shape dimensions + * \param[in] type must be one of TENSOR_ELEMENT_DATA_TYPE_xxxx + * \param[out] out Should be freed by calling ReleaseValue + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(CreateSparseTensorWithValuesAsOrtValue, _In_ const OrtMemoryInfo* info, _Inout_ void* p_data, + _In_ const int64_t* dense_shape, size_t dense_shape_len, + _In_ const int64_t* values_shape, size_t values_shape_len, + ONNXTensorElementDataType type, _Outptr_ OrtValue** out); + + /** + * This assigns Coo format indices to the SparseTensor that was created by + * OrtApi::CreateSparseTensorWithValuesAsOrtValue above. It also sets OrtSparseFormat to + * ORT_SPARSE_COO. This will not allocate any additional memory for data. The life span of + * indices_data buffer should eclipse the life span of this ::OrtValue. + * + * \param[in,out] ort_value ::OrtValue instance constructed with OrtApi::CreateSparseTensorWithValuesAsOrtValue + * \param[in,out] indices_data pointer to a user pre-allocated buffer or nullptr for fully sparse tensors. + * \param[in] indices_num number of COO indices. Should either be 0 for fully sparse tensors, be equal + * to the number of nnz values specified to OrtApi::CreateSparseTensorWithValuesAsOrtValue for 1-D {nnz} indices or + * be twice as number of nnz values for a 2-D indices {nnz, 2} + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(UseCooIndices, _Inout_ OrtValue* ort_value, _Inout_ int64_t* indices_data, size_t indices_num); + + /** + * The assigns CSR format indices to the SparseTensor that was created by + * OrtApi::CreateSparseTensorWithValuesAsOrtValue above. It also sets OrtSparseFormat to + * ORT_SPARSE_CSRC. This will not allocate any additional memory for data. The life spans of + * inner_data and outer_data buffers should eclipse the life span of this ::OrtValue. + * + * \param[in,out] ort_value ::OrtValue instance constructed with OrtApi::CreateSparseTensorWithValuesAsOrtValue + * \param[in,out] inner_data pointer to a user pre-allocated buffer or nullptr for fully sparse tensors. + * \param[in] inner_num number of inner CSR indices. Should either be 0 for fully sparse tensors or be equal + * to the number of nnz values specified to OrtApi::CreateSparseTensorWithValuesAsOrtValue. + * \param[in,out] outer_data pointer to user pre-allocated buffer or nullptr for fully sparse tensors. + * \param[in] outer_num number of CSR outer indices. Should either be 0 for fully sparse tensors or + * equal to rows + 1 of the dense shape. + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(UseCsrIndices, _Inout_ OrtValue* ort_value, _Inout_ int64_t* inner_data, size_t inner_num, + _Inout_ int64_t* outer_data, size_t outer_num); + + /** + * The assigns BlockSparse format indices to the SparseTensor that was created by + * OrtApi::CreateSparseTensorWithValuesAsOrtValue above. It also sets OrtSparseFormat to + * ORT_SPARSE_BLOCK_SPARSE. This will not allocate any additional memory for data. The life span of + * indices_data buffer must eclipse the lifespan of this ::OrtValue. + * + * \param[in,out] ort_value OrtValue instance constructed with OrtApi::CreateSparseTensorWithValuesAsOrtValue + * \param[in] indices_shape pointer to indices shape. Use {0} for fully sparse tensors + * \param[in] indices_shape_len length of the indices shape + * \param[in,out] indices_data pointer to user pre-allocated buffer or nullptr for fully sparse tensors. + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(UseBlockSparseIndices, _Inout_ OrtValue* ort_value, const int64_t* indices_shape, size_t indices_shape_len, _Inout_ int32_t* indices_data); + + /** \brief Returns sparse tensor format enum iff a given ort value contains an instance of sparse tensor. + * + * \param[in] ort_value ::OrtValue that contains an instance of sparse tensor + * \param[out] out pointer to out parameter + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(GetSparseTensorFormat, _In_ const OrtValue* ort_value, _Out_ enum OrtSparseFormat* out); + + /** \brief Returns data type and shape of sparse tensor values (nnz) iff ::OrtValue contains a SparseTensor. + * + * \param[in] ort_value An ::OrtValue that contains a fully constructed sparse tensor + * \param[out] out Must be freed by OrtApi::ReleaseTensorTypeAndShapeInfo + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(GetSparseTensorValuesTypeAndShape, _In_ const OrtValue* ort_value, _Outptr_ OrtTensorTypeAndShapeInfo** out); + + /** \brief Returns numeric data for sparse tensor values (nnz). For string values use GetStringTensor*(). + * + * \param[in] ort_value an instance of ::OrtValue containing sparse tensor + * \param[out] out returns a pointer to values data. Do not attempt to free this ptr. + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(GetSparseTensorValues, _In_ const OrtValue* ort_value, _Outptr_ const void** out); + + /** \brief Returns data type, shape for the type of indices specified by indices_format. + * + * \param[in] ort_value ::OrtValue containing sparse tensor. + * \param[in] indices_format One of the indices formats. It is an error to request a format that the sparse + * tensor does not contain. + * \param[out] out an instance of ::OrtTensorTypeAndShapeInfo. Must be freed by OrtApi::ReleaseTensorTypeAndShapeInfo + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(GetSparseTensorIndicesTypeShape, _In_ const OrtValue* ort_value, enum OrtSparseIndicesFormat indices_format, _Outptr_ OrtTensorTypeAndShapeInfo** out); + + /** \brief Returns indices data for the type of the indices specified by indices_format + * + * \param[in] ort_value ::OrtValue containing sparse tensor. + * \param[in] indices_format One of the indices formats. It is an error to request a format that the sparse tensor does not contain. + * \param[out] num_indices Pointer to where the number of indices entries is returned + * \param[out] indices Returned pointer to the indices data. Do not free the returned pointer as it refers to internal data owned by the ::OrtValue + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(GetSparseTensorIndices, _In_ const OrtValue* ort_value, enum OrtSparseIndicesFormat indices_format, _Out_ size_t* num_indices, _Outptr_ const void** indices); + /// @} + /// \name OrtSessionOptions + /// @{ + + /** + * \brief Sets out to 1 iff an optional type OrtValue has an element, 0 otherwise (OrtValue is None) + * Use this API to find if the optional type OrtValue is None or not. + * If the optional type OrtValue is not None, use the OrtValue just like any other OrtValue. + * For example, if you get an OrtValue that corresponds to Optional(tensor) and + * if HasValue() returns true, use it as tensor and so on. + + * \param[in] value Input OrtValue. + * \param[out] out indicating if the input OrtValue contains data (1) or if it is a None (0) + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(HasValue, _In_ const OrtValue* value, _Out_ int* out); + + /// @} + /// \name OrtKernelContext + /// Custom operator APIs. + /// @{ + + /** \brief Used for custom operators, gets the GPU compute stream to use to launch the custom a GPU kernel + * \see ::OrtCustomOp + * \param[in] context OrtKernelContext instance + * \param[out] out Returns pointer to a GPU compute stream that can be used to launch the custom GPU kernel. + * If retrieving the GPU compute stream is not relevant (GPU not enabled in the build, kernel partitioned to + * some other EP), then a nullptr is returned as the output param. + * Do not free or mutate the returned pointer as it refers to internal data owned by the underlying session. + * Only use it for custom kernel launching. + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(KernelContext_GetGPUComputeStream, _In_ const OrtKernelContext* context, _Outptr_ void** out); + + /// @} + /// \name GetTensorMemoryInfo + /// @{ + /** \brief Returns a pointer to the ::OrtMemoryInfo of a Tensor + * \param[in] value ::OrtValue containing tensor. + * \param[out] mem_info ::OrtMemoryInfo of the tensor. Do NOT free the returned pointer. It is valid for the lifetime of the ::OrtValue + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(GetTensorMemoryInfo, _In_ const OrtValue* value, _Out_ const OrtMemoryInfo** mem_info); + + /// @} + /// \name GetExecutionProviderApi + /// @{ + /** \brief Get a pointer to the requested version of the Execution Provider specific + * API extensions to the OrtApi + * \param[in] provider_name The name of the execution provider name. Currently only the following + * values are supported: "DML". + * \param[in] version Must be ::ORT_API_VERSION. + * \param[out] provider_api A void pointer containing a reference to the execution provider versioned api structure. + * For example, the provider_api pointer can be cast to the OrtDmlApi* when the provider_name is "DML". + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(GetExecutionProviderApi, _In_ const char* provider_name, _In_ uint32_t version, _Outptr_ const void** provider_api); + + /// @} + + /// \name SessionOptions + /// @{ + /** \brief Set custom thread creation function + * + * \param[in] options Session options + * \param[in] ort_custom_create_thread_fn Custom thread creation function + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(SessionOptionsSetCustomCreateThreadFn, _Inout_ OrtSessionOptions* options, _In_ OrtCustomCreateThreadFn ort_custom_create_thread_fn); + + /** \brief Set creation options for custom thread + * + * \param[in] options Session options + * \param[in] ort_custom_thread_creation_options Custom thread creation options (can be nullptr) + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(SessionOptionsSetCustomThreadCreationOptions, _Inout_ OrtSessionOptions* options, _In_ void* ort_custom_thread_creation_options); + + /** \brief Set custom thread join function + * + * \param[in] options Session options + * \param[in] ort_custom_join_thread_fn Custom join thread function, must not be nullptr when ort_custom_create_thread_fn is set + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(SessionOptionsSetCustomJoinThreadFn, _Inout_ OrtSessionOptions* options, _In_ OrtCustomJoinThreadFn ort_custom_join_thread_fn); + /// @} + + /// \name OrtThreadingOptions + /// @{ + /** \brief Set custom thread creation function for global thread pools + * + * \param[inout] tp_options + * \param[in] ort_custom_create_thread_fn Custom thread creation function + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(SetGlobalCustomCreateThreadFn, _Inout_ OrtThreadingOptions* tp_options, _In_ OrtCustomCreateThreadFn ort_custom_create_thread_fn); + + /** \brief Set custom thread creation options for global thread pools + * + * \param[inout] tp_options + * \param[in] ort_custom_thread_creation_options Custom thread creation options (can be nullptr) + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(SetGlobalCustomThreadCreationOptions, _Inout_ OrtThreadingOptions* tp_options, _In_ void* ort_custom_thread_creation_options); + + /** \brief Set custom thread join function for global thread pools + * + * \param[inout] tp_options + * \param[in] ort_custom_join_thread_fn Custom thread join function, must not be nullptr when global ort_custom_create_thread_fn is set + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(SetGlobalCustomJoinThreadFn, _Inout_ OrtThreadingOptions* tp_options, _In_ OrtCustomJoinThreadFn ort_custom_join_thread_fn); + /// @} + + /** \brief Synchronize bound inputs. The call may be necessary for some providers, such as cuda, + * in case the system that allocated bound memory operated on a different stream. However, the + * operation is provider specific and could be a no-op. + * + * \param[inout] binding_ptr + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(SynchronizeBoundInputs, _Inout_ OrtIoBinding* binding_ptr); + + /** \brief Synchronize bound outputs. The call may be necessary for some providers, such as cuda, + * in case the system that allocated bound memory operated on a different stream. However, the + * operation is provider specific and could be a no-op. + * + * \param[inout] binding_ptr + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(SynchronizeBoundOutputs, _Inout_ OrtIoBinding* binding_ptr); + + /// \name OrtSessionOptions + /// @{ + + /** \brief Append CUDA execution provider to the session options + * + * If CUDA is not available (due to a non CUDA enabled build), this function will return failure. + * + * This is slightly different from OrtApi::SessionOptionsAppendExecutionProvider_CUDA, it takes an + * ::OrtCUDAProviderOptions which is publicly defined. This takes an opaque ::OrtCUDAProviderOptionsV2 + * which must be created with OrtApi::CreateCUDAProviderOptions. + * + * For OrtApi::SessionOptionsAppendExecutionProvider_CUDA, the user needs to instantiate ::OrtCUDAProviderOptions + * as well as allocate/release buffers for some members of ::OrtCUDAProviderOptions. + * Here, OrtApi::CreateCUDAProviderOptions and Ortapi::ReleaseCUDAProviderOptions will do the memory management for you. + * + * \param[in] options + * \param[in] cuda_options + * + * \snippet{doc} snippets.dox OrtStatus Return Value + * + * \since Version 1.11. + */ + ORT_API2_STATUS(SessionOptionsAppendExecutionProvider_CUDA_V2, + _In_ OrtSessionOptions* options, _In_ const OrtCUDAProviderOptionsV2* cuda_options); + + /// @} + /// \name OrtCUDAProviderOptionsV2 + /// @{ + + /** \brief Create an OrtCUDAProviderOptionsV2 + * + * \param[out] out Newly created ::OrtCUDAProviderOptionsV2. Must be released with OrtApi::ReleaseCudaProviderOptions + * + * \snippet{doc} snippets.dox OrtStatus Return Value + * + * \since Version 1.11. + */ + ORT_API2_STATUS(CreateCUDAProviderOptions, _Outptr_ OrtCUDAProviderOptionsV2** out); + + /** \brief Set options in a CUDA Execution Provider. + * + * Please refer to https://onnxruntime.ai/docs/execution-providers/CUDA-ExecutionProvider.html#configuration-options + * to know the available keys and values. Key should be in null terminated string format of the member of ::OrtCUDAProviderOptionsV2 + * and value should be its related range. Recreates the options and only sets the supplied values. + * + * For example, key="device_id" and value="0" + * + * \param[in] cuda_options + * \param[in] provider_options_keys Array of UTF-8 null-terminated string for provider options keys + * \param[in] provider_options_values Array of UTF-8 null-terminated string for provider options values + * \param[in] num_keys Number of elements in the `provider_option_keys` and `provider_options_values` arrays + * + * \snippet{doc} snippets.dox OrtStatus Return Value + * + * \since Version 1.11. + */ + ORT_API2_STATUS(UpdateCUDAProviderOptions, _Inout_ OrtCUDAProviderOptionsV2* cuda_options, + _In_reads_(num_keys) const char* const* provider_options_keys, + _In_reads_(num_keys) const char* const* provider_options_values, + _In_ size_t num_keys); + + /** + * Get serialized CUDA provider options string. + * + * For example, "device_id=0;arena_extend_strategy=0;......" + * + * \param cuda_options - OrtCUDAProviderOptionsV2 instance + * \param allocator - a ptr to an instance of OrtAllocator obtained with CreateAllocator() or GetAllocatorWithDefaultOptions() + * the specified allocator will be used to allocate continuous buffers for output strings and lengths. + * \param ptr - is a UTF-8 null terminated string allocated using 'allocator'. The caller is responsible for using the same allocator to free it. + * + * \snippet{doc} snippets.dox OrtStatus Return Value + * + * \since Version 1.11. + */ + ORT_API2_STATUS(GetCUDAProviderOptionsAsString, _In_ const OrtCUDAProviderOptionsV2* cuda_options, _Inout_ OrtAllocator* allocator, _Outptr_ char** ptr); + + /** \brief Release an ::OrtCUDAProviderOptionsV2 + * + * \note This is an exception in the naming convention of other Release* functions, as the name of the method does not have the V2 suffix, but the type does + * + * \since Version 1.11. + */ + void(ORT_API_CALL* ReleaseCUDAProviderOptions)(_Frees_ptr_opt_ OrtCUDAProviderOptionsV2* input); + + /// @} + + /** \brief Append MIGraphX provider to session options + * + * If MIGraphX is not available (due to a non MIGraphX enabled build, or if MIGraphX is not installed on the system), this function will return failure. + * + * \param[in] options + * \param[in] migraphx_options + * + * \snippet{doc} snippets.dox OrtStatus Return Value + * + * \since Version 1.11. + */ + ORT_API2_STATUS(SessionOptionsAppendExecutionProvider_MIGraphX, + _In_ OrtSessionOptions* options, _In_ const OrtMIGraphXProviderOptions* migraphx_options); + + /** \brief Replace initialized Tensors with external data with the data provided in initializers. + * + * The function will find the initialized TensorProtos with external data in the graph with the provided names and + * replace them with the provided tensors. The API verifies that the TensorProto being replaced + * has an external data reference and has the same name, dimensions and data type as its replacement. The replacement + * will occur before any of the optimizations take place. The data will be copied into the graph + * since TensorProto can't refer to the user provided buffers. + * + * Once the model has been loaded, the OrtValue(s) added to SessionOptions instance will be removed + * from the internal SessionOptions copy to save memory, the user provided buffers can then be deallocated + * and the SessionOptions instance that refers to them can be destroyed. + * + * \param[in] options + * \param[in] initializer_names Array of null terminated UTF-8 encoded strings of the initializers names. + * \param[in] initializers Array of ::OrtValue type + * \param[in] num_initializers Number of elements in the initializer_names and initializers + * + * \snippet{doc} snippets.dox OrtStatus Return Value + * + * \since Version 1.12. + */ + ORT_API2_STATUS(AddExternalInitializers, _In_ OrtSessionOptions* options, + _In_reads_(num_initializers) const char* const* initializer_names, + _In_reads_(num_initializers) const OrtValue* const* initializers, size_t num_initializers); + + /** \brief: Create attribute of onnxruntime operator + * + * \param[in] name Name of the attribute + * \param[in] data Data content of the attribute + * \param[in] len Number of bytes stored in data + * \param[in] type Data type + * \param[out] op_attr Attribute that has been created, which must be released by OrtApi::ReleaseOpAttr + * + * \since Version 1.12. + */ + ORT_API2_STATUS(CreateOpAttr, + _In_ const char* name, + _In_ const void* data, + _In_ int len, + _In_ OrtOpAttrType type, + _Outptr_ OrtOpAttr** op_attr); + + /* \brief: Release op attribute + * + * \param[in] opAttr Attribute created by OrtApi::CreateOpAttr + * + * \since Version 1.12. + */ + ORT_CLASS_RELEASE(OpAttr); + + /** \brief: Create onnxruntime native operator + * + * \param[in] info Kernel info + * \param[in] op_name Operator name + * \param[in] domain Operator domain + * \param[in] version Operator opset version + * \param[in] type_constraint_names Name of the type contraints, such as "T" or "T1" + * \param[in] type_constraint_values Type of each contraints + * \param[in] type_constraint_count Number of contraints + * \param[in] attr_values Attributes used to initialize the operator + * \param[in] attr_count Number of the attributes + * \param[in] input_count Number of inputs + * \param[in] output_count Number of outputs + * \param[out] ort_op Operator that has been created + * + * \since Version 1.12. + */ + ORT_API2_STATUS(CreateOp, + _In_ const OrtKernelInfo* info, + _In_z_ const char* op_name, + _In_z_ const char* domain, + int version, + _In_reads_(type_constraint_count) const char** type_constraint_names, + _In_reads_(type_constraint_count) const ONNXTensorElementDataType* type_constraint_values, + int type_constraint_count, + _In_reads_(attr_count) const OrtOpAttr* const* attr_values, + int attr_count, + int input_count, + int output_count, + _Outptr_ OrtOp** ort_op); + + /** \brief: Invoke the operator created by OrtApi::CreateOp + * The inputs must follow the order as specified in onnx specification + * + * \param[in] context Kernel context + * \param[in] ort_op Operator that has been created + * \param[in] input_values Array of inputs + * \param[in] input_count Number of inputs + * \param[in] output_values Array of outputs + * \param[in] output_count Number of outputs + * + * \since Version 1.12. + */ + ORT_API2_STATUS(InvokeOp, + _In_ const OrtKernelContext* context, + _In_ const OrtOp* ort_op, + _In_ const OrtValue* const* input_values, + _In_ int input_count, + _Inout_ OrtValue* const* output_values, + _In_ int output_count); + + /* \brief: Release an onnxruntime operator + * + * \param[in] Op Operator created by OrtApi::CreateOp + * + * \since Version 1.12. + */ + ORT_CLASS_RELEASE(Op); + + /** \brief: Append execution provider to the session options. + * \param[in] options + * \param[in] provider_name - provider to add. + * \param[in] provider_options_keys - keys to configure the provider options + * \param[in] provider_options_values - values to configure the provider options + * \param[in] num_keys - number of keys passed in + * + * Currently supported provider names: + * QNNExecutionProvider (or QNN) + * OpenVINOExecutionProvider (or OpenVINO) + * XnnpackExecutionProvider (or XNNPACK) + * WebNNExecutionProvider (or WEBNN) + * WebGpuExecutionProvider (or WebGPU) + * AzureExecutionProvider (or AZURE) + * JsExecutionProvider (or JS) + * VitisAIExecutionProvider (or VitisAI) + * CoreMLExecutionProvider (or CoreML) + * + * Note: If an execution provider has a dedicated SessionOptionsAppendExecutionProvider_ function + * that should be used to add it. + * + * QNN supported keys: + * "backend_type": Type of QNN backend. Specifies a backend path that is the associated QNN backend library file + * name. E.g., given backend type "htp", on Windows, the backend path would be "QnnHtp.dll", and on other + * platforms, it would be "libQnnHtp.so". Mutually exclusive with "backend_path". + * Available options: + * -# "cpu" + * -# "gpu" + * -# "htp": Default. + * -# "saver" + * "backend_path": File path to QNN backend library. Mutually exclusive with "backend_type". + * "profiling_level": QNN profiling level. + * Available options: + * -# "off": Default. + * -# "basic" + * -# "detailed" + * "profiling_file_path": QNN profiling file path if ETW not enabled. + * "rpc_control_latency": QNN RPC control latency. + * "vtcm_mb": QNN VTCM size in MB. default to 0(not set). + * "htp_performance_mode": QNN performance mode. + * Available options: + * -# "burst" + * -# "balanced" + * -# "default": Default. + * -# "high_performance" + * -# "high_power_saver" + * -# "low_balanced" + * -# "extreme_power_saver" + * -# "low_power_saver" + * -# "power_saver" + * -# "sustained_high_performance" + * "qnn_saver_path": File path to the QNN Saver backend library. If specified, QNN Saver will be enabled and will + * dump QNN API calls to disk for replay/debugging. QNN Saver produces incorrect model inference results and + * may alter model/EP partitioning. Use only for debugging. + * "qnn_context_priority": QNN context priority. + * Available options: + * -# "low" + * -# "normal": Default. + * -# "normal_high" + * -# "high" + * "htp_graph_finalization_optimization_mode": Set the optimization mode for graph finalization on the HTP backend. + * Available options: + * -# "0": Default. + * -# "1": Faster preparation time, less optimal graph. + * -# "2": Longer preparation time, more optimal graph. + * -# "3": Longest preparation time, most likely even more optimal graph. See QNN SDK documentation for specific + * details. + * "soc_model": The SoC model number. Refer to the QNN SDK documentation for valid values. + * Defaults to "0" (unknown). + * "htp_arch": The minimum HTP architecture the driver will use to select compatible QNN operators. + * Available options: + * -# "0": Default (none). + * -# "68" + * -# "69" + * -# "73" + * -# "75" + * "device_id": The ID of the device to use when setting 'htp_arch'. Defaults to "0" (for single device). + * "enable_htp_fp16_precision": Used for float32 model for HTP backend. + * Enable the float32 model to be inferenced with fp16 precision. Otherwise, it will be fp32 precision. + * -# "0": With fp32 precision. + * -# "1": Default. With fp16 precision. + * "offload_graph_io_quantization": Offload graph input quantization and graph output dequantization to another + * execution provider (typically CPU EP). + * -# "0": Disabled. QNN EP will handle quantization and dequantization of graph I/O. + * -# "1": Enabled. This is the default value. + * "enable_htp_spill_fill_buffer": Enable HTP spill fill buffer setting. The flag is used while generating context + * binary. + * -# "0": Default. Disabled. + * -# "1": Enabled. + * "enable_htp_shared_memory_allocator": Enable the QNN HTP shared memory allocator. Requires libcdsprpc.so/dll to + * be available. + * -# "0": Default. Disabled. + * -# "1": Enabled. + * "dump_json_qnn_graph": Set to "1" to dump QNN graphs generated by QNN EP as JSON files. Each graph partition + * assigned to QNN EP is dumped to a separate file. + * "json_qnn_graph_dir": Directory in which to dump QNN JSON graphs. If not specified, QNN graphs are dumped in the + * program's current working directory. Ignored if "dump_json_qnn_graph" is not set. + * + * XNNPACK supported keys: + * "intra_op_num_threads": number of thread-pool size to use for XNNPACK execution provider. + * default value is 0, which means to use the session thread-pool size. + * + * \since Version 1.12. + */ + ORT_API2_STATUS(SessionOptionsAppendExecutionProvider, _In_ OrtSessionOptions* options, + _In_ const char* provider_name, + _In_reads_(num_keys) const char* const* provider_options_keys, + _In_reads_(num_keys) const char* const* provider_options_values, + _In_ size_t num_keys); + + /* \brief: Get a copy of kernel info + * + * \param[in] info Kernel info + * \param[out] info_copy Copy of kernel info + * + * \since Version 1.12. + */ + ORT_API2_STATUS(CopyKernelInfo, + _In_ const OrtKernelInfo* info, + _Outptr_ OrtKernelInfo** info_copy); + + /* \brief: Release kernel info + * + * \param[in] KernelInfo A copy of kernel info returned by CopyKernelInfo + * + * \since Version 1.12. + */ + ORT_CLASS_RELEASE(KernelInfo); + + /// \name Ort Training + /// @{ + /** \brief Gets the Training C Api struct + * + * Call this function to access the ::OrtTrainingApi structure that holds pointers to functions that enable + * training with onnxruntime. + * \note A NULL pointer will be returned and no error message will be printed if the training api + * is not supported with this build. A NULL pointer will be returned and an error message will be + * printed if the provided version is unsupported, for example when using a runtime older than the + * version created with this header file. + * + * \param[in] version Must be ::ORT_API_VERSION + * \return The ::OrtTrainingApi struct for the version requested. + * + * \since Version 1.13 + */ + const OrtTrainingApi*(ORT_API_CALL* GetTrainingApi)(uint32_t version)NO_EXCEPTION; + + /// @} + + /** \brief Append CANN provider to session options + * + * If CANN is not available (due to a non CANN enabled build, or if CANN is not installed on the system), this function will return failure. + * + * \param[in] options + * \param[in] cann_options + * + * \snippet{doc} snippets.dox OrtStatus Return Value + * + * \since Version 1.13. + */ + ORT_API2_STATUS(SessionOptionsAppendExecutionProvider_CANN, + _In_ OrtSessionOptions* options, _In_ const OrtCANNProviderOptions* cann_options); + + /** \brief Create an OrtCANNProviderOptions + * + * \param[out] out created ::OrtCANNProviderOptions. Must be released with OrtApi::ReleaseCANNProviderOptions + * + * \snippet{doc} snippets.dox OrtStatus Return Value + * + * \since Version 1.13. + */ + ORT_API2_STATUS(CreateCANNProviderOptions, _Outptr_ OrtCANNProviderOptions** out); + + /** \brief Set options in a CANN Execution Provider. + * + * \param[in] cann_options + * \param[in] provider_options_keys Array of UTF-8 null-terminated string for provider options keys + * \param[in] provider_options_values Array of UTF-8 null-terminated string for provider options values + * \param[in] num_keys Number of elements in the `provider_option_keys` and `provider_options_values` arrays + * + * \snippet{doc} snippets.dox OrtStatus Return Value + * + * \since Version 1.13. + */ + ORT_API2_STATUS(UpdateCANNProviderOptions, _Inout_ OrtCANNProviderOptions* cann_options, + _In_reads_(num_keys) const char* const* provider_options_keys, + _In_reads_(num_keys) const char* const* provider_options_values, + _In_ size_t num_keys); + + /** \brief Get serialized CANN provider options string. + * + * \param[in] cann_options OrtCANNProviderOptions instance + * \param[in] allocator a ptr to an instance of OrtAllocator obtained with CreateAllocator() + * or GetAllocatorWithDefaultOptions(), the specified allocator will be used to allocate + * continuous buffers for output strings and lengths. + * \param[out] ptr is a UTF-8 null terminated string allocated using 'allocator'. + * The caller is responsible for using the same allocator to free it. + * + * \snippet{doc} snippets.dox OrtStatus Return Value + * + * \since Version 1.13. + */ + ORT_API2_STATUS(GetCANNProviderOptionsAsString, _In_ const OrtCANNProviderOptions* cann_options, + _Inout_ OrtAllocator* allocator, _Outptr_ char** ptr); + + /** \brief Release an OrtCANNProviderOptions + * + * \param[in] input The pointer of OrtCANNProviderOptions which will been deleted + * + * \since Version 1.13. + */ + void(ORT_API_CALL* ReleaseCANNProviderOptions)(_Frees_ptr_opt_ OrtCANNProviderOptions* input); + + /* \brief Get OrtDevice type from MemoryInfo + * + * \since Version 1.14 + */ + void(ORT_API_CALL* MemoryInfoGetDeviceType)(_In_ const OrtMemoryInfo* ptr, _Out_ OrtMemoryInfoDeviceType* out); + + /* \brief Update the OrtEnv instance with custom log severity level + * + * \param[in] ort_env The OrtEnv instance being used + * \param[in] log_severity_level The log severity level. + * + * \since Version 1.14. + */ + ORT_API2_STATUS(UpdateEnvWithCustomLogLevel, _In_ OrtEnv* ort_env, OrtLoggingLevel log_severity_level); + + /* \brief Set affinities for intra op threads + * + * Affinity string follows format: + * logical_processor_id,logical_processor_id;logical_processor_id,logical_processor_id + * Semicolon isolates configurations among threads, while comma split processors where ith thread expected to attach to. + * e.g. 1,2,3;4,5 + * specifies affinities for two threads, with the 1st thread attach to the 1st, 2nd, and 3rd processor, and 2nd thread to the 4th and 5th. + * To ease the configuration, an "interval" is also allowed: + * e.g. 1-8;8-16;17-24 + * orders that the 1st thread runs on first eight processors, 2nd thread runs on next eight processors, and so forth. + * Note: + * 1. Once set, the number of thread affinities must equal to intra_op_num_threads - 1, + * ort does not set affinity on the main thread which is started and managed by the calling app; + * 2. For windows, ort will infer the group id from a logical processor id, for example, assuming there are two groups with each has 64 logical processors, + * an id of 64 will be inferred as the last processor of the 1st group, while 65 will be interpreted as the 1st processor of the second group. + * Hence 64-65 is an invalid configuration, because a windows thread cannot be attached to processors across group boundary. + * + * \since Version 1.14 + */ + ORT_API2_STATUS(SetGlobalIntraOpThreadAffinity, _Inout_ OrtThreadingOptions* tp_options, const char* affinity_string); + + /** \brief Register custom ops from a shared library. + * + * Loads a shared library (.dll on windows, .so on linux, etc) named 'library_name' and looks for this entry point: + * OrtStatus* RegisterCustomOps(OrtSessionOptions * options, const OrtApiBase* api); + * It then passes in the provided session options to this function along with the api base. + * + * The handle to the loaded library is automatically released by ORT when the last OrtSession that references the + * library handle is released. If no OrtSession is created, then the library handle is released when the provided + * OrtSessionOptions is released. + * + * \param[in] options The session options. + * \param[in] library_name The name of the shared library to load and register. Refer to OS-specific dynamic library + * loading utilities (e.g., LoadLibraryEx on Windows or dlopen on Linux/MacOS) for information + * on the format of library names and search paths. + * + * \snippet{doc} snippets.dox OrtStatus Return Value + * \since Version 1.14 + */ + ORT_API2_STATUS(RegisterCustomOpsLibrary_V2, _Inout_ OrtSessionOptions* options, _In_ const ORTCHAR_T* library_name); + + /** \brief Register custom ops by calling a RegisterCustomOpsFn function. + * + * Searches for registration_func_name and if found calls it. + * + * The library containing the function must either be linked against or previously loaded by the executable. + * + * If you want ONNX Runtime to load the library and manage its lifetime, use RegisterCustomOpsLibrary_V2. + * + * RegisterCustomOpsUsingFunction can be used in scenarios where it may not be possible for ONNX Runtime to load + * the library from a path. e.g. mobile platforms where the library must be linked into the app. + * + * The registration function must have the signature of RegisterCustomOpsFn: + * OrtStatus* (*fn)(OrtSessionOptions* options, const OrtApiBase* api); + * + * See https://onnxruntime.ai/docs/reference/operators/add-custom-op.html for details on how the registration + * function should be implemented. + * + * \param[in] options OrtSessionOptions that is passed through as the first argument in the call to the + * registration function. + * \param[in] registration_func_name Name of registration function to use. + * + * \snippet{doc} snippets.dox OrtStatus Return Value + * \since Version 1.14 + */ + ORT_API2_STATUS(RegisterCustomOpsUsingFunction, _Inout_ OrtSessionOptions* options, + _In_ const char* registration_func_name); + + /// \name OrtKernelInfo + /// Custom operator APIs. + /// @{ + + /** \brief Get the number of inputs from ::OrtKernelInfo. + * + * Used in the CreateKernel callback of an OrtCustomOp to query the number of inputs + * during kernel/session creation. + * + * \param[in] info Instance of ::OrtKernelInfo. + * \param[out] out Pointer to variable assigned with the result on success. + * + * \snippet{doc} snippets.dox OrtStatus Return Value + * \since Version 1.14 + */ + ORT_API2_STATUS(KernelInfo_GetInputCount, _In_ const OrtKernelInfo* info, _Out_ size_t* out); + + /** \brief Get the number of outputs from ::OrtKernelInfo. + * + * Used in the CreateKernel callback of an OrtCustomOp to query the number of outputs + * during kernel/session creation. + * + * \param[in] info Instance of ::OrtKernelInfo. + * \param[out] out Pointer to variable assigned with the result on success. + * + * \snippet{doc} snippets.dox OrtStatus Return Value + * \since Version 1.14 + */ + ORT_API2_STATUS(KernelInfo_GetOutputCount, _In_ const OrtKernelInfo* info, _Out_ size_t* out); + + /** \brief Get the name of a ::OrtKernelInfo's input. + * + * Used in the CreateKernel callback of an OrtCustomOp to query an input's name + * during kernel/session creation. + * + * If `out` is nullptr, the value of `size` is set to the size of the name + * string (including null-terminator), and a success status is returned. + * + * If the `size` parameter is greater than or equal to the name string's size, + * the value of `size` is set to the true size of the string (including null-terminator), + * the provided memory is filled with the string's contents, and a success status is returned. + * + * If the `size` parameter is less than the actual string's size and `out` + * is not nullptr, the value of `size` is set to the true size of the string + * and a failure status is returned. + * + * \param[in] info An instance of ::OrtKernelInfo. + * \param[in] index The index of the input name to get. Returns a failure status if out-of-bounds. + * \param[out] out Memory location into which to write the UTF-8 null-terminated string representing the input's name. + * \param[in,out] size Pointer to the size of the `out` buffer. See above comments for details. + * + * \snippet{doc} snippets.dox OrtStatus Return Value + * \since Version 1.14 + */ + ORT_API2_STATUS(KernelInfo_GetInputName, _In_ const OrtKernelInfo* info, size_t index, _Out_ char* out, + _Inout_ size_t* size); + + /** \brief Get the name of a ::OrtKernelInfo's output. + * + * Used in the CreateKernel callback of an OrtCustomOp to query an output's name + * during kernel/session creation. + * + * If `out` is nullptr, the value of `size` is set to the size of the name + * string (including null-terminator), and a success status is returned. + * + * If the `size` parameter is greater than or equal to the name string's size, + * the value of `size` is set to the true size of the string (including null-terminator), + * the provided memory is filled with the string's contents, and a success status is returned. + * + * If the `size` parameter is less than the actual string's size and `out` + * is not nullptr, the value of `size` is set to the true size of the string + * and a failure status is returned. + * + * \param[in] info An instance of ::OrtKernelInfo. + * \param[in] index The index of the output name to get. Returns a failure status if out-of-bounds. + * \param[out] out Memory location into which to write the UTF-8 null-terminated string representing the output's + * name. + * \param[in,out] size Pointer to the size of the `out` buffer. See above comments for details. + * + * \snippet{doc} snippets.dox OrtStatus Return Value + * \since Version 1.14 + */ + ORT_API2_STATUS(KernelInfo_GetOutputName, _In_ const OrtKernelInfo* info, size_t index, _Out_ char* out, + _Inout_ size_t* size); + + /** \brief Get the type information for a ::OrtKernelInfo's input. + * + * Used in the CreateKernel callback of an OrtCustomOp to query the shape and type information + * of an input during kernel/session creation. + * + * \param[in] info An instance of ::OrtKernelInfo. + * \param[in] index Which input to get the type information for + * \param[out] type_info Pointer set to the resulting ::OrtTypeInfo. Must be freed with OrtApi::ReleaseTypeInfo. + * + * \snippet{doc} snippets.dox OrtStatus Return Value + * \since Version 1.14 + */ + ORT_API2_STATUS(KernelInfo_GetInputTypeInfo, _In_ const OrtKernelInfo* info, size_t index, + _Outptr_ OrtTypeInfo** type_info); + + /** \brief Get the type information for a ::OrtKernelInfo's output. + * + * Used in the CreateKernel callback of an OrtCustomOp to query the shape and type information + * of an output during kernel/session creation. + * + * \param[in] info An instance of ::OrtKernelInfo. + * \param[in] index Which input to get the type information for + * \param[out] type_info Pointer set to the resulting ::OrtTypeInfo. Must be freed with OrtApi::ReleaseTypeInfo. + * + * \snippet{doc} snippets.dox OrtStatus Return Value + * \since Version 1.14 + */ + ORT_API2_STATUS(KernelInfo_GetOutputTypeInfo, _In_ const OrtKernelInfo* info, size_t index, + _Outptr_ OrtTypeInfo** type_info); + + /** \brief Get a ::OrtValue tensor stored as an attribute in the graph node. + * + * Used in the CreateKernel callback of an OrtCustomOp to get a tensor attribute. + * + * \param[in] info ::OrtKernelInfo instance. + * \param[in] name UTF-8 null-terminated string representing the attribute's name. + * \param[in] allocator Allocator used to allocate the internal tensor state. + * \param[out] out Returns newly created ::OrtValue. Must be freed with OrtApi::ReleaseValue, + * which will also free internal tensor state allocated with the provided allocator. + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(KernelInfoGetAttribute_tensor, _In_ const OrtKernelInfo* info, _In_z_ const char* name, + _Inout_ OrtAllocator* allocator, _Outptr_ OrtValue** out); + + /// @} + /// \name OrtSessionOptions + /// Custom operator APIs + /// @{ + + /** \brief Checks if the given session configuration entry exists. + * + * The config_key formats are defined in onnxruntime_session_options_config_keys.h + * + * Can be used in a custom operator library to check for session configuration entries + * that target one or more custom operators in the library. Example: The config entry + * custom_op.myop.some_key targets a custom op named "myop". + * + * \param[in] options The ::OrtSessionOptions instance. + * \param[in] config_key A null-terminated UTF-8 string representation of the configuration key. + * \param[out] out Pointer set to 1 if the entry exists and 0 otherwise. + * + * \snippet{doc} snippets.dox OrtStatus Return Value + * \since Version 1.14 + */ + ORT_API2_STATUS(HasSessionConfigEntry, _In_ const OrtSessionOptions* options, + _In_z_ const char* config_key, _Out_ int* out); + + /** \brief Get a session configuration value. + * + * Returns a failure status if the configuration key does not exist. + * The config_key and the format of config_value are defined in onnxruntime_session_options_config_keys.h + * + * If `config_value` is nullptr, the value of `size` is set to the true size of the string + * value (including null-terminator), and a success status is returned. + * + * If the `size` parameter is greater than or equal to the actual string value's size, + * the value of `size` is set to the true size of the string value, the provided memory + * is filled with the value's contents, and a success status is returned. + * + * If the `size` parameter is less than the actual string value's size and `config_value` + * is not nullptr, the value of `size` is set to the true size of the string value + * and a failure status is returned. + * + * Can be used in a custom operator library to get session configuration entries + * that target one or more custom operators in the library. Example: The config entry + * custom_op.myop.some_key targets a custom op named "myop". + * + * \param[in] options The session options. + * \param[in] config_key A null-terminated UTF-8 string representation of the config key. + * \param[in] config_value Pointer to memory where the null-terminated UTF-8 string value will be stored. + * \param[in,out] size Pointer to the size of the `config_value` buffer. See above comments for details. + * + * \snippet{doc} snippets.dox OrtStatus Return Value + * \since Version 1.14 + */ + ORT_API2_STATUS(GetSessionConfigEntry, _In_ const OrtSessionOptions* options, + _In_z_ const char* config_key, _Out_ char* config_value, _Inout_ size_t* size); + + /// @} + + /** \brief Append dnnl provider to session options + * + * If oneDNN is not available, this function will return failure. + * + * \param[in] options + * \param[in] dnnl_options + * + * \snippet{doc} snippets.dox OrtStatus Return Value + * + * \since Version 1.15. + */ + ORT_API2_STATUS(SessionOptionsAppendExecutionProvider_Dnnl, + _In_ OrtSessionOptions* options, _In_ const OrtDnnlProviderOptions* dnnl_options); + + /** \brief Create an OrtDnnlProviderOptions + * + * \param[out] out Newly created ::OrtDnnlProviderOptions. Must be released with OrtApi::ReleaseDnnlProviderOptions + * + * \snippet{doc} snippets.dox OrtStatus Return Value + * + * \since Version 1.15. + */ + ORT_API2_STATUS(CreateDnnlProviderOptions, _Outptr_ OrtDnnlProviderOptions** out); + + /** \brief Set options in a oneDNN Execution Provider. + * + * Key should be in null terminated string format of the member of ::OrtDnnlProviderOptions + * and value should be its related range. + * + * For example, key="use_arena" and value="1" + * + * \param[in] dnnl_options + * \param[in] provider_options_keys Array of UTF-8 null-terminated string for provider options keys + * \param[in] provider_options_values Array of UTF-8 null-terminated string for provider options values + * \param[in] num_keys Number of elements in the `provider_option_keys` and `provider_options_values` arrays + * + * \snippet{doc} snippets.dox OrtStatus Return Value + * + * \since Version 1.15. + */ + ORT_API2_STATUS(UpdateDnnlProviderOptions, _Inout_ OrtDnnlProviderOptions* dnnl_options, + _In_reads_(num_keys) const char* const* provider_options_keys, + _In_reads_(num_keys) const char* const* provider_options_values, + _In_ size_t num_keys); + + /** + * Get serialized oneDNN provider options string. + * + * For example, "use_arena=1;......" + * + * \param dnnl_options - OrtDnnlProviderOptions instance + * \param allocator - a ptr to an instance of OrtAllocator obtained with CreateAllocator() or GetAllocatorWithDefaultOptions() + * the specified allocator will be used to allocate continuous buffers for output strings and lengths. + * \param ptr - is a UTF-8 null terminated string allocated using 'allocator'. The caller is responsible for using the same allocator to free it. + * + * \snippet{doc} snippets.dox OrtStatus Return Value + * + * \since Version 1.15. + */ + ORT_API2_STATUS(GetDnnlProviderOptionsAsString, _In_ const OrtDnnlProviderOptions* dnnl_options, _Inout_ OrtAllocator* allocator, _Outptr_ char** ptr); + + /** \brief Release an ::OrtDnnlProviderOptions + * + * \since Version 1.15. + */ + void(ORT_API_CALL* ReleaseDnnlProviderOptions)(_Frees_ptr_opt_ OrtDnnlProviderOptions* input); + + /// \name OrtKernelInfo + /// Custom operator APIs. + /// @{ + + /** \brief Get the graph node name from ::OrtKernelInfo. + * + * If `out` is nullptr, the value of `size` is set to the size of the name + * string (including null-terminator), and a success status is returned. + * + * If the `size` parameter is greater than or equal to the name string's size, + * the value of `size` is set to the true size of the string (including null-terminator), + * the provided memory is filled with the string's contents, and a success status is returned. + * + * If the `size` parameter is less than the actual string's size and `out` + * is not nullptr, the value of `size` is set to the true size of the string + * and a failure status is returned. + * + * Can be used in a custom operator's CreateKernel callback to get the name of the operator's node name in the graph. + * + * \param[in] info An instance of ::OrtKernelInfo. + * \param[out] out Memory location into which to write the UTF-8 null-terminated string representing the name. + * \param[in,out] size Pointer to the size of the `out` buffer. See above comments for details. + * + * \snippet{doc} snippets.dox OrtStatus Return Value + * \since Version 1.15 + */ + ORT_API2_STATUS(KernelInfo_GetNodeName, _In_ const OrtKernelInfo* info, _Out_ char* out, _Inout_ size_t* size); + + /** \brief Get the session logger from ::OrtKernelInfo. + * + * Used in the CreateKernel callback of an OrtCustomOp to get a logger that can be used to log + * messages. + * + * \param[in] info An instance of ::OrtKernelInfo. + * \param[out] logger Pointer set to the session's ::OrtLogger. Owned by ONNX Runtime, so do not free. + * + * \snippet{doc} snippets.dox OrtStatus Return Value + * \since Version 1.15 + */ + ORT_API2_STATUS(KernelInfo_GetLogger, _In_ const OrtKernelInfo* info, _Outptr_ const OrtLogger** logger); + + /// @} + /// \name OrtKernelContext + /// Custom operator APIs. + /// @{ + + /** \brief Get the runtime logger from ::OrtKernelContext. + * + * Used in the KernelCompute callback of an OrtCustomOp to get a logger that can be used to log + * messages during inference. + * + * \param[in] context An instance of ::OrtKernelContext. + * \param[out] logger Pointer set to the kernel context's ::OrtLogger. Owned by ONNX Runtime, so do not free. + * + * \snippet{doc} snippets.dox OrtStatus Return Value + * \since Version 1.15 + */ + ORT_API2_STATUS(KernelContext_GetLogger, _In_ const OrtKernelContext* context, _Outptr_ const OrtLogger** logger); + + /// @} + /// \name OrtLogger + /// Custom operator APIs. + /// @{ + + /** \brief Logs a message at the given severity level using the provided ::OrtLogger. + * + * Only messages with a severity level equal or greater than the ::OrtLogger's logging severity level + * are logged. Use OrtApi::Logger_GetLoggingSeverityLevel to get the ::OrtLogger's logging severity + * level. + * + * Can be used in custom operators to log messages with the logger retrieved via OrtApi::KernelInfo_GetLogger. + * + * \param[in] logger The ::OrtLogger instance. + * \param[in] log_severity_level The message's severity level. + * \param[in] message The message to log. + * \param[in] file_path The filepath of the file in which the message is logged. Usually the value of ORT_FILE. + * \param[in] line_number The file line number in which the message is logged. Usually the value of __LINE__. + * \param[in] func_name The name of the function in which the message is logged. Usually the value of __FUNCTION__. + * + * \snippet{doc} snippets.dox OrtStatus Return Value + * \since Version 1.15 + */ + ORT_API2_STATUS(Logger_LogMessage, _In_ const OrtLogger* logger, OrtLoggingLevel log_severity_level, + _In_z_ const char* message, _In_z_ const ORTCHAR_T* file_path, int line_number, + _In_z_ const char* func_name); + + /** \brief Get the logging severity level of the ::OrtLogger. + * + * Can be used in a custom operator to get the logging serverity level of the ::OrtLogger associated with + * the ::OrtKernelInfo. + * + * \param[in] logger The ::OrtLogger instance. + * \param[out] out Pointer to variable assigned with the logging severity level on success. + * + * \snippet{doc} snippets.dox OrtStatus Return Value + * \since Version 1.15 + */ + ORT_API2_STATUS(Logger_GetLoggingSeverityLevel, _In_ const OrtLogger* logger, _Out_ OrtLoggingLevel* out); + + /// @} + + /** \brief Get a ::OrtValue tensor stored as a constant initializer in the graph node. + * + * Used in the CreateKernel callback of an OrtCustomOp to get a tensor value. + * + * \param[in] info ::OrtKernelInfo instance. + * \param[in] index The node index. + * \param[out] is_constant Is it a constant node input or not. + * \param[out] out The OrtValue tensor value. + * + * \snippet{doc} snippets.dox OrtStatus Return Value + * + * \since Version 1.15. + */ + ORT_API2_STATUS(KernelInfoGetConstantInput_tensor, _In_ const OrtKernelInfo* info, size_t index, _Out_ int* is_constant, _Outptr_ const OrtValue** out); + + /** \brief Get Optional Type information from an ::OrtTypeInfo + * + * This augments ::OrtTypeInfo to return an ::OrtOptionalTypeInfo when the type is optional. + * The OrtOptionalTypeInfo also has a nested ::OrtTypeInfo that describes the type of the optional value. + * ::OrtOptionalTypeInfo type can only appear within model metadata to describe inputs/outputs. + * The actual OrtValues that are supplied in place of optional type inputs should contain + * specific type that is described by ::OrtOptionalTypeInfo. + * + * So the picture: ::OrtTypeInfo -> ::OrtOptionalTypeInfo -> ::OrtTypeInfo (describes the type that can be supplied + * in place of the optional type when creating the actual ::OrtValue). + * + * \param[in] type_info + * \param[out] out A pointer to the ::OrtOptionalTypeInfo. Do not free this value, + * it is owned by OrtTypeInfo instance. When the type_info does not represent + * optional type, nullptr is returned in out. + * + * \snippet{doc} snippets.dox OrtStatus Return Value + * + * \since Version 1.15. + */ + ORT_API2_STATUS(CastTypeInfoToOptionalTypeInfo, _In_ const OrtTypeInfo* type_info, + _Outptr_result_maybenull_ const OrtOptionalTypeInfo** out); + + /** \brief Get OrtTypeInfo for the allowed contained type from an ::OrtOptionalTypeInfo. + * + * This augments ::OrtOptionalTypeInfo to return an ::OrtTypeInfo for the contained type. + * The OrtOptionalTypeInfo has a nested ::OrtTypeInfo that describes the type of the optional value. + * ::OrtOptionalTypeInfo type can only appear within model metadata to describe inputs/outputs. + * The actual OrtValues that are supplied in place of optional type inputs should contain + * specific type that is described by the returned ::OrtTypeInfo. + * + * \param[in] optional_type_info + * \param[out] out A copy of ::OrtTypeInfo for what the optional value could be. + * The user must free this value with ReleaseTypeInfo. + * + * \snippet{doc} snippets.dox OrtStatus Return Value + * + * \since Version 1.15. + */ + ORT_API2_STATUS(GetOptionalContainedTypeInfo, _In_ const OrtOptionalTypeInfo* optional_type_info, + _Outptr_ OrtTypeInfo** out); + + /** \brief Set a single string in a string tensor + * Do not zero terminate the string data. + * + * \param[in] value A string tensor + * \param[in] index - flat index of the element + * \param[in] length_in_bytes length of the buffer in utf-8 bytes (without the null terminator) + * \param[inout] buffer - address of return value + * + * \snippet{doc} snippets.dox OrtStatus Return Value + */ + ORT_API2_STATUS(GetResizedStringTensorElementBuffer, _Inout_ OrtValue* value, _In_ size_t index, _In_ size_t length_in_bytes, _Inout_ char** buffer); + + /** \brief Get Allocator from KernelContext for a specific memoryInfo. Please use C API ReleaseAllocator to release out object + * + * \param[in] context OrtKernelContext instance + * \param[in] mem_info OrtMemoryInfo instance + * \param[out] out A pointer to OrtAllocator. + * + * \snippet{doc} snippets.dox OrtStatus Return Value + * + * \since Version 1.15. + */ + ORT_API2_STATUS(KernelContext_GetAllocator, _In_ const OrtKernelContext* context, _In_ const OrtMemoryInfo* mem_info, _Outptr_ OrtAllocator** out); + + /** \brief Returns a null terminated string of the build info including git info and cxx flags + * + * \return UTF-8 encoded version string. Do not deallocate the returned buffer. + * + * \since Version 1.15. + */ + const char*(ORT_API_CALL* GetBuildInfoString)(void); + + /// \name OrtROCMProviderOptions + /// @{ + + /** \brief Create an OrtROCMProviderOptions + * + * \param[out] out Newly created ::OrtROCMProviderOptions. Must be released with OrtApi::ReleaseROCMProviderOptions + * + * \snippet{doc} snippets.dox OrtStatus Return Value + * + * \since Version 1.16. + */ + ORT_API2_STATUS(CreateROCMProviderOptions, _Outptr_ OrtROCMProviderOptions** out); + + /** \brief Set options in a ROCm Execution Provider. + * + * Please refer to https://onnxruntime.ai/docs/execution-providers/ROCm-ExecutionProvider.html + * to know the available keys and values. Key should be in null terminated string format of the member of + * ::OrtROCMProviderOptions and value should be its related range. + * + * For example, key="device_id" and value="0" + * + * \param[in] rocm_options + * \param[in] provider_options_keys Array of UTF-8 null-terminated string for provider options keys + * \param[in] provider_options_values Array of UTF-8 null-terminated string for provider options values + * \param[in] num_keys Number of elements in the `provider_option_keys` and `provider_options_values` arrays + * + * \snippet{doc} snippets.dox OrtStatus Return Value + * + * \since Version 1.16. + */ + ORT_API2_STATUS(UpdateROCMProviderOptions, _Inout_ OrtROCMProviderOptions* rocm_options, + _In_reads_(num_keys) const char* const* provider_options_keys, + _In_reads_(num_keys) const char* const* provider_options_values, + _In_ size_t num_keys); + + /** + * Get serialized ROCm provider options string. + * + * For example, "device_id=0;arena_extend_strategy=0;......" + * + * \param rocm_options - OrtROCMProviderOptions instance + * \param allocator - a ptr to an instance of OrtAllocator obtained with CreateAllocator() or GetAllocatorWithDefaultOptions() + * the specified allocator will be used to allocate continuous buffers for output strings and lengths. + * \param ptr - is a UTF-8 null terminated string allocated using 'allocator'. The caller is responsible for using the same allocator to free it. + * + * \snippet{doc} snippets.dox OrtStatus Return Value + * + * \since Version 1.16. + */ + ORT_API2_STATUS(GetROCMProviderOptionsAsString, _In_ const OrtROCMProviderOptions* rocm_options, _Inout_ OrtAllocator* allocator, _Outptr_ char** ptr); + + /** \brief Release an ::OrtROCMProviderOptions + * + * \note This is an exception in the naming convention of other Release* functions, as the name of the method does not have the V2 suffix, but the type does + * + * \since Version 1.16. + */ + void(ORT_API_CALL* ReleaseROCMProviderOptions)(_Frees_ptr_opt_ OrtROCMProviderOptions* input); + + /** \brief Create an allocator with specific type and register it with the ::OrtEnv + * This API enhance CreateAndRegisterAllocator that it can create an allocator with specific type, not just CPU allocator + * Enables sharing the allocator between multiple sessions that use the same env instance. + * Lifetime of the created allocator will be valid for the duration of the environment. + * Returns an error if an allocator with the same ::OrtMemoryInfo is already registered. + * \param[in] env OrtEnv instance + * \param[in] provider_type ExecutionProvider type + * \param[in] mem_info OrtMemoryInfo instance + * \param[in] arena_cfg Arena configuration + * \param[in] provider_options_keys key of the provider options map + * \param[in] provider_options_values value of the provider options map + * \param[in] num_keys Length of the provider options map + */ + ORT_API2_STATUS(CreateAndRegisterAllocatorV2, _Inout_ OrtEnv* env, _In_ const char* provider_type, _In_ const OrtMemoryInfo* mem_info, _In_ const OrtArenaCfg* arena_cfg, + _In_reads_(num_keys) const char* const* provider_options_keys, _In_reads_(num_keys) const char* const* provider_options_values, _In_ size_t num_keys); + + /** \brief Run the model asynchronously in a thread owned by intra op thread pool + * + * \param[in] session + * \param[in] run_options If nullptr, will use a default ::OrtRunOptions + * \param[in] input_names Array of null terminated UTF8 encoded strings of the input names + * \param[in] input Array of ::OrtValue%s of the input values + * \param[in] input_len Number of elements in the input_names and inputs arrays + * \param[in] output_names Array of null terminated UTF8 encoded strings of the output names + * \param[in] output_names_len Number of elements in the output_names and outputs array + * \param[out] output OrtValue* array of size output_names_len. + * On calling RunAsync, output[i] could either be a null or a pointer to a preallocated OrtValue. + * Later, the output array will be passed to run_async_callback with all null(s) filled with valid + * OrtValue pointer(s) allocated by onnxruntime. + * NOTE: it is customer's duty to finally release the output array and each of its member, + * regardless of whether the member (OrtValue*) is allocated by onnxruntime or preallocated by the customer. + * \param[in] run_async_callback Callback function on model run completion + * \param[in] user_data User data that pass back to run_async_callback + */ + ORT_API2_STATUS(RunAsync, _Inout_ OrtSession* session, _In_opt_ const OrtRunOptions* run_options, + _In_reads_(input_len) const char* const* input_names, + _In_reads_(input_len) const OrtValue* const* input, size_t input_len, + _In_reads_(output_names_len) const char* const* output_names, size_t output_names_len, + _Inout_updates_all_(output_names_len) OrtValue** output, + _In_ RunAsyncCallbackFn run_async_callback, _In_opt_ void* user_data); + + /** + * Update TensorRT EP provider option where its data type is pointer, for example 'user_compute_stream'. + * If the data type of the provider option can be represented by string please use UpdateTensorRTProviderOptions. + * + * Note: It's caller's responsibility to properly manage the lifetime of the instance pointed by this pointer. + * + * \param tensorrt_options - OrtTensorRTProviderOptionsV2 instance + * \param key - Name of the provider option + * \param value - A pointer to the instance that will be assigned to this provider option + * + * \since Version 1.16. + */ + ORT_API2_STATUS(UpdateTensorRTProviderOptionsWithValue, _Inout_ OrtTensorRTProviderOptionsV2* tensorrt_options, _In_ const char* key, _In_ void* value); + + /** + * Get TensorRT EP provider option where its data type is pointer. + * If the data type of the provider option can be represented by string please use GetTensorRTProviderOptionsAsString. + * + * \param tensorrt_options - OrtTensorRTProviderOptionsV2 instance + * \param key - Name of the provider option + * \param ptr - A pointer to the instance that is kept by the provider option + * + * \since Version 1.16. + */ + ORT_API2_STATUS(GetTensorRTProviderOptionsByName, _In_ const OrtTensorRTProviderOptionsV2* tensorrt_options, _In_ const char* key, _Outptr_ void** ptr); + + /** + * Update CUDA EP provider option where its data type is pointer, for example 'user_compute_stream'. + * If the data type of the provider option can be represented by string please use UpdateCUDAProviderOptions. + * + * Note: It's caller's responsibility to properly manage the lifetime of the instance pointed by this pointer. + * + * \param cuda_options - OrtCUDAProviderOptionsV2 instance + * \param key - Name of the provider option + * \param value - A pointer to the instance that will be assigned to this provider option + * + * \since Version 1.16. + */ + ORT_API2_STATUS(UpdateCUDAProviderOptionsWithValue, _Inout_ OrtCUDAProviderOptionsV2* cuda_options, _In_ const char* key, _In_ void* value); + + /** + * Get CUDA EP provider option where its data type is pointer. + * If the data type of the provider option can be represented by string please use GetCUDAProviderOptionsAsString. + * + * \param cuda_options - OrtCUDAProviderOptionsV2 instance + * \param key - Name of the provider option + * \param ptr - A pointer to the instance that is kept by the provider option + * + * \since Version 1.16. + */ + ORT_API2_STATUS(GetCUDAProviderOptionsByName, _In_ const OrtCUDAProviderOptionsV2* cuda_options, _In_ const char* key, _Outptr_ void** ptr); + + /** + * Get a EP resource. + * E.g. a cuda stream or a cublas handle + * + * \param context - Kernel context + * \param resource_version - Version of the resource + * \param resource_id - Type of resource + * \param resource - A pointer to returned resource + * + * \since Version 1.16. + */ + ORT_API2_STATUS(KernelContext_GetResource, _In_ const OrtKernelContext* context, _In_ int resource_version, + _In_ int resource_id, _Outptr_ void** resource); + + /** \brief Set user logging function + * + * By default the logger created by the CreateEnv* functions is used to create the session logger as well. + * This function allows a user to override this default session logger with a logger of their own choosing. This way + * the user doesn't have to create a separate environment with a custom logger. This addresses the problem when + * the user already created an env but now wants to use a different logger for a specific session (for debugging or + * other reasons). + * + * \param[in] options + * \param[in] user_logging_function A pointer to a logging function. + * \param[in] user_logging_param A pointer to arbitrary data passed as the ::OrtLoggingFunction `param` parameter to + * `user_logging_function`. This parameter is optional. + * + * \snippet{doc} snippets.dox OrtStatus Return Value + * + * \since Version 1.17. + */ + ORT_API2_STATUS(SetUserLoggingFunction, _Inout_ OrtSessionOptions* options, + _In_ OrtLoggingFunction user_logging_function, _In_opt_ void* user_logging_param); + + /** + * Get number of input from OrtShapeInferContext + * + * \param[in] context + * \param[out] out The number of inputs + * + * \since Version 1.17. + */ + ORT_API2_STATUS(ShapeInferContext_GetInputCount, _In_ const OrtShapeInferContext* context, _Out_ size_t* out); + + /** + * Get type and shape info of an input + * + * \param[in] context + * \param[in] index The index of the input + * \param[out] info Type shape info of the input + * + * \since Version 1.17. + */ + ORT_API2_STATUS(ShapeInferContext_GetInputTypeShape, _In_ const OrtShapeInferContext* context, _In_ size_t index, _Outptr_ OrtTensorTypeAndShapeInfo** info); + + /** + * Get attribute from OrtShapeInferContext. Note that OrtShapeInferContext is a per-node context, one could only read attribute from current node. + * + * \param[in] context + * \param[in] attr_name Name of the attribute + * \param[out] attr Handle of the attribute fetched + * + * \since Version 1.17. + */ + ORT_API2_STATUS(ShapeInferContext_GetAttribute, _In_ const OrtShapeInferContext* context, _In_ const char* attr_name, _Outptr_ const OrtOpAttr** attr); + + /** + * Set type and shape info of an output + * + * \param[in] context + * \param[in] index The index of the output + * \param[out] info Type shape info of the output + * + * \since Version 1.17. + */ + ORT_API2_STATUS(ShapeInferContext_SetOutputTypeShape, _In_ const OrtShapeInferContext* context, _In_ size_t index, _In_ const OrtTensorTypeAndShapeInfo* info); + + /** + * Set symbolic shape to type shape info + * + * \param[in] info Type shape info + * \param[in] dim_params Symbolic strings + * \param[in] dim_params_length Number of strings + * + * \since Version 1.17. + */ + ORT_API2_STATUS(SetSymbolicDimensions, _In_ OrtTensorTypeAndShapeInfo* info, _In_ const char* dim_params[], _In_ size_t dim_params_length); + + /** + * Read contents of an attribute to data + * + * \param[in] op_attr + * \param[in] type Attribute type + * \param[out] data Memory address to save raw content of the attribute + * \param[in] len Number of bytes allowed to store in data + * \param[out] out Number of bytes required to save the data when the call failed, or the real number of bytes saved to data on success + * + * \since Version 1.17. + */ + ORT_API2_STATUS(ReadOpAttr, _In_ const OrtOpAttr* op_attr, _In_ OrtOpAttrType type, _Inout_ void* data, _In_ size_t len, _Out_ size_t* out); + + /** \brief Set whether to use deterministic compute. + * + * Default is false. If set to true, this will enable deterministic compute for GPU kernels where possible. + * Note that this most likely will have a performance cost. + * + * \param[in] options + * \param[in] value + * + * \since Version 1.17. + */ + ORT_API2_STATUS(SetDeterministicCompute, _Inout_ OrtSessionOptions* options, bool value); + + /** + * Run fn in parallel + * + * \param[in] context + * \param[in] fn Function accepting usr_data and an integer as iterator + * \param[in] total The number of times fn is to be invoked + * \param[in] num_batch Number of batches by which the "total" is to be divided in maximum. When zero, there is no limit + * \param[in] usr_data User data to be passed back to fn + * + * \since Version 1.17. + */ + ORT_API2_STATUS(KernelContext_ParallelFor, _In_ const OrtKernelContext* context, _In_ void (*fn)(void*, size_t), _In_ size_t total, _In_ size_t num_batch, _In_ void* usr_data); + + /** \brief Append OpenVINO execution provider to the session options + * + * If OpenVINO is not available (due to a non OpenVINO enabled build, or if OpenVINO is not installed on the system), this function will fail. + * + * \param[in] options + * \param[in] provider_options_keys + * \param[in] provider_options_values + * \param[in] num_keys + * + * \snippet{doc} snippets.dox OrtStatus Return Value + * + * \since Version 1.17. + */ + ORT_API2_STATUS(SessionOptionsAppendExecutionProvider_OpenVINO_V2, + _In_ OrtSessionOptions* options, + _In_reads_(num_keys) const char* const* provider_options_keys, + _In_reads_(num_keys) const char* const* provider_options_values, + _In_ size_t num_keys); + + /** \brief Append VitisAI provider to session options + * + * If VitisAI is not available (due to a non VitisAI enabled build, or if VitisAI is not installed on the system), this function will return failure. + * + * \param[in] options + * \param[in] provider_options_keys + * \param[in] provider_options_values + * \param[in] num_keys + * + * \snippet{doc} snippets.dox OrtStatus Return Value + * + * \since Version 1.18. + */ + ORT_API2_STATUS(SessionOptionsAppendExecutionProvider_VitisAI, + _In_ OrtSessionOptions* options, + _In_reads_(num_keys) const char* const* provider_options_keys, + _In_reads_(num_keys) const char* const* provider_options_values, + _In_ size_t num_keys); + + /** \brief Get scratch buffer from the corresponding allocator under the sepcific OrtMemoryInfo object. + * NOTE: callers are responsible to release this scratch buffer from the corresponding allocator + * \param[in] context OrtKernelContext instance + * \param[in] mem_info OrtMemoryInfo instance + * \param[in] count_or_bytes How many bytes is this scratch buffer + * \param[out] out A pointer to the scrach buffer + * + * \snippet{doc} snippets.dox OrtStatus Return Value + * + * \since Version 1.18. + */ + ORT_API2_STATUS(KernelContext_GetScratchBuffer, _In_ const OrtKernelContext* context, _In_ const OrtMemoryInfo* mem_info, _In_ size_t count_or_bytes, _Outptr_ void** out); + + /** \brief Get allocator from KernelInfo for a specific memory type. Please use C API ReleaseAllocator to release out object + * + * \param[in] info OrtKernelInfo instance + * \param[in] mem_type OrtMemType object + * \param[out] out A pointer to OrtAllocator + * + * \snippet{doc} snippets.dox OrtStatus Return Value + * + * \since Version 1.18. + */ + ORT_API2_STATUS(KernelInfoGetAllocator, _In_ const OrtKernelInfo* info, _In_ OrtMemType mem_type, _Outptr_ OrtAllocator** out); + + /** \brief Replace initialized Tensors with external data with the provided files in memory + * + * The function will find the initialized TensorProtos with external data in the graph with the provided + * external file names and the file content in memory. The API gets the external file name, offset, data length + * from TensorProto, and locate the tensor data from the file in memory buffer. + * It creates a Tensor to replace the existing Tensor in graph. The replacement + * will occur before any of the optimizations take place. The data will be copied into the graph + * since TensorProto can't refer to the user provided buffers. + * + * \param[in] options + * \param[in] external_initializer_file_names Array of null terminated UTF-8 encoded strings of the file names + * which holds the external initializers. + * \param[in] external_initializer_file_buffer_array Array of pointers to the buffer of the file content. + * The buffer can be freed after session creation. + * \param[in] external_initializer_file_lengths Array of size_t to indicate the length of file content + * \param[in] num_external_initializer_files Number of external files + * + * \snippet{doc} snippets.dox OrtStatus Return Value + * + * \since Version 1.18. + */ + ORT_API2_STATUS(AddExternalInitializersFromFilesInMemory, _In_ OrtSessionOptions* options, + _In_reads_(num_external_initializer_files) const ORTCHAR_T* const* external_initializer_file_names, + _In_reads_(num_external_initializer_files) char* const* external_initializer_file_buffer_array, + _In_reads_(num_external_initializer_files) const size_t* external_initializer_file_lengths, + size_t num_external_initializer_files); + + /** \brief Create an OrtLoraAdapter + * + * The function attempts to locate file specified by adapter_file_path, read it and create an OrtLoraAdapter + * instance. The adapter_file_path should be a valid path to a file that contains a valid Lora Adapter + * format. The function attempts to validate the format at load time. The file will always be memory mapped, unless + * the platform does not support memory mapping, in which case the file will be read into memory. + * + * \param[in] adapter_file_path adapter file path. + * \param[in] allocator optional pointer to a device allocator. If specified + * data is copied to the device at some point before Run() is invoked. If nullptr, data stays on CPU. + * The data would still be copied to device if required by the model at inference time. + * \param[out] out A pointer to a newly created OrtLoraAdapter instance. Must be released with + * OrtApi::ReleaseLoraAdapter. + * + * \snippet{doc} snippets.dox OrtStatus Return Value + * + * \since Version 1.20. + */ + ORT_API2_STATUS(CreateLoraAdapter, const ORTCHAR_T* adapter_file_path, _In_ OrtAllocator* allocator, + _Outptr_ OrtLoraAdapter** out); + + /** \brief Create an OrtLoraAdapter + * + * The function copies the bytes from the array and creates an OrtLoraAdapter instance. + * + * + * \param[in] bytes pointer to a valid Lora Adapter format buffer. + * \param[in] num_bytes length of bytes buffer. + * \param[in] allocator optional pointer to a device allocator. If specified + * data is copied to the device at some point before Run() is invoked. If nullptr, data stays on CPU. + * The data would still be copied to device if required by the model at inference time. + * \param[out] out A pointer to a newly created OrtLoraAdapter instance. Must be released with + * OrtApi::ReleaseLoraAdapter. + * + * \snippet{doc} snippets.dox OrtStatus Return Value + * + * \since Version 1.20. + */ + ORT_API2_STATUS(CreateLoraAdapterFromArray, _In_ const void* bytes, size_t num_bytes, _In_ OrtAllocator* allocator, + _Outptr_ OrtLoraAdapter** out); + + /** \brief Release an ::OrtLoraAdapter obtained from OrtApi::CreateLoraAdapter + */ + ORT_CLASS_RELEASE(LoraAdapter); + + /** \brief Add the Lora Adapter to the list of active adapters. + * + * The function adds the Lora Adapter to the list of active adapters. The Lora Adapter must be created with + * OrtApi::CreateLoraAdapter or FromArray. The Lora Adapter will be used by the session to run the model. + * The instance of the OrtRunOptions can then be used to customize the Run() calls. + * More than one OrtLoraAdapter can be active at the same time. Lora Parameters that belong to different + * Lora adapters that will be active at the same time must not overlap. + * This setting does not affect RunWithBinding. + * + * \param[in] options OrtRunOptions instance + * \param[in] adapter OrtLoraAdapter instance + * + * \snippet{doc} snippets.dox OrtStatus Return Value + * + * \since Version 1.20. + */ + ORT_API2_STATUS(RunOptionsAddActiveLoraAdapter, _Inout_ OrtRunOptions* options, _In_ const OrtLoraAdapter* adapter); + + /// @} + /// \name OrtEpDynamicOptions + /// @{ + + /** \brief Set DynamicOptions for EPs (Execution Providers) + * + * Valid options can be found in `include\onnxruntime\core\session\onnxruntime_session_options_config_keys.h` + * Look for `kOrtEpDynamicOptions` + * + * \param[in] sess OrtSession + * \param[in] keys Array of null terminated UTF8 encoded strings of EP dynamic option keys + * \param[in] values Array of null terminated UTF8 encoded string of EP dynamic option values + * \param[in] kv_len Number of elements in the keys and values arrays + * + * \snippet{doc} snippets.dox OrtStatus Return Value + * + * \since Version 1.20. + */ + ORT_API2_STATUS(SetEpDynamicOptions, _Inout_ OrtSession* sess, _In_reads_(kv_len) const char* const* keys, + _In_reads_(kv_len) const char* const* values, _In_ size_t kv_len); + + /** \brief Release an OrtValueInfo instance if it was not added to an OrtGraph. + * \since Version 1.22. + */ + ORT_CLASS_RELEASE(ValueInfo); + + /** \brief Release an OrtNode if it was not added to an OrtGraph. + * \since Version 1.22. + */ + ORT_CLASS_RELEASE(Node); + + /** \brief Release an OrtGraph. + * \snippet{doc} snippets.dox OrtStatus Return Value + * \since Version 1.22. + */ + ORT_CLASS_RELEASE(Graph); + + /** \brief Release an OrtModel. + * \snippet{doc} snippets.dox OrtStatus Return Value + * \since Version 1.22. + */ + ORT_CLASS_RELEASE(Model); + + /** \brief Get the value name from an OrtValueInfo instance. + * \param[in] value_info The OrtValueInfo instance. + * \param[out] name The name of the OrtValueInfo + * \snippet{doc} snippets.dox OrtStatus Return Value + * \since Version 1.22. + */ + ORT_API2_STATUS(GetValueInfoName, _In_ const OrtValueInfo* value_info, _Out_ const char** name); + + /** \brief Get the type information from an OrtValueInfo instance. + * \param[in] value_info The OrtValueInfo instance. + * \param[out] type_info The type info of the OrtValueInfo + * \snippet{doc} snippets.dox OrtStatus Return Value + * \since Version 1.22. + */ + ORT_API2_STATUS(GetValueInfoTypeInfo, _In_ const OrtValueInfo* value_info, _Outptr_ const OrtTypeInfo** type_info); + + /** \brief Get the Model Editor API instance + * + * Get the Model Editor API instance to create a new model or augment an existing model. + * + * \return Model Editor API struct + * + * \since Version 1.22. + */ + const OrtModelEditorApi*(ORT_API_CALL* GetModelEditorApi)(); + + /** \brief Create an OrtValue for a Tensor that uses pre-existing memory. + * + * ORT will take ownership of the memory and free it using the provided deleter when no longer in use. + * + * \param[in] deleter OrtAllocator instance that will be used to free the memory. + * Only the OrtAllocator:Info and OrtAllocator::Release functions are required. + * The OrtMemoryInfo returned by OrtAllocator::Info must match the location of p_data. + * \param[in] p_data Pointer to the memory that will be used by the Tensor. ORT will take ownership of the memory. + * \param[in] p_data_len Length of the memory in bytes. + * \param[in] shape Dimensions of the Tensor. All values should be > 0. + * \param[in] shape_len Number of dimensions in the shape array. + * \param[in] type Data type of the Tensor. + * \param[out] out Newly created ::OrtValue. Must be freed with OrtApi::ReleaseValue + * + * \snippet{doc} snippets.dox OrtStatus Return Value + * + * \since Version 1.22. + */ + ORT_API2_STATUS(CreateTensorWithDataAndDeleterAsOrtValue, _In_ OrtAllocator* deleter, + _In_ void* p_data, size_t p_data_len, + _In_ const int64_t* shape, size_t shape_len, + ONNXTensorElementDataType type, + _Outptr_ OrtValue** out); + + /** \brief sets load cancellation flag to abort session loading process. + * + * \param[in] options instance that was passed to the session at creation time. + * \param[in] cancel setting this to true after model loading process was initiated will + * attempt to cancel the loading process. If cancellation is successful, CreateSession() + * CreateSessionFromArray() or any other session creation API that take session options as an + * argument will return an OrtStatus indicating that session loading was canceled at user request, + * error code ORT_MODEL_LOAD_CANCELED. + * The APIs above would not return any valid Session instance. This is the best case effort and the result + * is not guaranteed. The session may have already been created and initialized + * before the cancellation request was issued. + * + * \snippet{doc} snippets.dox OrtStatus Return Value + * + * \since Version 1.22. + */ + ORT_API2_STATUS(SessionOptionsSetLoadCancellationFlag, _Inout_ OrtSessionOptions* options, + _In_ bool cancel); + + /** \brief Get the Compile API instance. + * + * Get the Compile API instance to compile ONNX models. Execution providers that support compilation fuse a subgraph + * into an EPContext node that wraps a provider-specific binary representation of the subgraph. + * For more details about the EPContext design, refer to: + * \htmlonly + * EPContext design document. + * \endhtmlonly + * + * \return Compile API struct instance. + * + * \since Version 1.22. + */ + const OrtCompileApi*(ORT_API_CALL* GetCompileApi)(); + + // + // OrtKeyValuePairs + // + + /** \brief Create an OrtKeyValuePairs instance. + * + * \param[out] out A pointer to a newly created OrtKeyValuePairs instance. + * + * \note Must be released by calling ReleaseKeyValuePairs. + * + * \since Version 1.22. + */ + void(ORT_API_CALL* CreateKeyValuePairs)(_Outptr_ OrtKeyValuePairs** out); + + /** \brief Add a key-value pair to the OrtKeyValuePairs instance. + * + * \param[in] kvps OrtKeyValuePairs instance. + * \param[in] key Key to be added. + * \param[in] value Value to be added. + * + * \note The `key` and `value` are copied internally. + * + * \since Version 1.22. + */ + + void(ORT_API_CALL* AddKeyValuePair)(_In_ OrtKeyValuePairs* kvps, _In_ const char* key, _In_ const char* value); + + /** \brief Get the value associated with a key in the OrtKeyValuePairs instance. + * + * \param[in] kvps OrtKeyValuePairs instance. + * \param[in] key Key to be searched. + * + * \return The value associated with the key, or nullptr if the key does not exist. + * + * \since Version 1.22. + */ + const char*(ORT_API_CALL* GetKeyValue)(_In_ const OrtKeyValuePairs* kvps, _In_ const char* key); + + /** \brief Get all the key-value pairs from the OrtKeyValuePairs instance. + * + * \param[in] kvps OrtKeyValuePairs instance. + * \param[out] keys Array of keys from `kvps`. + * \param[out] values Array of values from `kvps`. + * \param[out] num_entries Number of entries in `keys` and `values`. + * + * \since Version 1.22. + */ + void(ORT_API_CALL* GetKeyValuePairs)(_In_ const OrtKeyValuePairs* kvps, + _Outptr_ const char* const** keys, _Outptr_ const char* const** values, + _Out_ size_t* num_entries); + + /** \brief Remove a key-value pair from the OrtKeyValuePairs instance. + * + * \param[in] kvps OrtKeyValuePairs instance. + * \param[in] key Key to be removed. No error if not found. + * + * \since Version 1.22. + */ + void(ORT_API_CALL* RemoveKeyValuePair)(_In_ OrtKeyValuePairs* kvps, _In_ const char* key); + + /** \brief Release an OrtKeyValuePairs instance. + * + * \param[in] input OrtKeyValuePairs instance to be released. + * + * \since Version 1.22. + */ + ORT_CLASS_RELEASE(KeyValuePairs); + + /** \brief Register an execution provider library with ORT. + * + * The library must export 'CreateEpFactories' and 'ReleaseEpFactory' functions. + * See OrtEpApi for more details. + * + * \param[in] env The OrtEnv instance to register the library in. + * \param[in] registration_name The name to register the execution provider library under. + * \param[in] path The path to the execution provider library. + * + * \snippet{doc} snippets.dox OrtStatus Return Value + * + * \since Version 1.22. + */ + ORT_API2_STATUS(RegisterExecutionProviderLibrary, _In_ OrtEnv* env, _In_ const char* registration_name, + _In_ const ORTCHAR_T* path); + + /** \brief Unregister an execution provider library with ORT. + * + * ORT will call ReleaseEpFactory for all factories created by the library, and unload the library. + * + * You MUST ensure there are no Session instances using execution providers created by the library + * before calling this function. + * + * \param[in] env The OrtEnv instance to unregister the library from. + * \param[in] registration_name The name the execution provider library was registered under. + * + * \snippet{doc} snippets.dox OrtStatus Return Value + * + * \since Version 1.22. + */ + ORT_API2_STATUS(UnregisterExecutionProviderLibrary, _In_ OrtEnv* env, _In_ const char* registration_name); + + /** \brief Get the list of available OrtEpDevice instances. + * + * Each OrtEpDevice instance contains details of the execution provider and the device it will use. + * + * \param[in] env The OrtEnv instance to query. + * \param[out] ep_devices The OrtEpDevice instances that the execution provider will use. + * \param[out] num_ep_devices The number of OrtEpDevice instances returned. + * + * \snippet{doc} snippets.dox OrtStatus Return Value + * + * \since Version 1.22. + */ + ORT_API2_STATUS(GetEpDevices, _In_ const OrtEnv* env, + _Outptr_ const OrtEpDevice* const** ep_devices, _Out_ size_t* num_ep_devices); + + /** \brief Append the execution provider that is responsible for the selected OrtEpDevice instances + * to the session options. + * + * \param[in] session_options Session options to add execution provider to. + * \param[in] env Environment that execution providers were registered with. + * \param[in] ep_devices One or more OrtEpDevice instances to create an execution provider for. + * Obtain from GetEpDevices. All OrtEpDevice instances must be from the same execution + * provider. It is only necessary to provide multiple OrtEpDevices if you want to use the + * same execution provider for multiple devices. + * e.g. the EP is capable of running on GPU and NPU. + * \param[in] num_ep_devices Number of OrtEpDevice instances. + * \param[in] ep_option_keys Optional keys to configure the execution provider. + * \param[in] ep_option_vals Optional values to configure the execution provider. + * \param[in] num_ep_options Number of execution provide options to add. + * + * \snippet{doc} snippets.dox OrtStatus Return Value + * + * \since Version 1.22. + */ + ORT_API2_STATUS(SessionOptionsAppendExecutionProvider_V2, _In_ OrtSessionOptions* session_options, + _In_ OrtEnv* env, + _In_reads_(num_ep_devices) const OrtEpDevice* const* ep_devices, _In_ size_t num_ep_devices, + _In_reads_(num_op_options) const char* const* ep_option_keys, + _In_reads_(num_op_options) const char* const* ep_option_vals, + size_t num_ep_options); + + /** \brief Set the execution provider selection policy for the session. + * + * Allows users to specify a device selection policy for automatic execution provider (EP) selection. + * If custom selection is required please use SessionOptionsSetEpSelectionPolicyDelegate instead. + * + * \param[in] session_options The OrtSessionOptions instance. + * \param[in] policy The device selection policy to use (see OrtExecutionProviderDevicePolicy). + * + * \since Version 1.22 + */ + ORT_API2_STATUS(SessionOptionsSetEpSelectionPolicy, _In_ OrtSessionOptions* session_options, + _In_ OrtExecutionProviderDevicePolicy policy); + + /** \brief Set the execution provider selection policy delegate for the session. + * + * Allows users to provide a custom device selection policy for automatic execution provider (EP) selection. + * + * \param[in] session_options The OrtSessionOptions instance. + * \param[in] delegate Delegate callback for custom selection. + * \param[in] delegate_state Optional state that will be passed to the delegate callback. nullptr if not required. + * + * \since Version 1.22 + */ + ORT_API2_STATUS(SessionOptionsSetEpSelectionPolicyDelegate, _In_ OrtSessionOptions* session_options, + _In_ EpSelectionDelegate delegate, + _In_opt_ void* delegate_state); + + /** \brief Get the hardware device type. + * + * \param[in] device The OrtHardwareDevice instance to query. + * \return The hardware device type. + * + * \since Version 1.22. + */ + OrtHardwareDeviceType(ORT_API_CALL* HardwareDevice_Type)(_In_ const OrtHardwareDevice* device); + + /** \brief Get the hardware device's vendor identifier. + * + * \param[in] device The OrtHardwareDevice instance to query. + * \return The hardware device vendor identifier. + * + * \since Version 1.22. + */ + uint32_t(ORT_API_CALL* HardwareDevice_VendorId)(_In_ const OrtHardwareDevice* device); + + /** \brief Get the hardware device's vendor name. + * + * \param[in] device The OrtHardwareDevice instance to query. + * \return The hardware device's vendor name. + * + * \since Version 1.22. + */ + const char*(ORT_API_CALL* HardwareDevice_Vendor)(_In_ const OrtHardwareDevice* device); + + /** \brief Get the hardware device's unique identifier. + * + * \param[in] device The OrtHardwareDevice instance to query. + * \return The device id. + * + * \note This is not a unique identifier. It identifies the hardware type when combined with vendor id. + * \since Version 1.22. + */ + uint32_t(ORT_API_CALL* HardwareDevice_DeviceId)(_In_ const OrtHardwareDevice* device); + + /** \brief Get hardware device metadata. + * + * \param[in] device The OrtHardwareDevice instance to query. + * \return An OrtKeyValuePairs instance containing the metadata for the device. + * Note: ORT owns the instance so the user must not call ReleaseKeyValuePairs with it. + * + * \since Version 1.22. + */ + const OrtKeyValuePairs*(ORT_API_CALL* HardwareDevice_Metadata)(_In_ const OrtHardwareDevice* device); + + /** \brief Get the execution provider name. + * + * \param[in] ep_device The OrtEpDevice instance to query. + * \return The execution provider name. + * + * \since Version 1.22. + */ + const char*(ORT_API_CALL* EpDevice_EpName)(_In_ const OrtEpDevice* ep_device); + + /** \brief Get the execution provider's vendor name. + * + * \param[in] ep_device The OrtEpDevice instance to query. + * \return The execution provider's vendor name. + * + * \since Version 1.22. + */ + const char*(ORT_API_CALL* EpDevice_EpVendor)(_In_ const OrtEpDevice* ep_device); + + /** \brief Get the metadata for the OrtEpDevice. + * + * \param[in] ep_device The OrtEpDevice instance to query. + * \return An OrtKeyValuePairs instance containing the metadata for the device. + * + * \since Version 1.22. + */ + const OrtKeyValuePairs*(ORT_API_CALL* EpDevice_EpMetadata)(_In_ const OrtEpDevice* ep_device); + + /** \brief Get the execution provider options for the OrtEpDevice. + * + * \param[in] ep_device The OrtEpDevice instance to query. + * \return An OrtKeyValuePairs instance containing the execution provider options for the device. + * + * \since Version 1.22. + */ + const OrtKeyValuePairs*(ORT_API_CALL* EpDevice_EpOptions)(_In_ const OrtEpDevice* ep_device); + + /** \brief Get the OrtHardwareDevice instance for the OrtEpDevice. + * + * \param[in] ep_device The OrtEpDevice instance to query. + * \return The OrtHardwareDevice instance for the device. + * + * \since Version 1.22. + */ + const OrtHardwareDevice*(ORT_API_CALL* EpDevice_Device)(_In_ const OrtEpDevice* ep_device); + + /** \brief Get the OrtEpApi instance for implementing an execution provider. + * + * \since Version 1.22. + */ + const OrtEpApi*(ORT_API_CALL* GetEpApi)(); +}; + +/* + * Steps to use a custom op: + * 1 Create an OrtCustomOpDomain with the domain name used by the custom ops + * 2 Create an OrtCustomOp structure for each op and add them to the domain + * 3 Call OrtAddCustomOpDomain to add the custom domain of ops to the session options + */ + +// Specifies some characteristics of inputs/outputs of custom ops: +// Specify if the inputs/outputs are one of: +// 1) Non-optional (input/output must be present in the node) +// 2) Optional (input/output may be absent in the node) +// 3) Variadic: A variadic input or output specifies N (i.e., the minimum arity) or more operands. +// Only the last input or output of a custom op may be marked as variadic. +// The homogeneity of the variadic input or output determines whether all operands must be of the same +// tensor element type. +typedef enum OrtCustomOpInputOutputCharacteristic { + INPUT_OUTPUT_REQUIRED = 0, + INPUT_OUTPUT_OPTIONAL, + INPUT_OUTPUT_VARIADIC, +} OrtCustomOpInputOutputCharacteristic; + +/* + * The OrtCustomOp structure defines a custom op's schema and its kernel callbacks. The callbacks are filled in by + * the implementor of the custom op. + */ +struct OrtCustomOp { + uint32_t version; // Must be initialized to ORT_API_VERSION + + // This callback creates the kernel, which is a user defined + // parameter that is passed to the Kernel* callbacks below. It is + // recommended to use CreateKernelV2 which allows for a safe error + // propagation by returning an OrtStatusPtr. + void*(ORT_API_CALL* CreateKernel)(_In_ const struct OrtCustomOp* op, _In_ const OrtApi* api, + _In_ const OrtKernelInfo* info); + + // Returns the name of the op + const char*(ORT_API_CALL* GetName)(_In_ const struct OrtCustomOp* op); + + // Returns the type of the execution provider, return nullptr to use CPU execution provider + const char*(ORT_API_CALL* GetExecutionProviderType)(_In_ const struct OrtCustomOp* op); + + // Returns the count and types of the input & output tensors + ONNXTensorElementDataType(ORT_API_CALL* GetInputType)(_In_ const struct OrtCustomOp* op, _In_ size_t index); + size_t(ORT_API_CALL* GetInputTypeCount)(_In_ const struct OrtCustomOp* op); + ONNXTensorElementDataType(ORT_API_CALL* GetOutputType)(_In_ const struct OrtCustomOp* op, _In_ size_t index); + size_t(ORT_API_CALL* GetOutputTypeCount)(_In_ const struct OrtCustomOp* op); + + // Perform a computation step. It is recommended to use + // KernelComputeV2 which allows for a safe error propagation by + // returning an OrtStatusPtr. + void(ORT_API_CALL* KernelCompute)(_In_ void* op_kernel, _In_ OrtKernelContext* context); + void(ORT_API_CALL* KernelDestroy)(_In_ void* op_kernel); + + // Returns the characteristics of the input & output tensors + OrtCustomOpInputOutputCharacteristic(ORT_API_CALL* GetInputCharacteristic)(_In_ const struct OrtCustomOp* op, _In_ size_t index); + OrtCustomOpInputOutputCharacteristic(ORT_API_CALL* GetOutputCharacteristic)(_In_ const struct OrtCustomOp* op, _In_ size_t index); + + // Returns the memory type of the input tensors. This API allows the custom op + // to place the inputs on specific devices. By default, it returns + // OrtMemTypeDefault, which means the input is placed on the default device for + // the execution provider. If the inputs need to be with different memory tyeps, + // this function can be overridden to return the specific memory types. + OrtMemType(ORT_API_CALL* GetInputMemoryType)(_In_ const struct OrtCustomOp* op, _In_ size_t index); + + // Returns the minimum number of input arguments expected for the variadic input. + // Applicable only for custom ops that have a variadic input. + int(ORT_API_CALL* GetVariadicInputMinArity)(_In_ const struct OrtCustomOp* op); + + // Returns true (non-zero) if all arguments of a variadic input have to be of the same type (homogeneous), + // and false (zero) otherwise. + // Applicable only for custom ops that have a variadic input. + int(ORT_API_CALL* GetVariadicInputHomogeneity)(_In_ const struct OrtCustomOp* op); + + // Returns the minimum number of output values expected for the variadic output. + // Applicable only for custom ops that have a variadic output. + int(ORT_API_CALL* GetVariadicOutputMinArity)(_In_ const struct OrtCustomOp* op); + + // Returns true (non-zero) if all outputs values of a variadic output have to be of the same type (homogeneous), + // and false (zero) otherwise. + // Applicable only for custom ops that have a variadic output. + int(ORT_API_CALL* GetVariadicOutputHomogeneity)(_In_ const struct OrtCustomOp* op); + + // Create the kernel state which is passed to each compute call. + OrtStatusPtr(ORT_API_CALL* CreateKernelV2)(_In_ const struct OrtCustomOp* op, _In_ const OrtApi* api, + _In_ const OrtKernelInfo* info, + _Out_ void** kernel); + + // Perform the computation step. + OrtStatusPtr(ORT_API_CALL* KernelComputeV2)(_In_ void* op_kernel, _In_ OrtKernelContext* context); + + OrtStatusPtr(ORT_API_CALL* InferOutputShapeFn)(_In_ const struct OrtCustomOp* op, _In_ OrtShapeInferContext*); + + // Get start range + int(ORT_API_CALL* GetStartVersion)(_In_ const struct OrtCustomOp* op); + int(ORT_API_CALL* GetEndVersion)(_In_ const struct OrtCustomOp* op); + + // Get the inplace_map that defines which output can reuse which input + // Callers will provide 2 raw int* and pass in their address, this function will fill these 2 arrays + // when return, output (*output_index)[i] may reuse the input (*input_index[i]). + // The return value is the size of these 2 arrays. + // Callers are responsible to delete these 2 arrays after use by calling OrtCustomOp::ReleaseMayInplace(). + size_t(ORT_API_CALL* GetMayInplace)(_Out_ int** input_index, _Out_ int** output_index); + + // Release the pointer input_index and output_index allocated from GetMayInplace() function. + // If GetMayInplace() is defined, this function MUST be defined as well. + void(ORT_API_CALL* ReleaseMayInplace)(_Frees_ptr_opt_ int* input_index, _Frees_ptr_opt_ int* output_index); + + // Same as GetMayInplace() and ReleaseMayInplace() + size_t(ORT_API_CALL* GetAliasMap)(_Out_ int** input_index, _Out_ int** output_index); + void(ORT_API_CALL* ReleaseAliasMap)(_Frees_ptr_opt_ int* input_index, _Frees_ptr_opt_ int* output_index); +}; + +/** + * ORT Model Editor API + */ + +/** + * \brief The OrtModelEditorApi struct provides functions to create or edit an ONNX model. + * + * See onnxruntime/test/shared_lib/test_model_editor_api.cc for example usage. + * + * \since Version 1.22. + */ +struct OrtModelEditorApi { + // Model building/editing requires a full build. We return nullptr from GetModelEditorApi if this is a minimal + // build, so it doesn't matter if there are no function pointers in this struct as a user will never get an + // OrtModelEditorApi instance. We do however need a dummy field to avoid empty struct warning. +#if defined(ORT_MINIMAL_BUILD) + const bool not_defined_in_this_build; +#else + /** \brief Create an OrtTypeInfo instance for a Tensor. + * + * Create an OrtTypeInfo instance for a Tensor to use as graph inputs/outputs with the Model Editor API. + * + * User can release `tensor_info` after creating the OrtTypeInfo. + * + * \param[in] tensor_info Tensor type and shape information. + * \param[out] type_info TypeInfo instance for the tensor. + * + * \snippet{doc} snippets.dox OrtStatus Return Value + * + * \since Version 1.22. + */ + ORT_API2_STATUS(CreateTensorTypeInfo, _In_ const OrtTensorTypeAndShapeInfo* tensor_info, + _Outptr_ OrtTypeInfo** type_info); + + /** \brief Create an OrtTypeInfo instance for a SparseTensor. + * + * Create an OrtTypeInfo instance for a SparseTensor to use as graph inputs/outputs with the Model Editor API. + * + * User can release `tensor_info` after creating the OrtTypeInfo. + * + * \param[in] tensor_info SparseTensor type and shape information. + * \param[out] type_info TypeInfo instance for the tensor. + * + * \snippet{doc} snippets.dox OrtStatus Return Value + * + * \since Version 1.22. + */ + ORT_API2_STATUS(CreateSparseTensorTypeInfo, _In_ const OrtTensorTypeAndShapeInfo* tensor_info, + _Outptr_ OrtTypeInfo** type_info); + + /** \brief Create an OrtTypeInfo instance for a Map. + * + * Create an OrtTypeInfo instance for a Map to use as graph inputs/outputs with the Model Editor API. + * + * User can release `map_value_type` after creating the OrtTypeInfo. + * + * \param[in] map_key_type Key type for the map. + * \param[in] map_value_type Value type for the map. + * \param[out] type_info TypeInfo instance for the map. + * + * \snippet{doc} snippets.dox OrtStatus Return Value + * + * \since Version 1.22. + */ + ORT_API2_STATUS(CreateMapTypeInfo, ONNXTensorElementDataType map_key_type, _In_ const OrtTypeInfo* map_value_type, + _Outptr_ OrtTypeInfo** type_info); + + /** \brief Create an OrtTypeInfo instance for a Sequence. + * + * Create an OrtTypeInfo instance for a Sequence to use as graph inputs/outputs with the Model Editor API. + * + * User can release `sequence_type` after creating the OrtTypeInfo. + * + * \param[in] sequence_type Sequence type and shape information. + * \param[out] type_info TypeInfo instance for the sequence. + * + * \snippet{doc} snippets.dox OrtStatus Return Value + * + * \since Version 1.22. + */ + ORT_API2_STATUS(CreateSequenceTypeInfo, _In_ const OrtTypeInfo* sequence_type, _Outptr_ OrtTypeInfo** type_info); + + /** \brief Create an OrtTypeInfo instance for an Optional. + * + * Create an OrtTypeInfo instance for an Optional to use as graph inputs/outputs with the Model Editor API. + * + * User can release `contained_type` after creating the OrtTypeInfo. + * + * \param[in] contained_type Tensor type and shape information. + * \param[out] type_info TypeInfo instance for the tensor. + * + * \snippet{doc} snippets.dox OrtStatus Return Value + * + * \since Version 1.22. + */ + ORT_API2_STATUS(CreateOptionalTypeInfo, _In_ const OrtTypeInfo* contained_type, _Outptr_ OrtTypeInfo** type_info); + + /** \brief Create an OrtValueInfo for use as an OrtGraph input or output. + * + * \param[in] name The name of the input or output. + * \param[in] type_info The type information for the input or output. The provided value is copied. + * \param[out] value_info The OrtValueInfo instance. + * + * \snippet{doc} snippets.dox OrtStatus Return Value + * + * \since Version 1.22. + */ + ORT_API2_STATUS(CreateValueInfo, _In_ const char* name, _In_ const OrtTypeInfo* type_info, + _Outptr_ OrtValueInfo** value_info); + + /** \brief Create an OrtNode to add to an OrtGraph. + * + * Create an OrtNode. + * + * Create attributes with CreateOpAttr. OrtOpAttr instances are copied. + * + * \param[in] operator_name The name of the operator. + * \param[in] domain_name The domain of the operator. Use an empty string for ONNX operators. + * \param[in] node_name The name of the node. + * \param[in] input_names The names of the inputs. + * \param[in] input_names_len The number of input names. + * \param[in] output_names The names of the outputs. + * \param[in] output_names_len The number of output names. + * \param[in] attributes The optional attributes of the node. + * \param[in] attribs_len The number of attributes. May be zero. + * \param[out] node The OrtNode instance. + * + * \snippet{doc} snippets.dox OrtStatus Return Value + * + * \since Version 1.22. + */ + ORT_API2_STATUS(CreateNode, _In_ const char* operator_name, _In_ const char* domain_name, _In_ const char* node_name, + _In_reads_(input_names_len) const char* const* input_names, size_t input_names_len, + _In_reads_(output_names_len) const char* const* output_names, size_t output_names_len, + _In_reads_(attribs_len) _In_opt_ OrtOpAttr** attributes, _In_ size_t attribs_len, + _Outptr_ OrtNode** node); + + /** \brief Create an OrtGraph + * \snippet{doc} snippets.dox OrtStatus Return Value + * \since Version 1.22. + */ + ORT_API2_STATUS(CreateGraph, _Outptr_ OrtGraph** graph); + + /** \brief Set the inputs for the OrtGraph. + * + * Set the graph inputs. This will replace any existing inputs with the new values. + * The OrtGraph takes ownership of the OrtValueInfo instances and you should NOT call ReleaseOrtValueInfo. + * + * \param[in] graph The OrtGraph instance to update. + * \param[in] inputs The input OrtValueInfo instances. + * \param[in] inputs_len The number of input OrtValueInfo instances. + * + * \snippet{doc} snippets.dox OrtStatus Return Value + * + * \since Version 1.22. + */ + ORT_API2_STATUS(SetGraphInputs, _Inout_ OrtGraph* graph, + _In_reads_(inputs_len) _In_ OrtValueInfo** inputs, _In_ size_t inputs_len); + + /** \brief Set the outputs for the OrtGraph. + * + * Set the graph outputs. This will replace any existing outputs with the new values. + * The OrtGraph takes ownership of the OrtValueInfo instances provided and you should NOT call ReleaseOrtValueInfo. + * + * \param[in] graph The OrtGraph instance to update. + * \param[in] outputs The output OrtValueInfo instances. + * \param[in] outputs_len The number of output OrtValueInfo instances. + * + * \snippet{doc} snippets.dox OrtStatus Return Value + * + * \since Version 1.22. + */ + ORT_API2_STATUS(SetGraphOutputs, _Inout_ OrtGraph* graph, + _In_reads_(outputs_len) _In_ OrtValueInfo** outputs, _In_ size_t outputs_len); + + /** \brief Add an initializer to the OrtGraph + * + * ORT will take ownership of the OrtValue and you should NOT call ReleaseOrtValue. + * + * Two options: + * + * Allocated memory: + * Use CreateTensorAsOrtValue (allocates memory) and populate the tensor with the data. + * Set `data_is_external` to false. + * + * Pre-existing memory: + * Use CreateTensorWithDataAsOrtValue or CreateTensorWithDataAndDeleterAsOrtValue to create an OrtValue + * with a tensor that contains a pointer to the existing data. + * Set `data_is_external` to true. + * + * The pointer must remain valid for the duration of the inference session. + * If using CreateTensorWithDataAsOrtValue you are responsible for freeing the memory after the inference session + * is released. + * If using CreateTensorWithDataAndDeleterAsOrtValue, ORT will free the memory using the provided deleter as + * soon as the OrtValue is no longer in use. + * + * NOTE: A tensor containing pre-existing memory MUST have 128 bytes of data or more. + * For smaller tensors use CreateTensorAsOrtValue. + * + * ONNX shape inferencing does not support external data. An initializer involved in shape inferencing is + * typically small (a single value or limited by the rank of a tensor) and uses less than 128 bytes of + * memory, so this limit acts as a simple catch-all rule to avoid issues. + * e.g. Reshape's `shape`, Clip's `min` and `max`, various ops `axes`. + * + * \param[in] graph The OrtGraph instance to update. + * \param[in] name The value name for the initializer. + * \param[in] tensor The OrtValue instance containing the tensor data. + * \param[in] data_is_external Set to true if the data is external and should not be copied. + * + * \snippet{doc} snippets.dox OrtStatus Return Value + * + * \since Version 1.22. + */ + ORT_API2_STATUS(AddInitializerToGraph, _Inout_ OrtGraph* graph, _In_ const char* name, _In_ OrtValue* tensor, + bool data_is_external); + + /** \brief Add an OrtNode to an OrtGraph + * + * Add the node to the graph. The OrtGraph will take ownership of OrtNode and you should NOT call ReleaseOrtNode. + * + * \param[in] graph The OrtGraph instance to update. + * \param[in] node The OrtNode instance to add to the graph. + * + * \snippet{doc} snippets.dox OrtStatus Return Value + * + * \since Version 1.22. + */ + ORT_API2_STATUS(AddNodeToGraph, _Inout_ OrtGraph* graph, _In_ OrtNode* node); + + /** \brief Create an OrtModel. + * + * Create an OrtModel. + * + * This can be used to build a new model, or to augment an existing model. + * + * \param[in] domain_names The domain names for the model. + * If augmenting an existing model add additional domains if needed. + * \param[in] opset_versions The opset versions for the model. + * If augmenting an existing model add additional opset versions if needed. + * \param[in] opset_entries_len The number of domain_names and opset_versions entries. + * Domain and opset entries should be 1:1 + * \param[out] model The OrtModel instance. + * + * \snippet{doc} snippets.dox OrtStatus Return Value + * + * \since Version 1.22. + */ + ORT_API2_STATUS(CreateModel, + _In_reads_(opset_entries_len) const char* const* domain_names, + _In_reads_(opset_entries_len) const int* opset_versions, + size_t opset_entries_len, + _Outptr_ OrtModel** model); + + /** \brief Add an OrtGraph to an OrtModel. + * + * Add the graph to a model. This should be called once when creating a new model. + * + * The OrtModel takes ownership of the OrtGraph and you should NOT call ReleaseOrtGraph. + * + * \param[in] model The OrtModel instance to update. + * \param[in] graph The OrtGraph instance to add to the model. + * + * \snippet{doc} snippets.dox OrtStatus Return Value + * + * \since Version 1.22. + */ + ORT_API2_STATUS(AddGraphToModel, _Inout_ OrtModel* model, _In_ OrtGraph* graph); + + /** \brief Create an OrtSession using the OrtModel. + * + * Create an inference session using the OrtModel instance. + * The OrtModel should have been populated with an OrtGraph containing nodes and initializers, and SetGraphInputs + * and SetGraphOutputs must have been called. + * This will validate the model, run optimizers, and prepare the session for inferencing. + * + * ReleaseOrtModel must be called to free the OrtModel after session creation. + * + * \param[in] env The OrtEnv instance. + * \param[in] model The OrtModel instance. + * \param[in] options The OrtSessionOptions instance. + * \param[out] out The OrtSession instance. + * + * \snippet{doc} snippets.dox OrtStatus Return Value + * + * \since Version 1.22. + */ + ORT_API2_STATUS(CreateSessionFromModel, _In_ const OrtEnv* env, _In_ const OrtModel* model, + _In_ const OrtSessionOptions* options, _Outptr_ OrtSession** out); + + /** \brief Create an OrtSession to augment an existing model. + * + * Create an OrtSession with an existing model that will be augmented with additional nodes and initializers. + * Nodes can be added before or after the existing nodes in the model. ONNX Runtime will connect the nodes when the + * model is finalized. + * + * To add nodes and initializers to the existing model, first create an OrtModel using CreateModel. + * Add nodes and initializers to the OrtModel using AddNodeToGraph and AddInitializerToGraph. + * Graph inputs/outputs should be updated with SetGraphInputs and SetGraphOutputs as needed to reflect changes made + * by the new nodes. The list of graph inputs/outputs should be for the overall model and not just the new nodes. + * + * Add the new information from the OrtModel to the original model using ApplyModelToSession, and prepare the + * session for inferencing by calling FinalizeModelEditorSession. + * + * \param{in} env The OrtEnv instance. + * \param{in} model_path The path to the existing ONNX model to augment. + * \param{in} options The OrtSessionOptions instance. + * \param{out} out The created OrtSession instance. + * \snippet{doc} snippets.dox OrtStatus Return Value + * + * \since Version 1.22. + */ + ORT_API2_STATUS(CreateModelEditorSession, _In_ const OrtEnv* env, _In_ const ORTCHAR_T* model_path, + _In_ const OrtSessionOptions* options, + _Outptr_ OrtSession** out); + + /** \brief Create an OrtSession to augment an existing model. + * + * Create an OrtSession with an existing model that will be augmented with additional nodes and initializers. + * Nodes can be added before or after the existing nodes in the model. ONNX Runtime will connect the nodes when the + * model is finalized. + * + * To add nodes and initializers to the existing model, first create an OrtModel using CreateModel. + * Add nodes and initializers to the OrtModel using AddNodeToGraph and AddInitializerToGraph. + * Graph inputs/outputs should be updated with SetGraphInputs and SetGraphOutputs as needed to reflect changes made + * by the new nodes. The list of graph inputs/outputs should be for the overall model and not just the new nodes. + * + * Add the new information from the OrtModel to the original model using ApplyModelToSession, and prepare the + * session for inferencing by calling FinalizeModelEditorSession. + * + * \param{in} env The OrtEnv instance. + * \param{in} model_data The model data for the existing model to augment. + * \param{in} model_data_length The length of the model data. + * \param{in} options The OrtSessionOptions instance. + * \param{out} out The created OrtSession instance. + * + * \snippet{doc} snippets.dox OrtStatus Return Value + * + * \since Version 1.22. + */ + ORT_API2_STATUS(CreateModelEditorSessionFromArray, _In_ const OrtEnv* env, + _In_ const void* model_data, size_t model_data_length, + _In_ const OrtSessionOptions* options, + _Outptr_ OrtSession** out); + + /** \brief Query the session for the opset version of a domain. + * + * When using the Model Editor API to augment a model, any new nodes must conform to the opset version of the + * original model. To do that the user must be able to discover that opset version. + * Returns an error if the domain is not used in the model. + * + * \param[in] session OrtSession to query + * \param[in] domain Domain to query. The ONNX domain is an empty string. + * \param[out] opset The opset version of the domain. + * + * \snippet{doc} snippets.dox OrtStatus Return Value + * + * \since Version 1.22. + */ + ORT_API2_STATUS(SessionGetOpsetForDomain, _In_ const OrtSession* session, _In_ const char* domain, _Out_ int* opset); + + /** \brief Apply changes to augment the ONNX model in a session created using CreateModelEditorSession[FromArray] + * + * Adds new nodes and updates graph inputs/outputs using `model` to augment the original ONNX model in the session. + * All changes will be validated. + * Call FinalizeModelEditorSession to prepare the session for inferencing. + * + * Existing input/outputs will only be updated if the OrtGraph inputs/outputs are set in the OrtModel. + * i.e. you don't need to call SetGraphInputs/SetGraphOutputs if they are unchanged. + * + * ReleaseOrtModel must be called to free the OrtModel after it is applied to the session. + * + * \param[in] session OrtSession to update. Session must have been created using CreateModelEditorSession[FromArray]. + * \param[in] model OrtModel containing new nodes, new initializers, and updated graph input and/or output info. + * + * \snippet{doc} snippets.dox OrtStatus Return Value + * + * \since Version 1.22. + */ + ORT_API2_STATUS(ApplyModelToModelEditorSession, _Inout_ OrtSession* session, _In_ OrtModel* model); + + /** \brief Finalize the Model Editor session that was created using CreateModelEditorSession[FromArray]. + * + * Finalize the Model Editor session that augmented an ONNX model by adding new nodes. + * This will run optimizers and prepare the session for inferencing. + * + * \param[in] session OrtSession to finalize. Session must have been created using CreateModelEditorSession[FromArray]. + * \param[in] options OrtSessionOptions to use for the session. + * \param[in] prepacked_weights_container Optional OrtPrepackedWeightsContainer to use for the session. + Set to nullptr if not used. + * \snippet{doc} snippets.dox OrtStatus Return Value + * + * \since Version 1.22. + */ + ORT_API2_STATUS(FinalizeModelEditorSession, _Inout_ OrtSession* session, _In_ const OrtSessionOptions* options, + _In_opt_ OrtPrepackedWeightsContainer* prepacked_weights_container); +#endif // !defined(ORT_MINIMAL_BUILD) +}; + +/** + * ORT Compile API + */ + +/** + * \brief The OrtCompileApi struct provides functions to compile ONNX models. + * + * Execution providers that support compilation fuse a subgraph into an EPContext node that wraps a provider-specific + * binary representation of the subgraph. + * For more details about the EPContext design, refer to: + * \htmlonly + * EPContext design document. + * \endhtmlonly + * + * Example (error handling not shown): + * OrtStatus* status = NULL; + * OrtCompileApi* compile_api = ort_api->GetCompileApi(); + * OrtModelCompilationOptions* compile_options = NULL; + * + * status = compile_api->CreateModelCompilationOptionsFromSessionOptions(env, session_options, &compile_options); + * status = compile_api->ModelCompilationOptions_SetInputModelPath(compile_options, ORT_TSTR("model.onnx")); + * status = compile_api->ModelCompilationOptions_SetOutputModelPath(compile_options, ORT_TSTR("model.compiled.onnx")); + * status = compile_api->CompileModel(env, compile_options); + * compile_api->ReleaseModelCompilationOptions(compile_options); + * + * \since Version 1.22. + */ +struct OrtCompileApi { + /// @} + /// \name OrtModelCompilationOptions + /// @{ + ORT_CLASS_RELEASE(ModelCompilationOptions); + + /** \brief Creates an OrtModelCompilationOptions object from an existing OrtSessionOptions object. + * + * An OrtModelCompilationOptions object contains the settings used to generate a compiled ONNX model. + * The OrtSessionOptions object has the execution providers with which the model will be compiled. + * + * ReleaseOrtModelCompilationsOptions must be called to free the OrtModelCompilationOptions after calling + * CompileModel. + * + * \param[in] env OrtEnv object. + * \param[in] session_options The OrtSessionOptions instance from which to create the OrtModelCompilationOptions. + * \param[out] out The created OrtModelCompilationOptions instance. + * \snippet{doc} snippets.dox OrtStatus Return Value + * + * \since Version 1.22. + */ + ORT_API2_STATUS(CreateModelCompilationOptionsFromSessionOptions, _In_ const OrtEnv* env, + _In_ const OrtSessionOptions* session_options, _Outptr_ OrtModelCompilationOptions** out); + + /** \brief Sets the file path to the input ONNX model to compile. + * + * The input model's location (e.g., file path or memory buffer) must be set with either + * ModelCompilationOptions_SetInputModelPath or ModelCompilationOptions_SetInputModelFromBuffer. + * + * \param[in] model_compile_options The OrtModelCompilationOptions instance. + * \param[in] input_model_path Null terminated string of the path (wchar on Windows, char otherwise). + * + * \snippet{doc} snippets.dox OrtStatus Return Value + * + * \since Version 1.22. + */ + ORT_API2_STATUS(ModelCompilationOptions_SetInputModelPath, _In_ OrtModelCompilationOptions* model_compile_options, + _In_ const ORTCHAR_T* input_model_path); + + /** \brief Sets the buffer that stores the bytes of the loaded ONNX model to compile. + * + * The input model's location (e.g., file path or memory buffer) must be set with either + * ModelCompilationOptions_SetInputModelPath or ModelCompilationOptions_SetInputModelFromBuffer. + * + * \param[in] model_compile_options The OrtModelCompilationOptions instance. + * \param[in] input_model_data Buffer containing the loaded ONNX model bytes. + * \param[in] input_model_data_size The number of bytes in the `input_model_data` buffer. + * + * \snippet{doc} snippets.dox OrtStatus Return Value + * + * \since Version 1.22. + */ + ORT_API2_STATUS(ModelCompilationOptions_SetInputModelFromBuffer, + _In_ OrtModelCompilationOptions* model_compile_options, + _In_ const void* input_model_data, + size_t input_model_data_size); + + /** \brief Sets the file path for the output ONNX model generated by CompileModel. + * + * The output model's location (e.g., file path or memory buffer) can be set with either + * ModelCompilationOptions_SetOutputModelPath or ModelCompilationOptions_SetOutputModelBuffer. + * + * If the output model's location is not set, ONNX Runtime will generate an output file with a path based on + * the input model's file path. Examples: + * /Path/my_model.onnx -> /Path/my_model_ctx.onnx + * /Path/my_model -> /Path/my_model_ctx.onnx + * + * \param[in] model_compile_options The OrtModelCompilationOptions instance. + * \param[in] output_model_path Null terminated string of the path (wchar on Windows, char otherwise). + * + * \snippet{doc} snippets.dox OrtStatus Return Value + * + * \since Version 1.22. + */ + ORT_API2_STATUS(ModelCompilationOptions_SetOutputModelPath, _In_ OrtModelCompilationOptions* model_compile_options, + _In_ const ORTCHAR_T* output_model_path); + + /** \brief Optionally sets the file that should store external initializers for the compiled ONNX model. + * If not set, initializers are stored within the model. + * + * Only initializers for nodes that were not compiled are stored in the external initializers file. + * Compiled nodes contain their initializer data within the `ep_cache_context` attribute of EPContext nodes. + * Refer to ModelCompilationOptions_SetEpContextEmbedMode. + * + * \param[in] model_compile_options The OrtModelCompilationOptions instance. + * \param[in] external_initializers_file_path Null terminated string of the path to the file. + * \param[in] external_initializers_size_threshold Initializers larger than this threshold are stored in the file. + * + * \snippet{doc} snippets.dox OrtStatus Return Value + * + * \since Version 1.22. + */ + ORT_API2_STATUS(ModelCompilationOptions_SetOutputModelExternalInitializersFile, + _In_ OrtModelCompilationOptions* model_compile_options, + _In_ const ORTCHAR_T* external_initializers_file_path, + size_t external_initializers_size_threshold); + + /** \brief Configures model compilation to store the output compiled ONNX model in a buffer. + * + * The caller passes an OrtAllocator that ONNX Runtime uses to allocate memory for the buffer. + * + * The output model's location (e.g., file path or memory buffer) can be set with either + * ModelCompilationOptions_SetOutputModelPath or ModelCompilationOptions_SetOutputModelBuffer. + * + * If the output model's location is not set, ONNX Runtime will generate an output file with a path based on + * the input model's file path. Examples: + * /Path/my_model.onnx -> /Path/my_model_ctx.onnx + * /Path/my_model -> /Path/my_model_ctx.onnx + * + * \param[in] model_compile_options The OrtModelCompilationOptions instance. + * \param[in] allocator The allocator used to allocate the buffer for the compiled model. + * \param[out] output_model_buffer_ptr Pointer to the buffer that stores the compiled model. + * \param[out] output_model_buffer_size_ptr Pointer set to the size of output model in bytes. + * + * \snippet{doc} snippets.dox OrtStatus Return Value + * + * \since Version 1.22. + */ + ORT_API2_STATUS(ModelCompilationOptions_SetOutputModelBuffer, + _In_ OrtModelCompilationOptions* model_compile_options, + _Inout_ OrtAllocator* allocator, + _Outptr_ void** output_model_buffer_ptr, + _Out_ size_t* output_model_buffer_size_ptr); + + /** \brief Enables or disables the embedding of EPContext binary data into the `ep_cache_context` attribute + * of EPContext nodes. Defaults to false. + * + * If enabled, the `ep_cache_context` attribute of EPContext nodes will store the context binary data, which may + * include weights for compiled subgraphs. + * + * If disabled, the `ep_cache_context` attribute of EPContext nodes will contain the path to the file containing the + * context binary data. The path is set by the execution provider creating the EPContext node. + * + * More details relate to EPContext design refers to: + * \htmlonly + * EPContext design document. + * \endhtmlonly + * + * \param[in] model_compile_options The OrtModelCompilationOptions instance. + * \param[in] embed_ep_context_in_model True to embed EPContext binary data into the EPContext node + * `ep_cache_context` attributes. + * + * \snippet{doc} snippets.dox OrtStatus Return Value + * + * \since Version 1.22. + */ + ORT_API2_STATUS(ModelCompilationOptions_SetEpContextEmbedMode, _In_ OrtModelCompilationOptions* model_compile_options, + bool embed_ep_context_in_model); + + /** \brief Compiles an input ONNX model with the given compilation options. + * + * \param[in] env OrtEnv object. + * \param[in] model_options The compilation options that defines compilation options for a model. + * + * \snippet{doc} snippets.dox OrtStatus Return Value + * + * \since Version 1.22. + */ + ORT_API2_STATUS(CompileModel, _In_ const OrtEnv* env, _In_ const OrtModelCompilationOptions* model_options); +}; + +ORT_RUNTIME_CLASS(Ep); +ORT_RUNTIME_CLASS(EpFactory); + +struct OrtEpApi { + /** \brief Create an OrtEpDevice for the EP and an OrtHardwareDevice. + * \param[in] ep_factory Execution provider factory that is creating the instance. + * \param[in] hardware_device Hardware device that the EP can utilize. + * \param[in] ep_metadata Optional OrtKeyValuePairs instance for execution provider metadata that may be used + * during execution provider selection and passed to CreateEp. + * ep_device will copy this instance and the user should call ReleaseKeyValuePairs. + * \param[in] ep_options Optional OrtKeyValuePairs instance for execution provider options that will be added + * to the Session configuration options if the execution provider is selected. + * ep_device will copy this instance and the user should call ReleaseKeyValuePairs. + * \param ep_device OrtExecutionDevice that is created. + * + * \since Version 1.22. + */ + ORT_API2_STATUS(CreateEpDevice, _In_ OrtEpFactory* ep_factory, + _In_ const OrtHardwareDevice* hardware_device, + _In_opt_ const OrtKeyValuePairs* ep_metadata, + _In_opt_ const OrtKeyValuePairs* ep_options, + _Out_ OrtEpDevice** ep_device); + + ORT_CLASS_RELEASE(EpDevice); +}; + +/** + * \brief The OrtEp struct provides functions to implement for an execution provider. + * \since Version 1.22. + */ +struct OrtEp { + /** \brief The ONNX Runtime version the execution provider was compiled with. + * + * Implementation should set to ORT_API_VERSION. + * ORT will use this to ensure it does not call functions that were not available when the library was compiled. + * + * \since Version 1.22. + */ + uint32_t ort_version_supported; + + /** \brief Get the execution provider name. + * + * \param[in] this_ptr The OrtEp instance. + * \return The execution provider name. + * + * \note Returned string is owned by ORT and valid until UnregisterExecutionProviderLibrary is called. + * + * \since Version 1.22. + */ + const char*(ORT_API_CALL* GetName)(const OrtEp* this_ptr); + + // OrtStatus* GetCapability(OrtEp* ep, const OrtGraph* graph, + // size_t* num_supported_subgraphs, + // OrtIndexedSubgraph** supported_subgraphs, OrtAllocator* allocator); + + // OrtStatus* Compile(OrtEp* ep, const OrtGraph** graphs, OrtNode** fused_graph_nodes, + // size_t count, OrtNodeComputeInfo* node_compute_infos); + + // TODO: Implement OrtEpApi and the complete OrtEp interface as the next step. +}; + +/** \brief The function signature that ORT will call to create OrtEpFactory instances. + * + * This must be available in a function called 'CreateEpFactories' in the execution provider library. + * + * \param[in] registered_name The name the execution library is registered with by RegisterExecutionProviderLibrary + * \param[in] ort_api_base The OrtApiBase instance that is used by the factory to get the OrtApi instance for the + * version of ORT that the library was compiled against. + * \param[in,out] factories The implementation should create and add OrtEpFactory instances to this + * pre-allocated array. + * i.e. usage is `factories[0] = new MyEpFactory();` + * \param[in] max_factories The maximum number of OrtEpFactory instances that can be added to `factories`. + * Current default is to allow 4 factories. This can be increased in the future if needed. + * \param[out] num_factories The number of OrtEpFactory instances created by the factory and added to `factories`. + * + * \snippet{doc} snippets.dox OrtStatus Return Value + * + * \since Version 1.22. + */ +typedef OrtStatus* (*CreateEpApiFactoriesFn)(_In_ const char* registered_name, _In_ const OrtApiBase* ort_api_base, + _Inout_ OrtEpFactory** factories, _In_ size_t max_factories, + _Out_ size_t* num_factories); + +/** \brief The function signature that ORT will call to release an OrtEpFactory instance. + * + * This must be available in a function called 'ReleaseEpFactory' in the execution provider library. + * + * \param[in] factory The OrtEpFactory instance to release. + * + * \snippet{doc} snippets.dox OrtStatus Return Value + * + * \since Version 1.22. + */ +typedef OrtStatus* (*ReleaseEpApiFactoryFn)(_In_ OrtEpFactory* factory); + +/** + * \brief The OrtEpFactory provides functions to create and manage execution providers. + * \since Version 1.22. + */ +struct OrtEpFactory { + /** \brief The ONNX Runtime version the execution provider was compiled with. + * + * Implementation should set to ORT_API_VERSION. + * ORT will use this to ensure it does not call functions that were not available when the library was compiled. + * + * \since Version 1.22. + */ + uint32_t ort_version_supported; + + /** \brief Get the name the of the execution provider that the factory creates. + * + * \param[in] this_ptr The OrtEpFactory instance. + * \return The name of the execution provider the factory creates. + * + * \since Version 1.22. + */ + const char*(ORT_API_CALL* GetName)(const OrtEpFactory* this_ptr); + + /** \brief Get the name of vendor who owns the execution provider that the factory creates. + * + * \param[in] this_ptr The OrtEpFactory instance. + * \return vendor The vendor name of the execution provider the factory creates. + * + * \since Version 1.22. + */ + const char*(ORT_API_CALL* GetVendor)(const OrtEpFactory* this_ptr); // return EP vendor + + /** \brief Get information from the execution provider if it supports the OrtHardwareDevice. + * + * \param[in] this_ptr The OrtEpFactory instance. + * Non-const as the factory is passed through to the CreateEp call via the OrtEpDevice. + * \param[in] devices The OrtHardwareDevice instances that are available. + * \param[in] num_devices The number of OrtHardwareDevice instances. + * \param[out] ep_devices OrtEpDevice instances for each OrtHardwareDevice that the EP can use. + * The implementation should call OrtEpApi::CreateEpDevice to create, and add the OrtEpDevice + * instances to this pre-allocated array. ORT will take ownership of the values returned. + * i.e. usage is `ep_devices[0] = ;` + * \param[in] max_ep_devices The maximum number of OrtEpDevices that can be added to ep_devices. + * Current default is 8. This can be increased if needed. + * \param[out] num_ep_devices The number of EP devices added to ep_devices. + * \return true if the factory can create an execution provider that uses `device`. + * + * \note ORT will take ownership or ep_metadata and/or ep_options if they are not null. + * + * \since Version 1.22. + */ + OrtStatus*(ORT_API_CALL* GetSupportedDevices)(_In_ OrtEpFactory* this_ptr, + _In_reads_(num_devices) const OrtHardwareDevice* const* devices, + _In_ size_t num_devices, + _Inout_ OrtEpDevice** ep_devices, + _In_ size_t max_ep_devices, + _Out_ size_t* num_ep_devices); + + /** \brief Function to create an OrtEp instance for use in a Session. + * + * ORT will call ReleaseEp to release the instance when it is no longer needed. + * + * \param[in] this_ptr The OrtEpFactory instance. + * \param[in] devices The OrtHardwareDevice instances that the execution provider was selected to use. + * \param[in] ep_metadata_pairs Execution provider metadata that was provided to OrtEpApi::CreateEpDevice, for each + * device. + * \param[in] num_devices The number of devices the execution provider was selected for. + * \param[in] session_options The OrtSessionOptions instance that contains the configuration options for the + * session. This will include ep_options from GetSupportedDevices as well as any + * user provided overrides. + * Execution provider options will have been added with a prefix of 'ep..'. + * The OrtSessionOptions instance will NOT be valid after this call and should not be + * stored for later use. + * \param[in] logger The OrtLogger instance for the session that the execution provider should use for logging. + * \param[out] ep The OrtEp instance created by the factory. + * + * \snippet{doc} snippets.dox OrtStatus Return Value + * + * \since Version . This is a placeholder. + */ + OrtStatus*(ORT_API_CALL* CreateEp)(_In_ OrtEpFactory* this_ptr, + _In_reads_(num_devices) const OrtHardwareDevice* const* devices, + _In_reads_(num_devices) const OrtKeyValuePairs* const* ep_metadata_pairs, + _In_ size_t num_devices, + _In_ const OrtSessionOptions* session_options, + _In_ const OrtLogger* logger, _Outptr_ OrtEp** ep); + + /** \brief Release the OrtEp instance. + * + * \param[in] this_ptr The OrtEpFactory instance. + * \param[in] ep The OrtEp instance to release. + * + * \since Version . This is a placeholder. + */ + void(ORT_API_CALL* ReleaseEp)(OrtEpFactory* this_ptr, struct OrtEp* ep); +}; + +/* + * This is the old way to add the CUDA provider to the session, please use SessionOptionsAppendExecutionProvider_CUDA above to access the latest functionality + * This function always exists, but will only succeed if Onnxruntime was built with CUDA support and the CUDA provider shared library exists + * + * \param device_id CUDA device id, starts from zero. + */ +ORT_API_STATUS(OrtSessionOptionsAppendExecutionProvider_CUDA, _In_ OrtSessionOptions* options, int device_id); + +/* + * This is the old way to add the ROCm provider to the session, please use + * SessionOptionsAppendExecutionProvider_ROCM above to access the latest functionality + * This function always exists, but will only succeed if Onnxruntime was built with + * HIP support and the ROCm provider shared library exists + * + * \param device_id HIP device id, starts from zero. + */ +ORT_API_STATUS(OrtSessionOptionsAppendExecutionProvider_ROCM, _In_ OrtSessionOptions* options, int device_id); + +/* + * This is the old way to add the MIGraphX provider to the session, please use + * SessionOptionsAppendExecutionProvider_MIGraphX above to access the latest functionality + * This function always exists, but will only succeed if Onnxruntime was built with + * HIP support and the MIGraphX provider shared library exists + * + * \param device_id HIP device id, starts from zero. + */ +ORT_API_STATUS(OrtSessionOptionsAppendExecutionProvider_MIGraphX, _In_ OrtSessionOptions* options, int device_id); + +/* + * This is the old way to add the oneDNN provider to the session, please use + * SessionOptionsAppendExecutionProvider_oneDNN above to access the latest functionality + * This function always exists, but will only succeed if Onnxruntime was built with + * oneDNN support and the oneDNN provider shared library exists + * + * \param use_arena zero: false. non-zero: true. + */ +ORT_API_STATUS(OrtSessionOptionsAppendExecutionProvider_Dnnl, _In_ OrtSessionOptions* options, int use_arena); + +/* + * This is the old way to add the TensorRT provider to the session, please use SessionOptionsAppendExecutionProvider_TensorRT_V2 above to access the latest functionality + * This function always exists, but will only succeed if Onnxruntime was built with TensorRT support and the TensorRT provider shared library exists + * + * \param device_id CUDA device id, starts from zero. + */ +ORT_API_STATUS(OrtSessionOptionsAppendExecutionProvider_Tensorrt, _In_ OrtSessionOptions* options, int device_id); + +#ifdef __cplusplus +} +#endif +/// @} diff --git a/src/Onnxruntime/CApi.hsc b/src/Onnxruntime/CApi.hsc new file mode 100644 --- /dev/null +++ b/src/Onnxruntime/CApi.hsc @@ -0,0 +1,5389 @@ +{-# LANGUAGE CApiFFI #-} +{-# LANGUAGE DataKinds #-} +{-# LANGUAGE DerivingStrategies #-} +{-# LANGUAGE ImportQualifiedPost #-} +{-# LANGUAGE LambdaCase #-} +{-# LANGUAGE OverloadedRecordDot #-} +{-# LANGUAGE PatternSynonyms #-} +{-# LANGUAGE RecordWildCards #-} +{-# LANGUAGE TypeData #-} +{-# LANGUAGE TypeFamilies #-} +{-# OPTIONS_GHC -Wno-unused-top-binds #-} + +module Onnxruntime.CApi + ( -- * API Base + OrtApiVersion, + OrtApiVersionType, + ortApiVersion, + ortGetApiBase, + ortApiBaseGetVersionString, + ortApiBaseGetApi, + + -- * API Types + OrtApiBase, + ExecutionMode ( + OrtSequential, + OrtParallel + ), + GraphOptimizationLevel ( + OrtDisableAll, + OrtEnableBasic, + OrtEnableExtended, + OrtEnableAll + ), + ONNXTensorElementDataType ( + ONNXTensorElementDataTypeUndefined, + ONNXTensorElementDataTypeFloat, + ONNXTensorElementDataTypeUint8, + ONNXTensorElementDataTypeInt8, + ONNXTensorElementDataTypeUint16, + ONNXTensorElementDataTypeInt16, + ONNXTensorElementDataTypeInt32, + ONNXTensorElementDataTypeInt64, + ONNXTensorElementDataTypeString, + ONNXTensorElementDataTypeBool, + ONNXTensorElementDataTypeFloat16, + ONNXTensorElementDataTypeDouble, + ONNXTensorElementDataTypeUint32, + ONNXTensorElementDataTypeUint64, + ONNXTensorElementDataTypeComplex64, + ONNXTensorElementDataTypeComplex128, + ONNXTensorElementDataTypeBfloat16, + ONNXTensorElementDataTypeFloat8e4m3fn, + ONNXTensorElementDataTypeFloat8e4m3fnuz, + ONNXTensorElementDataTypeFloat8e5m2, + ONNXTensorElementDataTypeFloat8e5m2fnuz, + ONNXTensorElementDataTypeUint4, + ONNXTensorElementDataTypeInt4 + ), + ONNXType ( + ONNXTypeUnknown, + ONNXTypeTensor, + ONNXTypeSequence, + ONNXTypeMap, + ONNXTypeOpaque, + ONNXTypeSparseTensor, + ONNXTypeOptional + ), + OrtAllocatorType ( + OrtInvalidAllocator, + OrtDeviceAllocator, + OrtArenaAllocator + ), + OrtErrorCode ( + OrtOk, + OrtFail, + OrtInvalidArgument, + OrtNoSuchfile, + OrtNoModel, + OrtEngineError, + OrtRuntimeException, + OrtInvalidProtobuf, + OrtModelLoaded, + OrtNotImplemented, + OrtInvalidGraph, + OrtEpFail + ), + OrtLoggingLevel ( + OrtLoggingLevelVerbose, + OrtLoggingLevelInfo, + OrtLoggingLevelWarning, + OrtLoggingLevelError, + OrtLoggingLevelFatal + ), + OrtMemType ( + OrtMemTypeCPUInput, + OrtMemTypeCPUOutput, + OrtMemTypeCPU, + OrtMemTypeDefault + ), + OrtApi, + OrtAllocator, + OrtEnv, + OrtMapTypeInfo, + OrtMemoryInfo, + OrtSession, + OrtSessionOptions, + OrtTensorTypeAndShapeInfo, + OrtTypeInfo, + OrtRunOptions, + OrtValue, + + -- * API Functions + ortApiGetErrorMessageAsString, + ortApiGetErrorMessage, + ortApiCreateEnv, + ortApiCreateSession, + ortApiRun, + ortApiCreateSessionOptions, + ortApiCloneSessionOptions, + ortApiSetOptimizedModelFilePath, + ortApiSetSessionExecutionMode, + ortApiEnableProfiling, + ortApiDisableProfiling, + ortApiEnableMemPattern, + ortApiDisableMemPattern, + ortApiEnableCpuMemArena, + ortApiDisableCpuMemArena, + ortApiSetSessionLogId, + ortApiSetSessionLogVerbosityLevel, + ortApiSetSessionLogSeverityLevel, + ortApiSetSessionGraphOptimizationLevel, + ortApiSetIntraOpNumThreads, + ortApiSetInterOpNumThreads, + ortApiSessionGetInputCount, + ortApiSessionGetOutputCount, + ortApiSessionGetInputTypeInfo, + ortApiSessionGetOutputTypeInfo, + ortApiCreateRunOptions, + ortApiRunOptionsSetRunLogVerbosityLevel, + ortApiRunOptionsSetRunLogSeverityLevel, + ortApiRunOptionsSetRunTag, + ortApiRunOptionsGetRunLogVerbosityLevel, + ortApiRunOptionsGetRunLogSeverityLevel, + ortApiRunOptionsGetRunTag, + ortApiRunOptionsSetTerminate, + ortApiRunOptionsUnsetTerminate, + ortApiCreateTensorAsOrtValue, + ortApiWithTensorWithDataAsOrtValue, + ortApiIsTensor, + ortApiCheckType, + ortApiCheckTensorElementDataType, + ortApiWithTensorData, + ortApiCastTypeInfoToTensorInfo, + ortApiGetOnnxTypeFromTypeInfo, + ortApiGetTensorElementType, + ortApiGetDimensionsCount, + ortApiGetDimensions, + ortApiGetTensorShapeElementCount, + ortApiGetTensorTypeAndShape, + ortApiGetTypeInfo, + ortApiGetValueType, + ortApiCreateMemoryInfo, + ortApiCreateCpuMemoryInfo, + ortApiGetAllocatorWithDefaultOptions, + ortApiAddFreeDimensionOverride + ) where + +import Control.Exception (Exception (..), assert, finally, throwIO) +import Control.Monad (unless) +import Data.ByteString (ByteString) +import Data.ByteString qualified as BS +import Data.ByteString.Char8 qualified as BSC +import Data.Coerce (coerce) +import Data.Kind (Type) +import Data.Proxy (Proxy (..)) +import Data.Vector.Storable (Vector) +import Data.Vector.Storable qualified as VS +import Data.Void (Void) +import Foreign +import Foreign.C.ConstPtr.Compat (ConstPtr (..)) +import Foreign.C.Types +import Foreign.C.String +import GHC.TypeLits (Natural) +import Text.Printf (printf) + +#if __has_include() +#include +#elif __has_include() +#include +#elif __has_include() +#include +#endif + +------------------------------------------------------------------------------- +-- ONNX Runtime: API Base +------------------------------------------------------------------------------- + +------------------------------------------------------------------------------- +-- OrtApiVersion + +type OrtApiVersion :: Natural +type OrtApiVersion = #const ORT_API_VERSION + +type OrtApiVersionType :: Type +type OrtApiVersionType = #{type uint32_t} + +{- | +The API version defined in this module. + +This value is used by some API functions to behave as this version of the header expects. +-} +ortApiVersion :: OrtApiVersionType +ortApiVersion = #const ORT_API_VERSION + +------------------------------------------------------------------------------- +-- OrtApiBase + +{- | +The helper interface to get the right version of 'OrtApi'. + +Get a pointer to this structure through 'ortGetApiBase'. +-} +newtype + {-# CTYPE "onnxruntime_c_api.h" "OrtApiBase" #-} + OrtApiBase = OrtApiBase { ortApiBaseConstPtr :: ConstPtr OrtApiBase } + +------------------------------------------------------------------------------- +-- ortApiGetBase + +{- | +The Onnxruntime library's entry point to access the C API. + +Call this to get the a pointer to an 'OrtApiBase'. +-} +ortGetApiBase :: IO OrtApiBase +ortGetApiBase = coerce _wrap_ortGetApiBase +{-# INLINE ortGetApiBase #-} + +foreign import capi unsafe + "onnxruntime_c_api.h OrtGetApiBase" + _wrap_ortGetApiBase :: + IO (ConstPtr OrtApiBase) + +------------------------------------------------------------------------------- +-- OrtApiBase::GetVersionString + +{- | +Returns a null terminated string of the version of the Onnxruntime library (eg: "1.8.1"). +-} +ortApiBaseGetVersionString :: + OrtApiBase -> + IO String +ortApiBaseGetVersionString ortApiBase = do + ConstPtr versionStringPtr <- _wrap_OrtApiBase_GetVersionString ortApiBase.ortApiBaseConstPtr + peekCString versionStringPtr + +foreign import capi unsafe + "Onnxruntime/CApi_hsc.h _wrap_OrtApiBase_GetVersionString" + _wrap_OrtApiBase_GetVersionString :: + ConstPtr OrtApiBase -> + IO (ConstPtr CChar) + +#{def + const char* _wrap_OrtApiBase_GetVersionString(const OrtApiBase* ortApiBase) { + return ortApiBase->GetVersionString(); + } +} + +------------------------------------------------------------------------------- +-- OrtApiBase::GetApi + + +data OrtApiUnsupportedVersionError + = ErrOrtApiUnsupportedVersion + -- | Requested version.. + !OrtApiVersionType + deriving (Eq, Show) + +instance Exception OrtApiUnsupportedVersionError + +{- | +Get a pointer to the requested version of the 'OrtApi' +-} +ortApiBaseGetApi :: + OrtApiBase -> + OrtApiVersionType -> + IO OrtApi +ortApiBaseGetApi ortApiBase version = do + ortApi <- coerce _wrap_OrtApiBase_GetApi ortApiBase version + if ortApi == nullPtr + then throwIO (ErrOrtApiUnsupportedVersion version) + else pure (coerce ortApi) +{-# INLINE ortApiBaseGetApi #-} + +foreign import capi unsafe + "Onnxruntime/CApi_hsc.h _wrap_OrtApiBase_GetApi" + _wrap_OrtApiBase_GetApi :: + ConstPtr OrtApiBase -> + OrtApiVersionType -> + IO (ConstPtr OrtApi) + +#{def + const OrtApi* _wrap_OrtApiBase_GetApi(const OrtApiBase* ortApiBase, uint32_t version) { + return ortApiBase->GetApi(version); + } +} + + +------------------------------------------------------------------------------- +-- ONNX Runtime: Primitive Types +------------------------------------------------------------------------------- + +-- NOTE: This section contains those types which are passed by value. +-- NOTE: The definitions in this section are SORTED ALPHABETICALLY. + +------------------------------------------------------------------------------- +-- ExecutionMode + +{- | +> typedef enum ExecutionMode { +> ORT_SEQUENTIAL = 0, +> ORT_PARALLEL = 1, +> } ExecutionMode; +-} +newtype + {-# CTYPE "onnxruntime_c_api.h" "ExecutionMode" #-} + ExecutionMode = ExecutionMode + { unExecutionMode :: #{type ExecutionMode} + } + deriving (Eq, Show) + +pattern OrtSequential :: ExecutionMode +pattern OrtSequential = ExecutionMode ( #{const ORT_SEQUENTIAL} ) + +pattern OrtParallel :: ExecutionMode +pattern OrtParallel = ExecutionMode ( #{const ORT_PARALLEL} ) + +{-# COMPLETE + OrtSequential, + OrtParallel + #-} + +------------------------------------------------------------------------------- +-- GraphOptimizationLevel + +{- | +> typedef enum GraphOptimizationLevel { +> ORT_DISABLE_ALL = 0, +> ORT_ENABLE_BASIC = 1, +> ORT_ENABLE_EXTENDED = 2, +> ORT_ENABLE_ALL = 99 +> } GraphOptimizationLevel; +-} +newtype + {-# CTYPE "onnxruntime_c_api.h" "GraphOptimizationLevel" #-} + GraphOptimizationLevel = GraphOptimizationLevel + { unGraphOptimizationLevel :: #{type GraphOptimizationLevel} + } + deriving (Eq, Show) + + +pattern OrtDisableAll :: GraphOptimizationLevel +pattern OrtDisableAll = GraphOptimizationLevel ( #{const ORT_DISABLE_ALL} ) + +pattern OrtEnableBasic :: GraphOptimizationLevel +pattern OrtEnableBasic = GraphOptimizationLevel ( #{const ORT_ENABLE_BASIC} ) + +pattern OrtEnableExtended :: GraphOptimizationLevel +pattern OrtEnableExtended = GraphOptimizationLevel ( #{const ORT_ENABLE_EXTENDED} ) + +pattern OrtEnableAll :: GraphOptimizationLevel +pattern OrtEnableAll = GraphOptimizationLevel ( #{const ORT_ENABLE_ALL} ) + +{-# COMPLETE + OrtDisableAll, + OrtEnableBasic, + OrtEnableExtended, + OrtEnableAll + #-} + +------------------------------------------------------------------------------- +-- ONNXTensorElementDataType + +{- | +> typedef enum ONNXTensorElementDataType { +> ONNX_TENSOR_ELEMENT_DATA_TYPE_UNDEFINED, +> ONNX_TENSOR_ELEMENT_DATA_TYPE_FLOAT, +> ONNX_TENSOR_ELEMENT_DATA_TYPE_UINT8, +> ONNX_TENSOR_ELEMENT_DATA_TYPE_INT8, +> ONNX_TENSOR_ELEMENT_DATA_TYPE_UINT16, +> ONNX_TENSOR_ELEMENT_DATA_TYPE_INT16, +> ONNX_TENSOR_ELEMENT_DATA_TYPE_INT32, +> ONNX_TENSOR_ELEMENT_DATA_TYPE_INT64, +> ONNX_TENSOR_ELEMENT_DATA_TYPE_STRING, +> ONNX_TENSOR_ELEMENT_DATA_TYPE_BOOL, +> ONNX_TENSOR_ELEMENT_DATA_TYPE_FLOAT16, +> ONNX_TENSOR_ELEMENT_DATA_TYPE_DOUBLE, +> ONNX_TENSOR_ELEMENT_DATA_TYPE_UINT32, +> ONNX_TENSOR_ELEMENT_DATA_TYPE_UINT64, +> ONNX_TENSOR_ELEMENT_DATA_TYPE_COMPLEX64, +> ONNX_TENSOR_ELEMENT_DATA_TYPE_COMPLEX128, +> ONNX_TENSOR_ELEMENT_DATA_TYPE_BFLOAT16, +> ONNX_TENSOR_ELEMENT_DATA_TYPE_FLOAT8E4M3FN, +> ONNX_TENSOR_ELEMENT_DATA_TYPE_FLOAT8E4M3FNUZ, +> ONNX_TENSOR_ELEMENT_DATA_TYPE_FLOAT8E5M2, +> ONNX_TENSOR_ELEMENT_DATA_TYPE_FLOAT8E5M2FNUZ, +> ONNX_TENSOR_ELEMENT_DATA_TYPE_UINT4, +> ONNX_TENSOR_ELEMENT_DATA_TYPE_INT4 +> } ONNXTensorElementDataType; +-} +newtype + {-# CTYPE "onnxruntime_c_api.h" "ONNXTensorElementDataType" #-} + ONNXTensorElementDataType = ONNXTensorElementDataType + { unONNXTensorElementDataType :: #{type ONNXTensorElementDataType} + } + deriving (Eq) + +pattern ONNXTensorElementDataTypeUndefined :: ONNXTensorElementDataType +pattern ONNXTensorElementDataTypeUndefined = ONNXTensorElementDataType ( #{const ONNX_TENSOR_ELEMENT_DATA_TYPE_UNDEFINED} ) + +pattern ONNXTensorElementDataTypeFloat :: ONNXTensorElementDataType +pattern ONNXTensorElementDataTypeFloat = ONNXTensorElementDataType ( #{const ONNX_TENSOR_ELEMENT_DATA_TYPE_FLOAT} ) + +pattern ONNXTensorElementDataTypeUint8 :: ONNXTensorElementDataType +pattern ONNXTensorElementDataTypeUint8 = ONNXTensorElementDataType ( #{const ONNX_TENSOR_ELEMENT_DATA_TYPE_UINT8} ) + +pattern ONNXTensorElementDataTypeInt8 :: ONNXTensorElementDataType +pattern ONNXTensorElementDataTypeInt8 = ONNXTensorElementDataType ( #{const ONNX_TENSOR_ELEMENT_DATA_TYPE_INT8} ) + +pattern ONNXTensorElementDataTypeUint16 :: ONNXTensorElementDataType +pattern ONNXTensorElementDataTypeUint16 = ONNXTensorElementDataType ( #{const ONNX_TENSOR_ELEMENT_DATA_TYPE_UINT16} ) + +pattern ONNXTensorElementDataTypeInt16 :: ONNXTensorElementDataType +pattern ONNXTensorElementDataTypeInt16 = ONNXTensorElementDataType ( #{const ONNX_TENSOR_ELEMENT_DATA_TYPE_INT16} ) + +pattern ONNXTensorElementDataTypeInt32 :: ONNXTensorElementDataType +pattern ONNXTensorElementDataTypeInt32 = ONNXTensorElementDataType ( #{const ONNX_TENSOR_ELEMENT_DATA_TYPE_INT32} ) + +pattern ONNXTensorElementDataTypeInt64 :: ONNXTensorElementDataType +pattern ONNXTensorElementDataTypeInt64 = ONNXTensorElementDataType ( #{const ONNX_TENSOR_ELEMENT_DATA_TYPE_INT64} ) + +pattern ONNXTensorElementDataTypeString :: ONNXTensorElementDataType +pattern ONNXTensorElementDataTypeString = ONNXTensorElementDataType ( #{const ONNX_TENSOR_ELEMENT_DATA_TYPE_STRING} ) + +pattern ONNXTensorElementDataTypeBool :: ONNXTensorElementDataType +pattern ONNXTensorElementDataTypeBool = ONNXTensorElementDataType ( #{const ONNX_TENSOR_ELEMENT_DATA_TYPE_BOOL} ) + +pattern ONNXTensorElementDataTypeFloat16 :: ONNXTensorElementDataType +pattern ONNXTensorElementDataTypeFloat16 = ONNXTensorElementDataType ( #{const ONNX_TENSOR_ELEMENT_DATA_TYPE_FLOAT16} ) + +pattern ONNXTensorElementDataTypeDouble :: ONNXTensorElementDataType +pattern ONNXTensorElementDataTypeDouble = ONNXTensorElementDataType ( #{const ONNX_TENSOR_ELEMENT_DATA_TYPE_DOUBLE} ) + +pattern ONNXTensorElementDataTypeUint32 :: ONNXTensorElementDataType +pattern ONNXTensorElementDataTypeUint32 = ONNXTensorElementDataType ( #{const ONNX_TENSOR_ELEMENT_DATA_TYPE_UINT32} ) + +pattern ONNXTensorElementDataTypeUint64 :: ONNXTensorElementDataType +pattern ONNXTensorElementDataTypeUint64 = ONNXTensorElementDataType ( #{const ONNX_TENSOR_ELEMENT_DATA_TYPE_UINT64} ) + +pattern ONNXTensorElementDataTypeComplex64 :: ONNXTensorElementDataType +pattern ONNXTensorElementDataTypeComplex64 = ONNXTensorElementDataType ( #{const ONNX_TENSOR_ELEMENT_DATA_TYPE_COMPLEX64} ) + +pattern ONNXTensorElementDataTypeComplex128 :: ONNXTensorElementDataType +pattern ONNXTensorElementDataTypeComplex128 = ONNXTensorElementDataType ( #{const ONNX_TENSOR_ELEMENT_DATA_TYPE_COMPLEX128} ) + +pattern ONNXTensorElementDataTypeBfloat16 :: ONNXTensorElementDataType +pattern ONNXTensorElementDataTypeBfloat16 = ONNXTensorElementDataType ( #{const ONNX_TENSOR_ELEMENT_DATA_TYPE_BFLOAT16} ) + +pattern ONNXTensorElementDataTypeFloat8e4m3fn :: ONNXTensorElementDataType +pattern ONNXTensorElementDataTypeFloat8e4m3fn = ONNXTensorElementDataType ( #{const ONNX_TENSOR_ELEMENT_DATA_TYPE_FLOAT8E4M3FN} ) + +pattern ONNXTensorElementDataTypeFloat8e4m3fnuz :: ONNXTensorElementDataType +pattern ONNXTensorElementDataTypeFloat8e4m3fnuz = ONNXTensorElementDataType ( #{const ONNX_TENSOR_ELEMENT_DATA_TYPE_FLOAT8E4M3FNUZ} ) + +pattern ONNXTensorElementDataTypeFloat8e5m2 :: ONNXTensorElementDataType +pattern ONNXTensorElementDataTypeFloat8e5m2 = ONNXTensorElementDataType ( #{const ONNX_TENSOR_ELEMENT_DATA_TYPE_FLOAT8E5M2} ) + +pattern ONNXTensorElementDataTypeFloat8e5m2fnuz :: ONNXTensorElementDataType +pattern ONNXTensorElementDataTypeFloat8e5m2fnuz = ONNXTensorElementDataType ( #{const ONNX_TENSOR_ELEMENT_DATA_TYPE_FLOAT8E5M2FNUZ} ) + +pattern ONNXTensorElementDataTypeUint4 :: ONNXTensorElementDataType +pattern ONNXTensorElementDataTypeUint4 = ONNXTensorElementDataType ( #{const ONNX_TENSOR_ELEMENT_DATA_TYPE_UINT4} ) + +pattern ONNXTensorElementDataTypeInt4 :: ONNXTensorElementDataType +pattern ONNXTensorElementDataTypeInt4 = ONNXTensorElementDataType ( #{const ONNX_TENSOR_ELEMENT_DATA_TYPE_INT4} ) + +{-# COMPLETE + ONNXTensorElementDataTypeUndefined, + ONNXTensorElementDataTypeFloat, + ONNXTensorElementDataTypeUint8, + ONNXTensorElementDataTypeInt8, + ONNXTensorElementDataTypeUint16, + ONNXTensorElementDataTypeInt16, + ONNXTensorElementDataTypeInt32, + ONNXTensorElementDataTypeInt64, + ONNXTensorElementDataTypeString, + ONNXTensorElementDataTypeBool, + ONNXTensorElementDataTypeFloat16, + ONNXTensorElementDataTypeDouble, + ONNXTensorElementDataTypeUint32, + ONNXTensorElementDataTypeUint64, + ONNXTensorElementDataTypeComplex64, + ONNXTensorElementDataTypeComplex128, + ONNXTensorElementDataTypeBfloat16, + ONNXTensorElementDataTypeFloat8e4m3fn, + ONNXTensorElementDataTypeFloat8e4m3fnuz, + ONNXTensorElementDataTypeFloat8e5m2, + ONNXTensorElementDataTypeFloat8e5m2fnuz, + ONNXTensorElementDataTypeUint4, + ONNXTensorElementDataTypeInt4 + #-} + +instance Show ONNXTensorElementDataType where + show = \case + ONNXTensorElementDataTypeUndefined -> "ONNXTensorElementDataTypeUndefined" + ONNXTensorElementDataTypeFloat -> "ONNXTensorElementDataTypeFloat" + ONNXTensorElementDataTypeUint8 -> "ONNXTensorElementDataTypeUint8" + ONNXTensorElementDataTypeInt8 -> "ONNXTensorElementDataTypeInt8" + ONNXTensorElementDataTypeUint16 -> "ONNXTensorElementDataTypeUint16" + ONNXTensorElementDataTypeInt16 -> "ONNXTensorElementDataTypeInt16" + ONNXTensorElementDataTypeInt32 -> "ONNXTensorElementDataTypeInt32" + ONNXTensorElementDataTypeInt64 -> "ONNXTensorElementDataTypeInt64" + ONNXTensorElementDataTypeString -> "ONNXTensorElementDataTypeString" + ONNXTensorElementDataTypeBool -> "ONNXTensorElementDataTypeBool" + ONNXTensorElementDataTypeFloat16 -> "ONNXTensorElementDataTypeFloat16" + ONNXTensorElementDataTypeDouble -> "ONNXTensorElementDataTypeDouble" + ONNXTensorElementDataTypeUint32 -> "ONNXTensorElementDataTypeUint32" + ONNXTensorElementDataTypeUint64 -> "ONNXTensorElementDataTypeUint64" + ONNXTensorElementDataTypeComplex64 -> "ONNXTensorElementDataTypeComplex64" + ONNXTensorElementDataTypeComplex128 -> "ONNXTensorElementDataTypeComplex128" + ONNXTensorElementDataTypeBfloat16 -> "ONNXTensorElementDataTypeBfloat16" + ONNXTensorElementDataTypeFloat8e4m3fn -> "ONNXTensorElementDataTypeFloat8e4m3fn" + ONNXTensorElementDataTypeFloat8e4m3fnuz -> "ONNXTensorElementDataTypeFloat8e4m3fnuz" + ONNXTensorElementDataTypeFloat8e5m2 -> "ONNXTensorElementDataTypeFloat8e5m2" + ONNXTensorElementDataTypeFloat8e5m2fnuz -> "ONNXTensorElementDataTypeFloat8e5m2fnuz" + ONNXTensorElementDataTypeUint4 -> "ONNXTensorElementDataTypeUint4" + ONNXTensorElementDataTypeInt4 -> "ONNXTensorElementDataTypeInt4" + +class Storable a => IsONNXTensorElementDataType a where + getONNXTensorElementDataType :: Proxy a -> ONNXTensorElementDataType + +instance IsONNXTensorElementDataType Float where + getONNXTensorElementDataType _ = ONNXTensorElementDataTypeFloat + +instance IsONNXTensorElementDataType Double where + getONNXTensorElementDataType _ = ONNXTensorElementDataTypeDouble + +instance IsONNXTensorElementDataType Int8 where + getONNXTensorElementDataType _ = ONNXTensorElementDataTypeInt8 + +instance IsONNXTensorElementDataType Int16 where + getONNXTensorElementDataType _ = ONNXTensorElementDataTypeInt16 + +instance IsONNXTensorElementDataType Int32 where + getONNXTensorElementDataType _ = ONNXTensorElementDataTypeInt32 + +instance IsONNXTensorElementDataType Int64 where + getONNXTensorElementDataType _ = ONNXTensorElementDataTypeInt64 + +instance IsONNXTensorElementDataType Word8 where + getONNXTensorElementDataType _ = ONNXTensorElementDataTypeUint8 + +instance IsONNXTensorElementDataType Word16 where + getONNXTensorElementDataType _ = ONNXTensorElementDataTypeUint16 + +instance IsONNXTensorElementDataType Word32 where + getONNXTensorElementDataType _ = ONNXTensorElementDataTypeUint32 + +instance IsONNXTensorElementDataType Word64 where + getONNXTensorElementDataType _ = ONNXTensorElementDataTypeUint64 + +-- NOTE: The following 'ONNXTensorElementDataType' types are unsupported: +-- +-- [@ONNXTensorElementDataTypeUndefined@]: +-- Unsupported as input format for obvious reasons. +-- [@ONNXTensorElementDataTypeString@]: +-- Maps to C++ type std::string +-- [@ONNXTensorElementDataTypeFloat16@]: +-- Maps to float16_t +-- [@ONNXTensorElementDataTypeComplex64@]: +-- Maps to C++ type std::complex +-- [@ONNXTensorElementDataTypeComplex128@]: +-- Maps to C++ type std::complex +-- [@ONNXTensorElementDataTypeBfloat16@]: +-- Maps to non-IEEE floating-point format based on IEEE754 single-precision. +-- [@ONNXTensorElementDataTypeFloat8e4m3fn@]: +-- Maps to non-IEEE floating-point format based on IEEE754 single-precision. +-- [@ONNXTensorElementDataTypeFloat8e4m3fnuz@]: +-- Maps to non-IEEE floating-point format based on IEEE754 single-precision. +-- [@ONNXTensorElementDataTypeFloat8e5m2@]: +-- Maps to non-IEEE floating-point format based on IEEE754 single-precision. +-- [@ONNXTensorElementDataTypeFloat8e5m2fnuz@]: +-- Maps to non-IEEE floating-point format based on IEEE754 single-precision. +-- [@ONNXTensorElementDataTypeUint4@]: +-- Maps to a pair of packed uint4 values. +-- [@ONNXTensorElementDataTypeInt4@]: +-- Maps to a pair of packed int4 values. + +-- | Type-level tag for supported 'ONNXTensorElementDataType' types. +type data ONNXTensorElementDataTypeTag + = ONNXTensorElementDataTypeTagFloat + | ONNXTensorElementDataTypeTagDouble + | ONNXTensorElementDataTypeTagInt8 + | ONNXTensorElementDataTypeTagInt16 + | ONNXTensorElementDataTypeTagInt32 + | ONNXTensorElementDataTypeTagInt64 + | ONNXTensorElementDataTypeTagUint8 + | ONNXTensorElementDataTypeTagUint16 + | ONNXTensorElementDataTypeTagUint32 + | ONNXTensorElementDataTypeTagUint64 + +------------------------------------------------------------------------------- +-- ONNXType + +{- +> typedef enum ONNXType { +> ONNX_TYPE_UNKNOWN, +> ONNX_TYPE_TENSOR, +> ONNX_TYPE_SEQUENCE, +> ONNX_TYPE_MAP, +> ONNX_TYPE_OPAQUE, +> ONNX_TYPE_SPARSETENSOR, +> ONNX_TYPE_OPTIONAL +> } ONNXType; +-} +newtype + {-# CTYPE "onnxruntime_c_api.h" "ONNXType" #-} + ONNXType = ONNXType + { unONNXType :: #{type ONNXType} + } + deriving (Eq) + +pattern ONNXTypeUnknown :: ONNXType +pattern ONNXTypeUnknown = ONNXType ( #{const ONNX_TYPE_UNKNOWN} ) + +pattern ONNXTypeTensor :: ONNXType +pattern ONNXTypeTensor = ONNXType ( #{const ONNX_TYPE_TENSOR} ) + +pattern ONNXTypeSequence :: ONNXType +pattern ONNXTypeSequence = ONNXType ( #{const ONNX_TYPE_SEQUENCE} ) + +pattern ONNXTypeMap :: ONNXType +pattern ONNXTypeMap = ONNXType ( #{const ONNX_TYPE_MAP} ) + +pattern ONNXTypeOpaque :: ONNXType +pattern ONNXTypeOpaque = ONNXType ( #{const ONNX_TYPE_OPAQUE} ) + +pattern ONNXTypeSparseTensor :: ONNXType +pattern ONNXTypeSparseTensor = ONNXType ( #{const ONNX_TYPE_SPARSETENSOR} ) + +pattern ONNXTypeOptional :: ONNXType +pattern ONNXTypeOptional = ONNXType ( #{const ONNX_TYPE_OPTIONAL} ) + +{-# COMPLETE + ONNXTypeUnknown, + ONNXTypeTensor, + ONNXTypeSequence, + ONNXTypeMap, + ONNXTypeOpaque, + ONNXTypeSparseTensor, + ONNXTypeOptional + #-} + +instance Show ONNXType where + show = \case + ONNXTypeUnknown -> "ONNXTypeUnknown" + ONNXTypeTensor -> "ONNXTypeTensor" + ONNXTypeSequence -> "ONNXTypeSequence" + ONNXTypeMap -> "ONNXTypeMap" + ONNXTypeOpaque -> "ONNXTypeOpaque" + ONNXTypeSparseTensor -> "ONNXTypeSparseTensor" + ONNXTypeOptional -> "ONNXTypeOptional" + +-- | Type-level tag for supported 'ONNXType' types. +type data ONNXTypeTag + = ONNXTypeTagTensor ONNXTensorElementDataTypeTag + +------------------------------------------------------------------------------- +-- OrtAllocatorType + +{- | +> typedef enum OrtAllocatorType { +> OrtInvalidAllocator = -1, +> OrtDeviceAllocator = 0, +> OrtArenaAllocator = 1 +> } OrtAllocatorType; +-} +newtype + {-# CTYPE "onnxruntime_c_api.h" "OrtAllocatorType" #-} + OrtAllocatorType = OrtAllocatorType + { unOrtAllocatorType :: #{type OrtAllocatorType} + } + deriving (Eq, Show) + +pattern OrtInvalidAllocator :: OrtAllocatorType +pattern OrtInvalidAllocator = OrtAllocatorType ( #{const OrtInvalidAllocator} ) + +pattern OrtDeviceAllocator :: OrtAllocatorType +pattern OrtDeviceAllocator = OrtAllocatorType ( #{const OrtDeviceAllocator} ) + +pattern OrtArenaAllocator :: OrtAllocatorType +pattern OrtArenaAllocator = OrtAllocatorType ( #{const OrtArenaAllocator} ) + +{-# COMPLETE + OrtInvalidAllocator, + OrtDeviceAllocator, + OrtArenaAllocator + #-} + +------------------------------------------------------------------------------- +-- OrtErrorCode + +{- | +> typedef enum OrtErrorCode { +> ORT_OK, +> ORT_FAIL, +> ORT_INVALID_ARGUMENT, +> ORT_NO_SUCHFILE, +> ORT_NO_MODEL, +> ORT_ENGINE_ERROR, +> ORT_RUNTIME_EXCEPTION, +> ORT_INVALID_PROTOBUF, +> ORT_MODEL_LOADED, +> ORT_NOT_IMPLEMENTED, +> ORT_INVALID_GRAPH, +> ORT_EP_FAIL, +> } OrtErrorCode; +-} +newtype + {-# CTYPE "onnxruntime_c_api.h" "OrtErrorCode" #-} + OrtErrorCode = OrtErrorCode + { unOrtErrorCode :: #{type OrtErrorCode} + } + deriving (Eq, Show) + +pattern OrtOk :: OrtErrorCode +pattern OrtOk = OrtErrorCode ( #{const ORT_OK} ) + +pattern OrtFail :: OrtErrorCode +pattern OrtFail = OrtErrorCode ( #{const ORT_FAIL} ) + +pattern OrtInvalidArgument :: OrtErrorCode +pattern OrtInvalidArgument = OrtErrorCode ( #{const ORT_INVALID_ARGUMENT} ) + +pattern OrtNoSuchfile :: OrtErrorCode +pattern OrtNoSuchfile = OrtErrorCode ( #{const ORT_NO_SUCHFILE} ) + +pattern OrtNoModel :: OrtErrorCode +pattern OrtNoModel = OrtErrorCode ( #{const ORT_NO_MODEL} ) + +pattern OrtEngineError :: OrtErrorCode +pattern OrtEngineError = OrtErrorCode ( #{const ORT_ENGINE_ERROR} ) + +pattern OrtRuntimeException :: OrtErrorCode +pattern OrtRuntimeException = OrtErrorCode ( #{const ORT_RUNTIME_EXCEPTION} ) + +pattern OrtInvalidProtobuf :: OrtErrorCode +pattern OrtInvalidProtobuf = OrtErrorCode ( #{const ORT_INVALID_PROTOBUF} ) + +pattern OrtModelLoaded :: OrtErrorCode +pattern OrtModelLoaded = OrtErrorCode ( #{const ORT_MODEL_LOADED} ) + +pattern OrtNotImplemented :: OrtErrorCode +pattern OrtNotImplemented = OrtErrorCode ( #{const ORT_NOT_IMPLEMENTED} ) + +pattern OrtInvalidGraph :: OrtErrorCode +pattern OrtInvalidGraph = OrtErrorCode ( #{const ORT_INVALID_GRAPH} ) + +pattern OrtEpFail :: OrtErrorCode +pattern OrtEpFail = OrtErrorCode ( #{const ORT_EP_FAIL} ) + +{-# COMPLETE + OrtOk, + OrtFail, + OrtInvalidArgument, + OrtNoSuchfile, + OrtNoModel, + OrtEngineError, + OrtRuntimeException, + OrtInvalidProtobuf, + OrtModelLoaded, + OrtNotImplemented, + OrtInvalidGraph, + OrtEpFail + #-} + +instance Exception OrtErrorCode where + displayException = \case + OrtOk -> "ORT_OK" + OrtFail -> "ORT_FAIL" + OrtInvalidArgument -> "ORT_INVALID_ARGUMENT" + OrtNoSuchfile -> "ORT_NO_SUCHFILE" + OrtNoModel -> "ORT_NO_MODEL" + OrtEngineError -> "ORT_ENGINE_ERROR" + OrtRuntimeException -> "ORT_RUNTIME_EXCEPTION" + OrtInvalidProtobuf -> "ORT_INVALID_PROTOBUF" + OrtModelLoaded -> "ORT_MODEL_LOADED" + OrtNotImplemented -> "ORT_NOT_IMPLEMENTED" + OrtInvalidGraph -> "ORT_INVALID_GRAPH" + OrtEpFail -> "ORT_EP_FAIL" + +------------------------------------------------------------------------------- +-- OrtLoggingLevel + +{-| +> typedef enum OrtLoggingLevel { +> ORT_LOGGING_LEVEL_VERBOSE, ///< Verbose informational messages (least severe). +> ORT_LOGGING_LEVEL_INFO, ///< Informational messages. +> ORT_LOGGING_LEVEL_WARNING, ///< Warning messages. +> ORT_LOGGING_LEVEL_ERROR, ///< Error messages. +> ORT_LOGGING_LEVEL_FATAL, ///< Fatal error messages (most severe). +> } OrtLoggingLevel; +-} +newtype + {-# CTYPE "onnxruntime_c_api.h" "OrtLoggingLevel" #-} + OrtLoggingLevel = OrtLoggingLevel + { unOrtLoggingLevel :: #{type OrtLoggingLevel} + } + deriving (Eq, Show) + +pattern OrtLoggingLevelVerbose :: OrtLoggingLevel +pattern OrtLoggingLevelVerbose = OrtLoggingLevel ( #{const ORT_LOGGING_LEVEL_VERBOSE} ) + +pattern OrtLoggingLevelInfo :: OrtLoggingLevel +pattern OrtLoggingLevelInfo = OrtLoggingLevel ( #{const ORT_LOGGING_LEVEL_INFO} ) + +pattern OrtLoggingLevelWarning :: OrtLoggingLevel +pattern OrtLoggingLevelWarning = OrtLoggingLevel ( #{const ORT_LOGGING_LEVEL_WARNING} ) + +pattern OrtLoggingLevelError :: OrtLoggingLevel +pattern OrtLoggingLevelError = OrtLoggingLevel ( #{const ORT_LOGGING_LEVEL_ERROR} ) + +pattern OrtLoggingLevelFatal :: OrtLoggingLevel +pattern OrtLoggingLevelFatal = OrtLoggingLevel ( #{const ORT_LOGGING_LEVEL_FATAL} ) + +{-# COMPLETE + OrtLoggingLevelVerbose, + OrtLoggingLevelInfo, + OrtLoggingLevelWarning, + OrtLoggingLevelError, + OrtLoggingLevelFatal + #-} + +------------------------------------------------------------------------------- +-- OrtMemType + +{- | +> typedef enum OrtMemType { +> OrtMemTypeCPUInput = -2, +> OrtMemTypeCPUOutput = -1, +> OrtMemTypeCPU = OrtMemTypeCPUOutput, +> OrtMemTypeDefault = 0, +> } OrtMemType; +-} +newtype + {-# CTYPE "onnxruntime_c_api.h" "OrtMemType" #-} + OrtMemType = OrtMemType + { unOrtMemType :: #{type OrtMemType} + } + deriving (Eq, Show) + +pattern OrtMemTypeCPUInput :: OrtMemType +pattern OrtMemTypeCPUInput = OrtMemType ( #{const OrtMemTypeCPUInput} ) + +pattern OrtMemTypeCPUOutput :: OrtMemType +pattern OrtMemTypeCPUOutput = OrtMemType ( #{const OrtMemTypeCPUOutput} ) + +pattern OrtMemTypeCPU :: OrtMemType +pattern OrtMemTypeCPU = OrtMemType ( #{const OrtMemTypeCPU} ) + +pattern OrtMemTypeDefault :: OrtMemType +pattern OrtMemTypeDefault = OrtMemType ( #{const OrtMemTypeDefault} ) + +{-# COMPLETE + OrtMemTypeCPUInput, + OrtMemTypeCPUOutput, + OrtMemTypeCPU, + OrtMemTypeDefault + #-} + +------------------------------------------------------------------------------- +-- ONNX Runtime: Types +------------------------------------------------------------------------------- + +-- NOTE: This section contains those types which are passed by reference. +-- NOTE: The definitions in this section are SORTED ALPHABETICALLY. + +------------------------------------------------------------------------------- +-- OrtApi + +newtype + {-# CTYPE "onnxruntime_c_api.h" "OrtApi" #-} + OrtApi = OrtApi { ortApiConstPtr :: ConstPtr OrtApi } + +class HasOrtApi a where + type CType a + getOrtApi :: a -> IO OrtApi + withCTypePtr :: a -> (Ptr (CType a) -> IO b) -> IO b + +-- | Marshall a list of ONNX Runtime types as an array of pointers. +withCTypeArrayLen :: + (HasOrtApi a) => + [a] -> + (Int -> Ptr (Ptr (CType a)) -> IO b) -> + IO b +withCTypeArrayLen = withArrayLenWith withCTypePtr + +-- | Internal helper. +withCStringArrayLen :: + [String] -> + (Int -> Ptr CString -> IO a) -> + IO a +withCStringArrayLen = withArrayLenWith withCString + +-- | Internal helper. +withArrayLenWith :: + (forall c. a -> (Ptr b -> IO c) -> IO c) -> + [a] -> + (Int -> Ptr (Ptr b) -> IO r) -> + IO r +withArrayLenWith withPtr xs action = go xs [] + where + go [] acc = withArrayLen (reverse acc) action + go (y : ys) acc = withPtr y (\yPtr -> go ys (yPtr : acc)) + +------------------------------------------------------------------------------- +-- OrtAllocator + +data + {-# CTYPE "onnxruntime_c_api.h" "COrtAllocator" #-} + COrtAllocator + +#{def + typedef OrtAllocator COrtAllocator; +} + +newtype + {-# CTYPE "Onnxruntime/CApi_hsc.h" "HsOrtAllocator" #-} + OrtAllocator = OrtAllocator { ortAllocatorForeignPtr :: ForeignPtr OrtAllocator } + +#{def + typedef struct HsOrtAllocator { + const OrtApi* ortApi; + COrtAllocator* ortAllocator; + } HsOrtAllocator; +} + +instance HasOrtApi OrtAllocator where + type CType OrtAllocator = COrtAllocator + getOrtApi ortAllocator = + withOrtAllocatorPtr ortAllocator $ \ortAllocatorPtr -> + OrtApi <$> #{peek HsOrtAllocator, ortApi} ortAllocatorPtr + withCTypePtr = withCOrtAllocatorPtr + +-- | Internal helper. +withOrtAllocatorPtr :: + OrtAllocator -> + (Ptr OrtAllocator -> IO a) -> + IO a +withOrtAllocatorPtr ortAllocator = + withForeignPtr ortAllocator.ortAllocatorForeignPtr + +-- | Internal helper. +withCOrtAllocatorPtr :: + OrtAllocator -> + (Ptr COrtAllocator -> IO a) -> + IO a +withCOrtAllocatorPtr ortAllocator action = + withOrtAllocatorPtr ortAllocator $ \ortAllocatorPtr -> do + cOrtAllocatorPtr <- #{peek HsOrtAllocator, ortAllocator} ortAllocatorPtr + action cOrtAllocatorPtr + +-- | Internal helper. +wrapCOrtAllocator :: + OrtApi -> + Ptr COrtAllocator -> + IO OrtAllocator +wrapCOrtAllocator ortApi rawOrtAllocatorPtr = do + ortAllocatorPtr <- _wrap_COrtAllocator ortApi.ortApiConstPtr rawOrtAllocatorPtr + ortAllocatorForeignPtr <- newForeignPtr _wrap_OrtApi_ReleaseAllocator ortAllocatorPtr + pure $ OrtAllocator ortAllocatorForeignPtr + +foreign import capi unsafe + "Onnxruntime/CApi_hsc.h _wrap_COrtAllocator" + _wrap_COrtAllocator :: + ConstPtr OrtApi -> + Ptr COrtAllocator -> + IO (Ptr OrtAllocator) + +#{def + HsOrtAllocator* _wrap_COrtAllocator( + const OrtApi* ortApi, + COrtAllocator* ortAllocator + ) { + HsOrtAllocator *out = malloc(sizeof *out); + out->ortApi = ortApi; + out->ortAllocator = ortAllocator; + return out; + } +} + +foreign import capi unsafe + "Onnxruntime/CApi_hsc.h &_wrap_OrtApi_ReleaseAllocator" + _wrap_OrtApi_ReleaseAllocator :: + FunPtr ( + Ptr OrtAllocator -> + IO () + ) + +#{def + void _wrap_OrtApi_ReleaseAllocator(HsOrtAllocator* ortAllocator) { + ortAllocator->ortApi->ReleaseAllocator(ortAllocator->ortAllocator); + free(ortAllocator); + } +} + +------------------------------------------------------------------------------- +-- OrtEnv + +data + {-# CTYPE "onnxruntime_c_api.h" "COrtEnv" #-} + COrtEnv + +#{def + typedef OrtEnv COrtEnv; +} + +newtype + {-# CTYPE "Onnxruntime/CApi_hsc.h" "HsOrtEnv" #-} + OrtEnv = OrtEnv { ortEnvForeignPtr :: ForeignPtr OrtEnv } + +#{def + typedef struct HsOrtEnv { + const OrtApi* ortApi; + COrtEnv* ortEnv; + } HsOrtEnv; +} + +instance HasOrtApi OrtEnv where + type CType OrtEnv = COrtEnv + getOrtApi ortEnv = + withOrtEnvPtr ortEnv $ \ortEnvPtr -> + OrtApi <$> #{peek HsOrtEnv, ortApi} ortEnvPtr + withCTypePtr = withCOrtEnvPtr + +-- | Internal helper. +withOrtEnvPtr :: + OrtEnv -> + (Ptr OrtEnv -> IO a) -> + IO a +withOrtEnvPtr ortEnv = + withForeignPtr ortEnv.ortEnvForeignPtr + +-- | Internal helper. +withCOrtEnvPtr :: + OrtEnv -> + (Ptr COrtEnv -> IO a) -> + IO a +withCOrtEnvPtr ortEnv action = + withOrtEnvPtr ortEnv $ \ortEnvPtr -> do + cOrtEnvPtr <- #{peek HsOrtEnv, ortEnv} ortEnvPtr + action cOrtEnvPtr + +-- | Internal helper. +wrapCOrtEnv :: + OrtApi -> + Ptr COrtEnv -> + IO OrtEnv +wrapCOrtEnv ortApi rawOrtEnvPtr = do + ortEnvPtr <- _wrap_COrtEnv ortApi.ortApiConstPtr rawOrtEnvPtr + ortEnvForeignPtr <- newForeignPtr _wrap_OrtApi_ReleaseEnv ortEnvPtr + pure $ OrtEnv ortEnvForeignPtr + +foreign import capi unsafe + "Onnxruntime/CApi_hsc.h _wrap_COrtEnv" + _wrap_COrtEnv :: + ConstPtr OrtApi -> + Ptr COrtEnv -> + IO (Ptr OrtEnv) + +#{def + HsOrtEnv* _wrap_COrtEnv( + const OrtApi* ortApi, + COrtEnv* ortEnv + ) { + HsOrtEnv *out = malloc(sizeof *out); + out->ortApi = ortApi; + out->ortEnv = ortEnv; + return out; + } +} + +foreign import capi unsafe + "Onnxruntime/CApi_hsc.h &_wrap_OrtApi_ReleaseEnv" + _wrap_OrtApi_ReleaseEnv :: + FunPtr ( + Ptr OrtEnv -> + IO () + ) + +#{def + void _wrap_OrtApi_ReleaseEnv(HsOrtEnv* ortEnv) { + ortEnv->ortApi->ReleaseEnv(ortEnv->ortEnv); + free(ortEnv); + } +} + +------------------------------------------------------------------------------- +-- OrtMapTypeInfo + +data + {-# CTYPE "onnxruntime_c_api.h" "COrtMapTypeInfo" #-} + COrtMapTypeInfo + +#{def + typedef OrtMapTypeInfo COrtMapTypeInfo; +} + +newtype + {-# CTYPE "Onnxruntime/CApi_hsc.h" "HsOrtMapTypeInfo" #-} + OrtMapTypeInfo = OrtMapTypeInfo { ortMapTypeInfoForeignPtr :: ForeignPtr OrtMapTypeInfo } + +#{def + typedef struct HsOrtMapTypeInfo { + const OrtApi* ortApi; + COrtMapTypeInfo* ortMapTypeInfo; + } HsOrtMapTypeInfo; +} + +instance HasOrtApi OrtMapTypeInfo where + type CType OrtMapTypeInfo = COrtMapTypeInfo + getOrtApi ortMapTypeInfo = + withOrtMapTypeInfoPtr ortMapTypeInfo $ \ortMapTypeInfoPtr -> + OrtApi <$> #{peek HsOrtMapTypeInfo, ortApi} ortMapTypeInfoPtr + withCTypePtr = withCOrtMapTypeInfoPtr + +-- | Internal helper. +withOrtMapTypeInfoPtr :: + OrtMapTypeInfo -> + (Ptr OrtMapTypeInfo -> IO a) -> + IO a +withOrtMapTypeInfoPtr ortMapTypeInfo = + withForeignPtr ortMapTypeInfo.ortMapTypeInfoForeignPtr + +-- | Internal helper. +withCOrtMapTypeInfoPtr :: + OrtMapTypeInfo -> + (Ptr COrtMapTypeInfo -> IO a) -> + IO a +withCOrtMapTypeInfoPtr ortMapTypeInfo action = + withOrtMapTypeInfoPtr ortMapTypeInfo $ \ortMapTypeInfoPtr -> do + cOrtMapTypeInfoPtr <- #{peek HsOrtMapTypeInfo, ortMapTypeInfo} ortMapTypeInfoPtr + action cOrtMapTypeInfoPtr + +-- | Internal helper. +wrapCOrtMapTypeInfo :: + OrtApi -> + Ptr COrtMapTypeInfo -> + IO OrtMapTypeInfo +wrapCOrtMapTypeInfo ortApi rawOrtMapTypeInfoPtr = do + ortMapTypeInfoPtr <- _wrap_COrtMapTypeInfo ortApi.ortApiConstPtr rawOrtMapTypeInfoPtr + ortMapTypeInfoForeignPtr <- newForeignPtr _wrap_OrtApi_ReleaseMapTypeInfo ortMapTypeInfoPtr + pure $ OrtMapTypeInfo ortMapTypeInfoForeignPtr + +foreign import capi unsafe + "Onnxruntime/CApi_hsc.h _wrap_COrtMapTypeInfo" + _wrap_COrtMapTypeInfo :: + ConstPtr OrtApi -> + Ptr COrtMapTypeInfo -> + IO (Ptr OrtMapTypeInfo) + +#{def + HsOrtMapTypeInfo* _wrap_COrtMapTypeInfo( + const OrtApi* ortApi, + COrtMapTypeInfo* ortMapTypeInfo + ) { + HsOrtMapTypeInfo *out = malloc(sizeof *out); + out->ortApi = ortApi; + out->ortMapTypeInfo = ortMapTypeInfo; + return out; + } +} + +foreign import capi unsafe + "Onnxruntime/CApi_hsc.h &_wrap_OrtApi_ReleaseMapTypeInfo" + _wrap_OrtApi_ReleaseMapTypeInfo :: + FunPtr ( + Ptr OrtMapTypeInfo -> + IO () + ) + +#{def + void _wrap_OrtApi_ReleaseMapTypeInfo(HsOrtMapTypeInfo* ortMapTypeInfo) { + ortMapTypeInfo->ortApi->ReleaseMapTypeInfo(ortMapTypeInfo->ortMapTypeInfo); + free(ortMapTypeInfo); + } +} + +------------------------------------------------------------------------------- +-- OrtMemoryInfo + +data + {-# CTYPE "onnxruntime_c_api.h" "COrtMemoryInfo" #-} + COrtMemoryInfo + +#{def + typedef OrtMemoryInfo COrtMemoryInfo; +} + +newtype + {-# CTYPE "Onnxruntime/CApi_hsc.h" "HsOrtMemoryInfo" #-} + OrtMemoryInfo = OrtMemoryInfo { ortMemoryInfoForeignPtr :: ForeignPtr OrtMemoryInfo } + +#{def + typedef struct HsOrtMemoryInfo { + const OrtApi* ortApi; + COrtMemoryInfo* ortMemoryInfo; + } HsOrtMemoryInfo; +} + +instance HasOrtApi OrtMemoryInfo where + type CType OrtMemoryInfo = COrtMemoryInfo + getOrtApi ortMemoryInfo = + withOrtMemoryInfoPtr ortMemoryInfo $ \ortMemoryInfoPtr -> + OrtApi <$> #{peek HsOrtMemoryInfo, ortApi} ortMemoryInfoPtr + withCTypePtr = withCOrtMemoryInfoPtr + +-- | Internal helper. +withOrtMemoryInfoPtr :: + OrtMemoryInfo -> + (Ptr OrtMemoryInfo -> IO a) -> + IO a +withOrtMemoryInfoPtr ortMemoryInfo = + withForeignPtr ortMemoryInfo.ortMemoryInfoForeignPtr + +-- | Internal helper. +withCOrtMemoryInfoPtr :: + OrtMemoryInfo -> + (Ptr COrtMemoryInfo -> IO a) -> + IO a +withCOrtMemoryInfoPtr ortMemoryInfo action = + withOrtMemoryInfoPtr ortMemoryInfo $ \ortMemoryInfoPtr -> do + cOrtMemoryInfoPtr <- #{peek HsOrtMemoryInfo, ortMemoryInfo} ortMemoryInfoPtr + action cOrtMemoryInfoPtr + +-- | Internal helper. +wrapCOrtMemoryInfo :: + OrtApi -> + Ptr COrtMemoryInfo -> + IO OrtMemoryInfo +wrapCOrtMemoryInfo ortApi rawOrtMemoryInfoPtr = do + ortMemoryInfoPtr <- _wrap_COrtMemoryInfo ortApi.ortApiConstPtr rawOrtMemoryInfoPtr + ortMemoryInfoForeignPtr <- newForeignPtr _wrap_OrtApi_ReleaseMemoryInfo ortMemoryInfoPtr + pure $ OrtMemoryInfo ortMemoryInfoForeignPtr + +foreign import capi unsafe + "Onnxruntime/CApi_hsc.h _wrap_COrtMemoryInfo" + _wrap_COrtMemoryInfo :: + ConstPtr OrtApi -> + Ptr COrtMemoryInfo -> + IO (Ptr OrtMemoryInfo) + +#{def + HsOrtMemoryInfo* _wrap_COrtMemoryInfo( + const OrtApi* ortApi, + COrtMemoryInfo* ortMemoryInfo + ) { + HsOrtMemoryInfo *out = malloc(sizeof *out); + out->ortApi = ortApi; + out->ortMemoryInfo = ortMemoryInfo; + return out; + } +} + +foreign import capi unsafe + "Onnxruntime/CApi_hsc.h &_wrap_OrtApi_ReleaseMemoryInfo" + _wrap_OrtApi_ReleaseMemoryInfo :: + FunPtr ( + Ptr OrtMemoryInfo -> + IO () + ) + +#{def + void _wrap_OrtApi_ReleaseMemoryInfo(HsOrtMemoryInfo* ortMemoryInfo) { + ortMemoryInfo->ortApi->ReleaseMemoryInfo(ortMemoryInfo->ortMemoryInfo); + free(ortMemoryInfo); + } +} + +------------------------------------------------------------------------------- +-- OrtSession + +data + {-# CTYPE "onnxruntime_c_api.h" "COrtSession" #-} + COrtSession + +#{def + typedef OrtSession COrtSession; +} + +newtype + {-# CTYPE "Onnxruntime/CApi_hsc.h" "HsOrtSession" #-} + OrtSession = OrtSession { ortSessionForeignPtr :: ForeignPtr OrtSession } + +#{def + typedef struct HsOrtSession { + const OrtApi* ortApi; + COrtSession* ortSession; + } HsOrtSession; +} + +instance HasOrtApi OrtSession where + type CType OrtSession = COrtSession + getOrtApi ortSession = + withOrtSessionPtr ortSession $ \ortSessionPtr -> + OrtApi <$> #{peek HsOrtSession, ortApi} ortSessionPtr + withCTypePtr = withCOrtSessionPtr + +-- | Internal helper. +withOrtSessionPtr :: + OrtSession -> + (Ptr OrtSession -> IO a) -> + IO a +withOrtSessionPtr ortSession = + withForeignPtr ortSession.ortSessionForeignPtr + +-- | Internal helper. +withCOrtSessionPtr :: + OrtSession -> + (Ptr COrtSession -> IO a) -> + IO a +withCOrtSessionPtr ortSession action = + withOrtSessionPtr ortSession $ \ortSessionPtr -> do + cOrtSessionPtr <- #{peek HsOrtSession, ortSession} ortSessionPtr + action cOrtSessionPtr + +-- | Internal helper. +wrapCOrtSession :: + OrtApi -> + Ptr COrtSession -> + IO OrtSession +wrapCOrtSession ortApi rawOrtSessionPtr = do + ortSessionPtr <- _wrap_COrtSession ortApi.ortApiConstPtr rawOrtSessionPtr + ortSessionForeignPtr <- newForeignPtr _wrap_OrtApi_ReleaseSession ortSessionPtr + pure $ OrtSession ortSessionForeignPtr + +foreign import capi unsafe + "Onnxruntime/CApi_hsc.h _wrap_COrtSession" + _wrap_COrtSession :: + ConstPtr OrtApi -> + Ptr COrtSession -> + IO (Ptr OrtSession) + +#{def + HsOrtSession* _wrap_COrtSession( + const OrtApi* ortApi, + COrtSession* ortSession + ) { + HsOrtSession *out = malloc(sizeof *out); + out->ortApi = ortApi; + out->ortSession = ortSession; + return out; + } +} + +foreign import capi unsafe + "Onnxruntime/CApi_hsc.h &_wrap_OrtApi_ReleaseSession" + _wrap_OrtApi_ReleaseSession :: + FunPtr ( + Ptr OrtSession -> + IO () + ) + +#{def + void _wrap_OrtApi_ReleaseSession(HsOrtSession* ortSession) { + ortSession->ortApi->ReleaseSession(ortSession->ortSession); + free(ortSession); + } +} + +------------------------------------------------------------------------------- +-- OrtSessionOptions + +data + {-# CTYPE "onnxruntime_c_api.h" "COrtSessionOptions" #-} + COrtSessionOptions + +#{def + typedef OrtSessionOptions COrtSessionOptions; +} + +newtype + {-# CTYPE "Onnxruntime/CApi_hsc.h" "HsOrtSessionOptions" #-} + OrtSessionOptions = OrtSessionOptions { ortSessionOptionsForeignPtr :: ForeignPtr OrtSessionOptions } + +#{def + typedef struct HsOrtSessionOptions { + const OrtApi* ortApi; + COrtSessionOptions* ortSessionOptions; + } HsOrtSessionOptions; +} + +instance HasOrtApi OrtSessionOptions where + type CType OrtSessionOptions = COrtSessionOptions + getOrtApi ortSessionOptions = + withOrtSessionOptionsPtr ortSessionOptions $ \ortSessionOptionsPtr -> + OrtApi <$> #{peek HsOrtSessionOptions, ortApi} ortSessionOptionsPtr + withCTypePtr = withCOrtSessionOptionsPtr + +-- | Internal helper. +withOrtSessionOptionsPtr :: + OrtSessionOptions -> + (Ptr OrtSessionOptions -> IO a) -> + IO a +withOrtSessionOptionsPtr ortSessionOptions = + withForeignPtr ortSessionOptions.ortSessionOptionsForeignPtr + +-- | Internal helper. +withCOrtSessionOptionsPtr :: + OrtSessionOptions -> + (Ptr COrtSessionOptions -> IO a) -> + IO a +withCOrtSessionOptionsPtr ortSessionOptions action = + withOrtSessionOptionsPtr ortSessionOptions $ \ortSessionOptionsPtr -> do + cOrtSessionOptionsPtr <- #{peek HsOrtSessionOptions, ortSessionOptions} ortSessionOptionsPtr + action cOrtSessionOptionsPtr + +-- | Internal helper. +wrapCOrtSessionOptions :: + OrtApi -> + Ptr COrtSessionOptions -> + IO OrtSessionOptions +wrapCOrtSessionOptions ortApi rawOrtSessionOptionsPtr = do + ortSessionOptionsPtr <- _wrap_COrtSessionOptions ortApi.ortApiConstPtr rawOrtSessionOptionsPtr + ortSessionOptionsForeignPtr <- newForeignPtr _wrap_OrtApi_ReleaseSessionOptions ortSessionOptionsPtr + pure $ OrtSessionOptions ortSessionOptionsForeignPtr + +foreign import capi unsafe + "Onnxruntime/CApi_hsc.h _wrap_COrtSessionOptions" + _wrap_COrtSessionOptions :: + ConstPtr OrtApi -> + Ptr COrtSessionOptions -> + IO (Ptr OrtSessionOptions) + +#{def + HsOrtSessionOptions* _wrap_COrtSessionOptions( + const OrtApi* ortApi, + COrtSessionOptions* ortSessionOptions + ) { + HsOrtSessionOptions *out = malloc(sizeof *out); + out->ortApi = ortApi; + out->ortSessionOptions = ortSessionOptions; + return out; + } +} + +foreign import capi unsafe + "Onnxruntime/CApi_hsc.h &_wrap_OrtApi_ReleaseSessionOptions" + _wrap_OrtApi_ReleaseSessionOptions :: + FunPtr ( + Ptr OrtSessionOptions -> + IO () + ) + +#{def + void _wrap_OrtApi_ReleaseSessionOptions(HsOrtSessionOptions* ortSessionOptions) { + ortSessionOptions->ortApi->ReleaseSessionOptions(ortSessionOptions->ortSessionOptions); + free(ortSessionOptions); + } +} + +------------------------------------------------------------------------------- +-- OrtStatus + +data + {-# CTYPE "onnxruntime_c_api.h" "OrtStatus" #-} + OrtStatus + +foreign import capi unsafe + "Onnxruntime/CApi_hsc.h _wrap_OrtApi_ReleaseStatus" + _wrap_OrtApi_ReleaseStatus :: + OrtApi -> + Ptr OrtStatus -> + IO () + +#{def + void _wrap_OrtApi_ReleaseStatus(const OrtApi* ortApi, OrtStatus* ortStatus) { + ortApi->ReleaseStatus(ortStatus); + } +} + +data OrtError = OrtError + { ortErrorCode :: {-# UNPACK #-} !OrtErrorCode + , ortErrorMessage :: {-# UNPACK #-} !ByteString + } + deriving stock (Eq, Show) + +instance Exception OrtError where + displayException ortError = + printf "ERROR[%s]: %s" + (displayException ortError.ortErrorCode) + (BSC.unpack ortError.ortErrorMessage) + +handleOrtStatus :: + OrtApi -> + Ptr OrtStatus -> + IO a -> + IO a +handleOrtStatus ortApi ortStatusPtr action + | ortStatusPtr == nullPtr = action + | otherwise = do + let actionOrError = do + ortErrorCode <- ortApiGetErrorCode ortApi ortStatusPtr + if ortErrorCode == OrtOk then action else do + ortErrorMessage <- ortApiGetErrorMessage ortApi ortStatusPtr + throwIO OrtError {..} + let cleanupStatus = + _wrap_OrtApi_ReleaseStatus ortApi ortStatusPtr + actionOrError `finally` cleanupStatus + +------------------------------------------------------------------------------- +-- OrtTensorTypeAndShapeInfo + +data + {-# CTYPE "onnxruntime_c_api.h" "COrtTensorTypeAndShapeInfo" #-} + COrtTensorTypeAndShapeInfo + +#{def + typedef OrtTensorTypeAndShapeInfo COrtTensorTypeAndShapeInfo; +} + +data + {-# CTYPE "Onnxruntime/CApi_hsc.h" "HsOrtTensorTypeAndShapeInfo" #-} + OrtTensorTypeAndShapeInfo + = OrtTensorTypeAndShapeInfo { ortTensorTypeAndShapeInfoForeignPtr :: ForeignPtr OrtTensorTypeAndShapeInfo } + | OrtTensorTypeAndShapeInfoFromOrtTypeInfo { ortTypeInfo :: OrtTypeInfo, ortTensorTypeAndShapeInfoForeignPtr :: ForeignPtr OrtTensorTypeAndShapeInfo } + +#{def + typedef struct HsOrtTensorTypeAndShapeInfo { + const OrtApi* ortApi; + COrtTensorTypeAndShapeInfo* ortTensorTypeAndShapeInfo; + } HsOrtTensorTypeAndShapeInfo; +} + +instance HasOrtApi OrtTensorTypeAndShapeInfo where + type CType OrtTensorTypeAndShapeInfo = COrtTensorTypeAndShapeInfo + getOrtApi ortTensorTypeAndShapeInfo = + withOrtTensorTypeAndShapeInfoPtr ortTensorTypeAndShapeInfo $ \ortTensorTypeAndShapeInfoPtr -> + OrtApi <$> #{peek HsOrtTensorTypeAndShapeInfo, ortApi} ortTensorTypeAndShapeInfoPtr + withCTypePtr = withCOrtTensorTypeAndShapeInfoPtr + +-- | Internal helper. +withOrtTensorTypeAndShapeInfoPtr :: + OrtTensorTypeAndShapeInfo -> + (Ptr OrtTensorTypeAndShapeInfo -> IO a) -> + IO a +withOrtTensorTypeAndShapeInfoPtr ortTensorTypeAndShapeInfo = + withForeignPtr ortTensorTypeAndShapeInfo.ortTensorTypeAndShapeInfoForeignPtr + +-- | Internal helper. +withCOrtTensorTypeAndShapeInfoPtr :: + OrtTensorTypeAndShapeInfo -> + (Ptr COrtTensorTypeAndShapeInfo -> IO a) -> + IO a +withCOrtTensorTypeAndShapeInfoPtr ortTensorTypeAndShapeInfo action = + withOrtTensorTypeAndShapeInfoPtr ortTensorTypeAndShapeInfo $ \ortTensorTypeAndShapeInfoPtr -> do + cOrtTensorTypeAndShapeInfoPtr <- #{peek HsOrtTensorTypeAndShapeInfo, ortTensorTypeAndShapeInfo} ortTensorTypeAndShapeInfoPtr + action cOrtTensorTypeAndShapeInfoPtr + +-- | Internal helper. +wrapCOrtTensorTypeAndShapeInfo :: + OrtApi -> + Ptr COrtTensorTypeAndShapeInfo -> + IO OrtTensorTypeAndShapeInfo +wrapCOrtTensorTypeAndShapeInfo ortApi rawOrtTensorTypeAndShapeInfoPtr = do + ortTensorTypeAndShapeInfoPtr <- _wrap_COrtTensorTypeAndShapeInfo ortApi.ortApiConstPtr rawOrtTensorTypeAndShapeInfoPtr + ortTensorTypeAndShapeInfoForeignPtr <- newForeignPtr _wrap_OrtApi_ReleaseTensorTypeAndShapeInfo ortTensorTypeAndShapeInfoPtr + pure $ OrtTensorTypeAndShapeInfo ortTensorTypeAndShapeInfoForeignPtr + +foreign import capi unsafe + "Onnxruntime/CApi_hsc.h _wrap_COrtTensorTypeAndShapeInfo" + _wrap_COrtTensorTypeAndShapeInfo :: + ConstPtr OrtApi -> + Ptr COrtTensorTypeAndShapeInfo -> + IO (Ptr OrtTensorTypeAndShapeInfo) + +#{def + HsOrtTensorTypeAndShapeInfo* _wrap_COrtTensorTypeAndShapeInfo( + const OrtApi* ortApi, + COrtTensorTypeAndShapeInfo* ortTensorTypeAndShapeInfo + ) { + HsOrtTensorTypeAndShapeInfo *out = malloc(sizeof *out); + out->ortApi = ortApi; + out->ortTensorTypeAndShapeInfo = ortTensorTypeAndShapeInfo; + return out; + } +} + +foreign import capi unsafe + "Onnxruntime/CApi_hsc.h &_wrap_OrtApi_ReleaseTensorTypeAndShapeInfo" + _wrap_OrtApi_ReleaseTensorTypeAndShapeInfo :: + FunPtr ( + Ptr OrtTensorTypeAndShapeInfo -> + IO () + ) + +#{def + void _wrap_OrtApi_ReleaseTensorTypeAndShapeInfo(HsOrtTensorTypeAndShapeInfo* ortTensorTypeAndShapeInfo) { + ortTensorTypeAndShapeInfo->ortApi->ReleaseTensorTypeAndShapeInfo(ortTensorTypeAndShapeInfo->ortTensorTypeAndShapeInfo); + free(ortTensorTypeAndShapeInfo); + } +} + +-- | Internal helper. +wrapCOrtTensorTypeAndShapeInfoFromOrtTypeInfo :: + OrtApi -> + OrtTypeInfo -> + Ptr COrtTensorTypeAndShapeInfo -> + IO OrtTensorTypeAndShapeInfo +wrapCOrtTensorTypeAndShapeInfoFromOrtTypeInfo ortApi ortTypeInfo rawOrtTensorTypeAndShapeInfoPtr = do + ortTensorTypeAndShapeInfoPtr <- _wrap_COrtTensorTypeAndShapeInfo ortApi.ortApiConstPtr rawOrtTensorTypeAndShapeInfoPtr + ortTensorTypeAndShapeInfoForeignPtr <- newForeignPtr _wrap_OrtApi_OrtTensorTypeAndShapeInfoFromOrtTypeInfo ortTensorTypeAndShapeInfoPtr + pure $ OrtTensorTypeAndShapeInfoFromOrtTypeInfo ortTypeInfo ortTensorTypeAndShapeInfoForeignPtr + +foreign import capi unsafe + "Onnxruntime/CApi_hsc.h &_wrap_OrtApi_OrtTensorTypeAndShapeInfoFromOrtTypeInfo" + _wrap_OrtApi_OrtTensorTypeAndShapeInfoFromOrtTypeInfo :: + FunPtr ( + Ptr OrtTensorTypeAndShapeInfo -> + IO () + ) + +#{def + void _wrap_OrtApi_OrtTensorTypeAndShapeInfoFromOrtTypeInfo(HsOrtTensorTypeAndShapeInfo* ortTensorTypeAndShapeInfo) { + free(ortTensorTypeAndShapeInfo); + } +} + +------------------------------------------------------------------------------- +-- OrtTypeInfo + +data + {-# CTYPE "onnxruntime_c_api.h" "COrtTypeInfo" #-} + COrtTypeInfo + +#{def + typedef OrtTypeInfo COrtTypeInfo; +} + +newtype + {-# CTYPE "Onnxruntime/CApi_hsc.h" "HsOrtTypeInfo" #-} + OrtTypeInfo = OrtTypeInfo { ortTypeInfoForeignPtr :: ForeignPtr OrtTypeInfo } + +#{def + typedef struct HsOrtTypeInfo { + const OrtApi* ortApi; + COrtTypeInfo* ortTypeInfo; + } HsOrtTypeInfo; +} + +instance HasOrtApi OrtTypeInfo where + type CType OrtTypeInfo = COrtTypeInfo + getOrtApi ortTypeInfo = + withOrtTypeInfoPtr ortTypeInfo $ \ortTypeInfoPtr -> + OrtApi <$> #{peek HsOrtTypeInfo, ortApi} ortTypeInfoPtr + withCTypePtr = withCOrtTypeInfoPtr + +-- | Internal helper. +withOrtTypeInfoPtr :: + OrtTypeInfo -> + (Ptr OrtTypeInfo -> IO a) -> + IO a +withOrtTypeInfoPtr ortTypeInfo = + withForeignPtr ortTypeInfo.ortTypeInfoForeignPtr + +-- | Internal helper. +withCOrtTypeInfoPtr :: + OrtTypeInfo -> + (Ptr COrtTypeInfo -> IO a) -> + IO a +withCOrtTypeInfoPtr ortTypeInfo action = + withOrtTypeInfoPtr ortTypeInfo $ \ortTypeInfoPtr -> do + cOrtTypeInfoPtr <- #{peek HsOrtTypeInfo, ortTypeInfo} ortTypeInfoPtr + action cOrtTypeInfoPtr + +-- | Internal helper. +wrapCOrtTypeInfo :: + OrtApi -> + Ptr COrtTypeInfo -> + IO OrtTypeInfo +wrapCOrtTypeInfo ortApi rawOrtTypeInfoPtr = do + ortTypeInfoPtr <- _wrap_COrtTypeInfo ortApi.ortApiConstPtr rawOrtTypeInfoPtr + ortTypeInfoForeignPtr <- newForeignPtr _wrap_OrtApi_ReleaseTypeInfo ortTypeInfoPtr + pure $ OrtTypeInfo ortTypeInfoForeignPtr + +foreign import capi unsafe + "Onnxruntime/CApi_hsc.h _wrap_COrtTypeInfo" + _wrap_COrtTypeInfo :: + ConstPtr OrtApi -> + Ptr COrtTypeInfo -> + IO (Ptr OrtTypeInfo) + +#{def + HsOrtTypeInfo* _wrap_COrtTypeInfo( + const OrtApi* ortApi, + COrtTypeInfo* ortTypeInfo + ) { + HsOrtTypeInfo *out = malloc(sizeof *out); + out->ortApi = ortApi; + out->ortTypeInfo = ortTypeInfo; + return out; + } +} + +foreign import capi unsafe + "Onnxruntime/CApi_hsc.h &_wrap_OrtApi_ReleaseTypeInfo" + _wrap_OrtApi_ReleaseTypeInfo :: + FunPtr ( + Ptr OrtTypeInfo -> + IO () + ) + +#{def + void _wrap_OrtApi_ReleaseTypeInfo(HsOrtTypeInfo* ortTypeInfo) { + ortTypeInfo->ortApi->ReleaseTypeInfo(ortTypeInfo->ortTypeInfo); + free(ortTypeInfo); + } +} + +------------------------------------------------------------------------------- +-- OrtRunOptions + +data + {-# CTYPE "onnxruntime_c_api.h" "COrtRunOptions" #-} + COrtRunOptions + +#{def + typedef OrtRunOptions COrtRunOptions; +} + +newtype + {-# CTYPE "Onnxruntime/CApi_hsc.h" "HsOrtRunOptions" #-} + OrtRunOptions = OrtRunOptions { ortRunOptionsForeignPtr :: ForeignPtr OrtRunOptions } + +#{def + typedef struct HsOrtRunOptions { + const OrtApi* ortApi; + COrtRunOptions* ortRunOptions; + } HsOrtRunOptions; +} + +instance HasOrtApi OrtRunOptions where + type CType OrtRunOptions = COrtRunOptions + getOrtApi ortRunOptions = + withOrtRunOptionsPtr ortRunOptions $ \ortRunOptionsPtr -> + OrtApi <$> #{peek HsOrtRunOptions, ortApi} ortRunOptionsPtr + withCTypePtr = withCOrtRunOptionsPtr + +-- | Internal helper. +withOrtRunOptionsPtr :: + OrtRunOptions -> + (Ptr OrtRunOptions -> IO a) -> + IO a +withOrtRunOptionsPtr ortRunOptions = + withForeignPtr ortRunOptions.ortRunOptionsForeignPtr + +-- | Internal helper. +withCOrtRunOptionsPtr :: + OrtRunOptions -> + (Ptr COrtRunOptions -> IO a) -> + IO a +withCOrtRunOptionsPtr ortRunOptions action = + withOrtRunOptionsPtr ortRunOptions $ \ortRunOptionsPtr -> do + cOrtRunOptionsPtr <- #{peek HsOrtRunOptions, ortRunOptions} ortRunOptionsPtr + action cOrtRunOptionsPtr + +-- | Internal helper. +wrapCOrtRunOptions :: + OrtApi -> + Ptr COrtRunOptions -> + IO OrtRunOptions +wrapCOrtRunOptions ortApi rawOrtRunOptionsPtr = do + ortRunOptionsPtr <- _wrap_COrtRunOptions ortApi.ortApiConstPtr rawOrtRunOptionsPtr + ortRunOptionsForeignPtr <- newForeignPtr _wrap_OrtApi_ReleaseRunOptions ortRunOptionsPtr + pure $ OrtRunOptions ortRunOptionsForeignPtr + +foreign import capi unsafe + "Onnxruntime/CApi_hsc.h _wrap_COrtRunOptions" + _wrap_COrtRunOptions :: + ConstPtr OrtApi -> + Ptr COrtRunOptions -> + IO (Ptr OrtRunOptions) + +#{def + HsOrtRunOptions* _wrap_COrtRunOptions( + const OrtApi* ortApi, + COrtRunOptions* ortRunOptions + ) { + HsOrtRunOptions *out = malloc(sizeof *out); + out->ortApi = ortApi; + out->ortRunOptions = ortRunOptions; + return out; + } +} + +foreign import capi unsafe + "Onnxruntime/CApi_hsc.h &_wrap_OrtApi_ReleaseRunOptions" + _wrap_OrtApi_ReleaseRunOptions :: + FunPtr ( + Ptr OrtRunOptions -> + IO () + ) + +#{def + void _wrap_OrtApi_ReleaseRunOptions(HsOrtRunOptions* ortRunOptions) { + ortRunOptions->ortApi->ReleaseRunOptions(ortRunOptions->ortRunOptions); + free(ortRunOptions); + } +} + +------------------------------------------------------------------------------- +-- OrtValue + +data + {-# CTYPE "onnxruntime_c_api.h" "COrtValue" #-} + COrtValue + +#{def + typedef OrtValue COrtValue; +} + +newtype + {-# CTYPE "Onnxruntime/CApi_hsc.h" "HsOrtValue" #-} + OrtValue = OrtValue { ortValueForeignPtr :: ForeignPtr OrtValue } + +#{def + typedef struct HsOrtValue { + const OrtApi* ortApi; + COrtValue* ortValue; + } HsOrtValue; +} + +instance HasOrtApi OrtValue where + type CType OrtValue = COrtValue + getOrtApi ortValue = + withOrtValuePtr ortValue $ \ortValuePtr -> + OrtApi <$> #{peek HsOrtValue, ortApi} ortValuePtr + withCTypePtr = withCOrtValuePtr + +-- | Internal helper. +withOrtValuePtr :: + OrtValue -> + (Ptr OrtValue -> IO a) -> + IO a +withOrtValuePtr ortValue = + withForeignPtr ortValue.ortValueForeignPtr + +-- | Internal helper. +withCOrtValuePtr :: + OrtValue -> + (Ptr COrtValue -> IO a) -> + IO a +withCOrtValuePtr ortValue action = + withOrtValuePtr ortValue $ \ortValuePtr -> do + cOrtValuePtr <- #{peek HsOrtValue, ortValue} ortValuePtr + action cOrtValuePtr + +-- | Internal helper. +wrapCOrtValue :: + OrtApi -> + Ptr COrtValue -> + IO OrtValue +wrapCOrtValue ortApi rawOrtValuePtr = do + ortValuePtr <- _wrap_COrtValue ortApi.ortApiConstPtr rawOrtValuePtr + ortValueForeignPtr <- newForeignPtr _wrap_OrtApi_ReleaseValue ortValuePtr + pure $ OrtValue ortValueForeignPtr + +foreign import capi unsafe + "Onnxruntime/CApi_hsc.h _wrap_COrtValue" + _wrap_COrtValue :: + ConstPtr OrtApi -> + Ptr COrtValue -> + IO (Ptr OrtValue) + +#{def + HsOrtValue* _wrap_COrtValue( + const OrtApi* ortApi, + COrtValue* ortValue + ) { + HsOrtValue *out = malloc(sizeof *out); + out->ortApi = ortApi; + out->ortValue = ortValue; + return out; + } +} + +foreign import capi unsafe + "Onnxruntime/CApi_hsc.h &_wrap_OrtApi_ReleaseValue" + _wrap_OrtApi_ReleaseValue :: + FunPtr ( + Ptr OrtValue -> + IO () + ) + +#{def + void _wrap_OrtApi_ReleaseValue(HsOrtValue* ortValue) { + ortValue->ortApi->ReleaseValue(ortValue->ortValue); + free(ortValue); + } +} + +------------------------------------------------------------------------------- +-- ONNX Runtime: API Function +------------------------------------------------------------------------------- + +------------------------------------------------------------------------------- +-- OrtApi::GetErrorCode + +foreign import capi unsafe + "Onnxruntime/CApi_hsc.h _wrap_OrtApi_GetErrorCode" + ortApiGetErrorCode :: + OrtApi -> + Ptr OrtStatus -> + IO OrtErrorCode + +#{def + OrtErrorCode _wrap_OrtApi_GetErrorCode(const OrtApi* ortApi, OrtStatus* ortStatus) { + return ortApi->GetErrorCode(ortStatus); + } +} + +------------------------------------------------------------------------------- +-- OrtApi::GetErrorMessage + +ortApiGetErrorMessageAsString :: + OrtApi -> + Ptr OrtStatus -> + IO String +ortApiGetErrorMessageAsString ortApi ortStatusPtr = do + ConstPtr msgPtr <- _wrap_OrtApi_GetErrorMessage ortApi ortStatusPtr + peekCString msgPtr + +ortApiGetErrorMessage :: + OrtApi -> + Ptr OrtStatus -> + IO ByteString +ortApiGetErrorMessage ortApi ortStatusPtr = do + ConstPtr msgPtr <- _wrap_OrtApi_GetErrorMessage ortApi ortStatusPtr + print (msgPtr == nullPtr) + BS.packCString msgPtr + +foreign import capi unsafe + "Onnxruntime/CApi_hsc.h _wrap_OrtApi_GetErrorMessage" + _wrap_OrtApi_GetErrorMessage :: + OrtApi -> + Ptr OrtStatus -> + IO (ConstPtr CChar) + +#{def + const char* _wrap_OrtApi_GetErrorMessage(const OrtApi* ortApi, OrtStatus* ortStatus) { + return ortApi->GetErrorMessage(ortStatus); + } +} + +------------------------------------------------------------------------------- +-- OrtApi::CreateEnv + +{- | +> ORT_API2_STATUS(CreateEnv, +> OrtLoggingLevel log_severity_level, +> _In_ const char* logid, +> _Outptr_ OrtEnv** out +> ); +-} +ortApiCreateEnv :: + OrtApi -> + OrtLoggingLevel -> + String -> + IO OrtEnv +ortApiCreateEnv ortApi logSeverityLevel logid = do + withCString logid $ \logidPtr -> do + alloca $ \outPtr -> do + ortStatusPtr <- + _wrap_OrtApi_CreateEnv + ortApi.ortApiConstPtr + logSeverityLevel + (ConstPtr logidPtr) -- NOTE: This is unsafe. + outPtr + handleOrtStatus ortApi ortStatusPtr $ do + wrapCOrtEnv ortApi + =<< peek outPtr + +foreign import capi unsafe + "Onnxruntime/CApi_hsc.h _wrap_OrtApi_CreateEnv" + _wrap_OrtApi_CreateEnv :: + ConstPtr OrtApi -> + OrtLoggingLevel -> + ConstPtr CChar -> + Ptr (Ptr COrtEnv) -> + IO (Ptr OrtStatus) + +#{def + OrtStatus* _wrap_OrtApi_CreateEnv( + const OrtApi* ortApi, + OrtLoggingLevel logSeverityLevel, + const char* logid, + COrtEnv** out + ) { + return ortApi->CreateEnv(logSeverityLevel, logid, out); + } +} + +------------------------------------------------------------------------------- +-- OrtApi::CreateEnvWithCustomLogger + +{- +> ORT_API2_STATUS(CreateEnvWithCustomLogger, +> _In_ OrtLoggingFunction logging_function, +> _In_opt_ void* logger_param, +> _In_ OrtLoggingLevel log_severity_level, +> _In_ const char* logid, +> _Outptr_ OrtEnv** out +> ); +-} + +-- TODO: unimplemented + +------------------------------------------------------------------------------- +-- OrtApi::EnableTelemetryEvents + +{- +> ORT_API2_STATUS(EnableTelemetryEvents, +> _In_ const OrtEnv* env +>); +-} + +-- TODO: unimplemented + +------------------------------------------------------------------------------- +-- OrtApi::DisableTelemetryEvents + +{- +> ORT_API2_STATUS(DisableTelemetryEvents, +> _In_ const OrtEnv* env +>); +-} + +-- TODO: unimplemented + +------------------------------------------------------------------------------- +-- OrtApi::CreateSession + +{- +> ORT_API2_STATUS(CreateSession, +> _In_ const OrtEnv* env, +> _In_ const ORTCHAR_T* model_path, +> _In_ const OrtSessionOptions* options, +> _Outptr_ OrtSession** out +> ); +-} +ortApiCreateSession :: + OrtEnv -> + FilePath -> + OrtSessionOptions -> + IO OrtSession +ortApiCreateSession ortEnv modelPath options = do + ortApi <- getOrtApi ortEnv + alloca $ \outPtr -> do + withCTypePtr ortEnv $ \cOrtEnvPtr -> do + withCString modelPath $ \modelPathPtr -> do + withCTypePtr options $ \cOrtSessionOptionsPtr -> do + ortStatusPtr <- + _wrap_OrtApi_CreateSession + ortApi + cOrtEnvPtr + (ConstPtr modelPathPtr) -- NOTE: This is unsafe. + cOrtSessionOptionsPtr + outPtr + handleOrtStatus ortApi ortStatusPtr $ do + wrapCOrtSession ortApi + =<< peek outPtr + +foreign import capi unsafe + "Onnxruntime/CApi_hsc.h _wrap_OrtApi_CreateSession" + _wrap_OrtApi_CreateSession :: + OrtApi -> + Ptr COrtEnv -> + ConstPtr CChar -> + Ptr COrtSessionOptions -> + Ptr (Ptr COrtSession) -> + IO (Ptr OrtStatus) + +#{def + OrtStatus* _wrap_OrtApi_CreateSession( + OrtApi* ortApi, + COrtEnv* ortEnv, + const ORTCHAR_T* modelPath, + COrtSessionOptions* options, + COrtSession** out + ) { + return ortApi->CreateSession( + ortEnv, + modelPath, + options, + out + ); + } +} + +------------------------------------------------------------------------------- +-- OrtApi::CreateSessionFromArray + +{- +> ORT_API2_STATUS(CreateSessionFromArray, +> _In_ const OrtEnv* env, +> _In_ const void* model_data, +> size_t model_data_length, +> _In_ const OrtSessionOptions* options, +> _Outptr_ OrtSession** out +> ); +-} + +-- TODO: unimplemented + +------------------------------------------------------------------------------- +-- OrtApi::Run + +{- +> ORT_API2_STATUS(Run, +> _Inout_ OrtSession* session, +> _In_opt_ const OrtRunOptions* run_options, +> _In_reads_(input_len) const char* const* input_names, +> _In_reads_(input_len) const OrtValue* const* inputs, +> size_t input_len, +> _In_reads_(output_names_len) const char* const* output_names, +> size_t output_names_len, +> _Inout_updates_all_(output_names_len) OrtValue** outputs +> ); +-} +ortApiRun :: + OrtSession -> + OrtRunOptions -> + [String] -> + [OrtValue] -> + [String] -> + IO [OrtValue] +ortApiRun ortSession ortRunOptions inputNames inputs outputNames = do + ortApi <- getOrtApi ortSession + alloca @(Ptr COrtValue) $ \outputsPtr -> + withCTypePtr ortSession $ \cOrtSessionPtr -> + withCTypePtr ortRunOptions $ \cOrtRunOptionsPtr -> + withCStringArrayLen inputNames $ \inputLen cInputNames -> + withCTypeArrayLen inputs $ \inputLen' cInputs -> + withCStringArrayLen outputNames $ \outputLen cOutputNames -> + -- TODO turn into a proper exception + assert (inputLen == inputLen') $ do + ortStatusPtr <- + _wrap_OrtApi_Run + ortApi + cOrtSessionPtr + cOrtRunOptionsPtr + (coerce cInputNames) + (coerce cInputs) + (fromIntegral inputLen) + (coerce cOutputNames) + (fromIntegral outputLen) + outputsPtr + + handleOrtStatus ortApi ortStatusPtr $ do + traverse (wrapCOrtValue ortApi) + =<< peekArray outputLen outputsPtr + +foreign import capi unsafe + "Onnxruntime/CApi_hsc.h _wrap_OrtApi_Run" + _wrap_OrtApi_Run :: + OrtApi -> + Ptr COrtSession -> + Ptr COrtRunOptions -> + ConstPtr (ConstPtr CChar) -> + ConstPtr (ConstPtr COrtValue) -> + ( #{type size_t} ) -> + ConstPtr (ConstPtr CChar) -> + ( #{type size_t} ) -> + Ptr (Ptr COrtValue) -> + IO (Ptr OrtStatus) + +#{def + OrtStatus* _wrap_OrtApi_Run( + OrtApi* ortApi, + COrtSession* session, + COrtRunOptions* run_options, + const char* const* input_names, + const COrtValue* const* inputs, + size_t input_len, + const char* const* output_names, + size_t output_names_len, + COrtValue** outputs + ) { + return ortApi->Run( + session, + run_options, + input_names, + inputs, + input_len, + output_names, + output_names_len, + outputs + ); + } +} +------------------------------------------------------------------------------- +-- OrtApi::CreateSessionOptions + +{- | +> ORT_API2_STATUS(CreateSessionOptions, +> _Outptr_ OrtSessionOptions** options +> ); +-} +ortApiCreateSessionOptions :: + OrtApi -> + IO OrtSessionOptions +ortApiCreateSessionOptions ortApi = do + alloca $ \outPtr -> do + ortStatusPtr <- + _wrap_OrtApi_CreateSessionOptions + ortApi.ortApiConstPtr + outPtr + handleOrtStatus ortApi ortStatusPtr $ do + cOrtSessionOptionsPtr <- peek outPtr + wrapCOrtSessionOptions ortApi cOrtSessionOptionsPtr + +foreign import capi unsafe + "Onnxruntime/CApi_hsc.h _wrap_OrtApi_CreateSessionOptions" + _wrap_OrtApi_CreateSessionOptions :: + ConstPtr OrtApi -> + Ptr (Ptr COrtSessionOptions) -> + IO (Ptr OrtStatus) + +#{def + OrtStatus* _wrap_OrtApi_CreateSessionOptions( + const OrtApi* ortApi, + COrtSessionOptions** out + ) { + return ortApi->CreateSessionOptions(out); + } +} + +------------------------------------------------------------------------------- +-- OrtApi::CloneSessionOptions + +{- | +> ORT_API2_STATUS(CloneSessionOptions, +> _In_ const OrtSessionOptions* in_options, +> _Outptr_ OrtSessionOptions** out_options + ); +-} +ortApiCloneSessionOptions :: + OrtSessionOptions -> + IO OrtSessionOptions +ortApiCloneSessionOptions inOptions = do + ortApi <- getOrtApi inOptions + withOrtSessionOptionsPtr inOptions $ \inOptionsPtr -> do + alloca $ \outPtr -> do + ortStatusPtr <- + _wrap_OrtApi_CloneSessionOptions + inOptionsPtr + outPtr + handleOrtStatus ortApi ortStatusPtr $ + wrapCOrtSessionOptions ortApi + =<< peek outPtr + +foreign import capi unsafe + "Onnxruntime/CApi_hsc.h _wrap_OrtApi_CloneSessionOptions" + _wrap_OrtApi_CloneSessionOptions :: + Ptr OrtSessionOptions -> + Ptr (Ptr COrtSessionOptions) -> + IO (Ptr OrtStatus) + +#{def + OrtStatus* _wrap_OrtApi_CloneSessionOptions( + HsOrtSessionOptions* inOptions, + COrtSessionOptions** out + ) { + return inOptions->ortApi->CloneSessionOptions( + inOptions->ortSessionOptions, + out + ); + } +} + +------------------------------------------------------------------------------- +-- OrtApi::SetOptimizedModelFilePath + +{- | +> ORT_API2_STATUS(SetOptimizedModelFilePath, +> _Inout_ OrtSessionOptions* options, +> _In_ const ORTCHAR_T* optimized_model_filepath +> ); +-} +ortApiSetOptimizedModelFilePath :: + OrtSessionOptions -> + FilePath -> + IO () +ortApiSetOptimizedModelFilePath options optimizedModelFilepath = do + ortApi <- getOrtApi options + withCString optimizedModelFilepath $ \optimizedModelFilepathPtr -> + withOrtSessionOptionsPtr options $ \optionsPtr -> do + ortStatusPtr <- + _wrap_OrtApi_SetOptimizedModelFilePath + optionsPtr + (ConstPtr optimizedModelFilepathPtr) -- NOTE: This is unsafe. + handleOrtStatus ortApi ortStatusPtr $ do + pure () + +foreign import capi unsafe + "Onnxruntime/CApi_hsc.h _wrap_OrtApi_SetOptimizedModelFilePath" + _wrap_OrtApi_SetOptimizedModelFilePath :: + Ptr OrtSessionOptions -> + ConstPtr CChar -> + IO (Ptr OrtStatus) + +#{def + OrtStatus* _wrap_OrtApi_SetOptimizedModelFilePath( + HsOrtSessionOptions* options, + const ORTCHAR_T* optimizedModelFilepath + ) { + return options->ortApi->SetOptimizedModelFilePath( + options->ortSessionOptions, + optimizedModelFilepath + ); + } +} + +------------------------------------------------------------------------------- +-- OrtApi::SetSessionExecutionMode + +{- +> ORT_API2_STATUS(SetSessionExecutionMode, +> _Inout_ OrtSessionOptions* options, +> ExecutionMode execution_mode +> ); +-} +ortApiSetSessionExecutionMode :: + OrtSessionOptions -> + ExecutionMode -> + IO () +ortApiSetSessionExecutionMode options executionMode = do + ortApi <- getOrtApi options + withOrtSessionOptionsPtr options $ \optionsPtr -> do + ortStatusPtr <- + _wrap_OrtApi_SetSessionExecutionMode + optionsPtr + executionMode + handleOrtStatus ortApi ortStatusPtr $ do + pure () + +foreign import capi unsafe + "Onnxruntime/CApi_hsc.h _wrap_OrtApi_SetSessionExecutionMode" + _wrap_OrtApi_SetSessionExecutionMode :: + Ptr OrtSessionOptions -> + ExecutionMode -> + IO (Ptr OrtStatus) + +#{def + OrtStatus* _wrap_OrtApi_SetSessionExecutionMode( + HsOrtSessionOptions* options, + int executionMode + ) { + return options->ortApi->SetSessionExecutionMode( + options->ortSessionOptions, + executionMode + ); + } +} + +------------------------------------------------------------------------------- +-- OrtApi::EnableProfiling + +{- | +> ORT_API2_STATUS(EnableProfiling, +> _Inout_ OrtSessionOptions* options, +> _In_ const ORTCHAR_T* profile_file_prefix +> ); +-} +ortApiEnableProfiling :: + OrtSessionOptions -> + FilePath -> + IO () +ortApiEnableProfiling options profileFilePrefix = do + ortApi <- getOrtApi options + withCString profileFilePrefix $ \profileFilePrefixPtr -> + withOrtSessionOptionsPtr options $ \optionsPtr -> do + ortStatusPtr <- + _wrap_OrtApi_EnableProfiling + optionsPtr + (ConstPtr profileFilePrefixPtr) -- NOTE: This is unsafe. + handleOrtStatus ortApi ortStatusPtr $ do + pure () + +foreign import capi unsafe + "Onnxruntime/CApi_hsc.h _wrap_OrtApi_EnableProfiling" + _wrap_OrtApi_EnableProfiling :: + Ptr OrtSessionOptions -> + ConstPtr CChar -> + IO (Ptr OrtStatus) + +#{def + OrtStatus* _wrap_OrtApi_EnableProfiling( + HsOrtSessionOptions* options, + const ORTCHAR_T* profileFilePrefix + ) { + return options->ortApi->EnableProfiling( + options->ortSessionOptions, + profileFilePrefix + ); + } +} + +------------------------------------------------------------------------------- +-- OrtApi::DisableProfiling + +{- | +> ORT_API2_STATUS(DisableProfiling, +> _Inout_ OrtSessionOptions* options +> ); +-} +ortApiDisableProfiling :: + OrtSessionOptions -> + IO () +ortApiDisableProfiling options = do + ortApi <- getOrtApi options + withOrtSessionOptionsPtr options $ \optionsPtr -> do + ortStatusPtr <- + _wrap_OrtApi_DisableProfiling + optionsPtr + handleOrtStatus ortApi ortStatusPtr $ do + pure () + +foreign import capi unsafe + "Onnxruntime/CApi_hsc.h _wrap_OrtApi_DisableProfiling" + _wrap_OrtApi_DisableProfiling :: + Ptr OrtSessionOptions -> + IO (Ptr OrtStatus) + +#{def + OrtStatus* _wrap_OrtApi_DisableProfiling( + HsOrtSessionOptions* options + ) { + return options->ortApi->DisableProfiling(options->ortSessionOptions); + } +} + +------------------------------------------------------------------------------- +-- OrtApi::EnableMemPattern + +{- | +> ORT_API2_STATUS(EnableMemPattern, +> _Inout_ OrtSessionOptions* options +> ); +-} +ortApiEnableMemPattern :: + OrtSessionOptions -> + IO () +ortApiEnableMemPattern options = do + ortApi <- getOrtApi options + withOrtSessionOptionsPtr options $ \optionsPtr -> do + ortStatusPtr <- + _wrap_OrtApi_EnableMemPattern + optionsPtr + handleOrtStatus ortApi ortStatusPtr $ do + pure () + +foreign import capi unsafe + "Onnxruntime/CApi_hsc.h _wrap_OrtApi_EnableMemPattern" + _wrap_OrtApi_EnableMemPattern :: + Ptr OrtSessionOptions -> + IO (Ptr OrtStatus) + +#{def + OrtStatus* _wrap_OrtApi_EnableMemPattern( + HsOrtSessionOptions* options + ) { + return options->ortApi->EnableMemPattern(options->ortSessionOptions); + } +} + +------------------------------------------------------------------------------- +-- OrtApi::DisableMemPattern + +{- | +> ORT_API2_STATUS(DisableMemPattern, +> _Inout_ OrtSessionOptions* options +> ); +-} +ortApiDisableMemPattern :: + OrtSessionOptions -> + IO () +ortApiDisableMemPattern options = do + ortApi <- getOrtApi options + withOrtSessionOptionsPtr options $ \optionsPtr -> do + ortStatusPtr <- + _wrap_OrtApi_DisableMemPattern + optionsPtr + handleOrtStatus ortApi ortStatusPtr $ do + pure () + +foreign import capi unsafe + "Onnxruntime/CApi_hsc.h _wrap_OrtApi_DisableMemPattern" + _wrap_OrtApi_DisableMemPattern :: + Ptr OrtSessionOptions -> + IO (Ptr OrtStatus) + +#{def + OrtStatus* _wrap_OrtApi_DisableMemPattern( + HsOrtSessionOptions* options + ) { + return options->ortApi->DisableMemPattern(options->ortSessionOptions); + } +} + +------------------------------------------------------------------------------- +-- OrtApi::EnableCpuMemArena + +{- | +> ORT_API2_STATUS(EnableCpuMemArena, +> _Inout_ OrtSessionOptions* options +> ); +-} +ortApiEnableCpuMemArena :: + OrtSessionOptions -> + IO () +ortApiEnableCpuMemArena options = do + ortApi <- getOrtApi options + withOrtSessionOptionsPtr options $ \optionsPtr -> do + ortStatusPtr <- + _wrap_OrtApi_EnableCpuMemArena + optionsPtr + handleOrtStatus ortApi ortStatusPtr $ do + pure () + +foreign import capi unsafe + "Onnxruntime/CApi_hsc.h _wrap_OrtApi_EnableCpuMemArena" + _wrap_OrtApi_EnableCpuMemArena :: + Ptr OrtSessionOptions -> + IO (Ptr OrtStatus) + +#{def + OrtStatus* _wrap_OrtApi_EnableCpuMemArena( + HsOrtSessionOptions* options + ) { + return options->ortApi->EnableCpuMemArena(options->ortSessionOptions); + } +} + +------------------------------------------------------------------------------- +-- OrtApi::DisableCpuMemArena + +{- | +> ORT_API2_STATUS(DisableCpuMemArena, +> _Inout_ OrtSessionOptions* options +> ); +-} +ortApiDisableCpuMemArena :: + OrtSessionOptions -> + IO () +ortApiDisableCpuMemArena options = do + ortApi <- getOrtApi options + withOrtSessionOptionsPtr options $ \optionsPtr -> do + ortStatusPtr <- + _wrap_OrtApi_DisableCpuMemArena + optionsPtr + handleOrtStatus ortApi ortStatusPtr $ do + pure () + +foreign import capi unsafe + "Onnxruntime/CApi_hsc.h _wrap_OrtApi_DisableCpuMemArena" + _wrap_OrtApi_DisableCpuMemArena :: + Ptr OrtSessionOptions -> + IO (Ptr OrtStatus) + +#{def + OrtStatus* _wrap_OrtApi_DisableCpuMemArena( + HsOrtSessionOptions* options + ) { + return options->ortApi->DisableCpuMemArena(options->ortSessionOptions); + } +} + +------------------------------------------------------------------------------- +-- OrtApi::SetSessionLogId + +{- | +> ORT_API2_STATUS(SetSessionLogId, +> _Inout_ OrtSessionOptions* options, +> const char* logid +> ); +-} +ortApiSetSessionLogId :: + OrtSessionOptions -> + String -> + IO () +ortApiSetSessionLogId options logid = do + ortApi <- getOrtApi options + withCString logid $ \logidPtr -> + withOrtSessionOptionsPtr options $ \optionsPtr -> do + ortStatusPtr <- + _wrap_OrtApi_SetSessionLogId + optionsPtr + (ConstPtr logidPtr) -- NOTE: This is unsafe. + handleOrtStatus ortApi ortStatusPtr $ do + pure () + +foreign import capi unsafe + "Onnxruntime/CApi_hsc.h _wrap_OrtApi_SetSessionLogId" + _wrap_OrtApi_SetSessionLogId :: + Ptr OrtSessionOptions -> + ConstPtr CChar -> + IO (Ptr OrtStatus) + +#{def + OrtStatus* _wrap_OrtApi_SetSessionLogId( + HsOrtSessionOptions* options, + const char* logid + ) { + return options->ortApi->SetSessionLogId( + options->ortSessionOptions, + logid + ); + } +} + +------------------------------------------------------------------------------- +-- OrtApi::SetSessionLogVerbosityLevel + +{- | +> ORT_API2_STATUS(SetSessionLogVerbosityLevel, +> _Inout_ OrtSessionOptions* options, +> int session_log_verbosity_level +> ); +-} +ortApiSetSessionLogVerbosityLevel :: + OrtSessionOptions -> + Int -> + IO () +ortApiSetSessionLogVerbosityLevel options sessionLogVerbosityLevel = do + ortApi <- getOrtApi options + withOrtSessionOptionsPtr options $ \optionsPtr -> do + ortStatusPtr <- + _wrap_OrtApi_SetSessionLogVerbosityLevel + optionsPtr + (fromIntegral sessionLogVerbosityLevel) + handleOrtStatus ortApi ortStatusPtr $ do + pure () + +foreign import capi unsafe + "Onnxruntime/CApi_hsc.h _wrap_OrtApi_SetSessionLogVerbosityLevel" + _wrap_OrtApi_SetSessionLogVerbosityLevel :: + Ptr OrtSessionOptions -> + CInt -> + IO (Ptr OrtStatus) + +#{def + OrtStatus* _wrap_OrtApi_SetSessionLogVerbosityLevel( + HsOrtSessionOptions* options, + int sessionLogVerbosityLevel + ) { + return options->ortApi->SetSessionLogVerbosityLevel( + options->ortSessionOptions, + sessionLogVerbosityLevel + ); + } +} + +------------------------------------------------------------------------------- +-- OrtApi::SetSessionLogSeverityLevel + +{- +> ORT_API2_STATUS(SetSessionLogSeverityLevel, +> _Inout_ OrtSessionOptions* options, +> int session_log_severity_level +> ); +-} +ortApiSetSessionLogSeverityLevel :: + OrtSessionOptions -> + OrtLoggingLevel -> + IO () +ortApiSetSessionLogSeverityLevel options sessionLogSeverityLevel = do + ortApi <- getOrtApi options + withOrtSessionOptionsPtr options $ \optionsPtr -> do + ortStatusPtr <- + _wrap_OrtApi_SetSessionLogSeverityLevel + optionsPtr + sessionLogSeverityLevel + handleOrtStatus ortApi ortStatusPtr $ do + pure () + +foreign import capi unsafe + "Onnxruntime/CApi_hsc.h _wrap_OrtApi_SetSessionLogSeverityLevel" + _wrap_OrtApi_SetSessionLogSeverityLevel :: + Ptr OrtSessionOptions -> + OrtLoggingLevel -> + IO (Ptr OrtStatus) + +#{def + OrtStatus* _wrap_OrtApi_SetSessionLogSeverityLevel( + HsOrtSessionOptions* options, + int sessionLogSeverityLevel + ) { + return options->ortApi->SetSessionLogSeverityLevel( + options->ortSessionOptions, + sessionLogSeverityLevel + ); + } +} + +------------------------------------------------------------------------------- +-- OrtApi::SetSessionGraphOptimizationLevel + +{- +> ORT_API2_STATUS(SetSessionGraphOptimizationLevel, +> _Inout_ OrtSessionOptions* options, +> GraphOptimizationLevel graph_optimization_level +> ); +-} +ortApiSetSessionGraphOptimizationLevel :: + OrtSessionOptions -> + GraphOptimizationLevel -> + IO () +ortApiSetSessionGraphOptimizationLevel options graphOptimizationlevel = do + ortApi <- getOrtApi options + withOrtSessionOptionsPtr options $ \optionsPtr -> do + ortStatusPtr <- + _wrap_OrtApi_SetSessionGraphOptimizationLevel + optionsPtr + graphOptimizationlevel + handleOrtStatus ortApi ortStatusPtr $ do + pure () + +foreign import capi unsafe + "Onnxruntime/CApi_hsc.h _wrap_OrtApi_SetSessionGraphOptimizationLevel" + _wrap_OrtApi_SetSessionGraphOptimizationLevel :: + Ptr OrtSessionOptions -> + GraphOptimizationLevel -> + IO (Ptr OrtStatus) + +#{def + OrtStatus* _wrap_OrtApi_SetSessionGraphOptimizationLevel( + HsOrtSessionOptions* options, + int graphOptimizationlevel + ) { + return options->ortApi->SetSessionGraphOptimizationLevel( + options->ortSessionOptions, + graphOptimizationlevel + ); + } +} + +------------------------------------------------------------------------------- +-- OrtApi::SetIntraOpNumThreads + +{- | +> ORT_API2_STATUS(SetIntraOpNumThreads, +> _Inout_ OrtSessionOptions* options, +> int intra_op_num_threads +> ); +-} +ortApiSetIntraOpNumThreads :: + OrtSessionOptions -> + Int -> + IO () +ortApiSetIntraOpNumThreads options intraOpNumThreads = do + ortApi <- getOrtApi options + withOrtSessionOptionsPtr options $ \optionsPtr -> do + ortStatusPtr <- + _wrap_OrtApi_SetIntraOpNumThreads + optionsPtr + (fromIntegral intraOpNumThreads) + handleOrtStatus ortApi ortStatusPtr $ do + pure () + +foreign import capi unsafe + "Onnxruntime/CApi_hsc.h _wrap_OrtApi_SetIntraOpNumThreads" + _wrap_OrtApi_SetIntraOpNumThreads :: + Ptr OrtSessionOptions -> + CInt -> + IO (Ptr OrtStatus) + +#{def + OrtStatus* _wrap_OrtApi_SetIntraOpNumThreads( + HsOrtSessionOptions* options, + int intraOpNumThreads + ) { + return options->ortApi->SetIntraOpNumThreads( + options->ortSessionOptions, + intraOpNumThreads + ); + } +} + +------------------------------------------------------------------------------- +-- OrtApi::SetInterOpNumThreads + +{- +> ORT_API2_STATUS(SetInterOpNumThreads, +> _Inout_ OrtSessionOptions* options, +> int inter_op_num_threads +> ); +-} +ortApiSetInterOpNumThreads :: + OrtSessionOptions -> + Int -> + IO () +ortApiSetInterOpNumThreads options interOpNumThreads = do + ortApi <- getOrtApi options + withOrtSessionOptionsPtr options $ \optionsPtr -> do + ortStatusPtr <- + _wrap_OrtApi_SetInterOpNumThreads + optionsPtr + (fromIntegral interOpNumThreads) + handleOrtStatus ortApi ortStatusPtr $ do + pure () + +foreign import capi unsafe + "Onnxruntime/CApi_hsc.h _wrap_OrtApi_SetInterOpNumThreads" + _wrap_OrtApi_SetInterOpNumThreads :: + Ptr OrtSessionOptions -> + CInt -> + IO (Ptr OrtStatus) + +#{def + OrtStatus* _wrap_OrtApi_SetInterOpNumThreads( + HsOrtSessionOptions* options, + int interOpNumThreads + ) { + return options->ortApi->SetInterOpNumThreads( + options->ortSessionOptions, + interOpNumThreads + ); + } +} + +------------------------------------------------------------------------------- +-- OrtApi::CreateCustomOpDomain + +-- ORT_API2_STATUS(CreateCustomOpDomain, _In_ const char* domain, _Outptr_ OrtCustomOpDomain** out); + +-- TODO: unimplemented + +------------------------------------------------------------------------------- +-- OrtApi::CustomOpDomain_Add + +-- ORT_API2_STATUS(CustomOpDomain_Add, _Inout_ OrtCustomOpDomain* custom_op_domain, _In_ const OrtCustomOp* op); + +-- TODO: unimplemented + +------------------------------------------------------------------------------- +-- OrtApi::AddCustomOpDomain + +-- ORT_API2_STATUS(AddCustomOpDomain, _Inout_ OrtSessionOptions* options, _In_ OrtCustomOpDomain* custom_op_domain); + +-- TODO: unimplemented + +------------------------------------------------------------------------------- +-- OrtApi::RegisterCustomOpsLibrary + +-- ORT_API2_STATUS(RegisterCustomOpsLibrary, _Inout_ OrtSessionOptions* options, _In_ const char* library_path, _Outptr_ void** library_handle); + +-- TODO: unimplemented + +------------------------------------------------------------------------------- +-- OrtApi::SessionGetInputCount + +{- | +> ORT_API2_STATUS(SessionGetInputCount, +> _In_ const OrtSession* session, +> _Out_ size_t* out +> ); +-} +ortApiSessionGetInputCount :: + OrtSession -> + IO Word64 +ortApiSessionGetInputCount ortSession = do + ortApi <- getOrtApi ortSession + withOrtSessionPtr ortSession $ \ortSessionPtr -> + alloca $ \outPtr -> do + ortStatusPtr <- + _wrap_OrtApi_SessionGetInputCount + (ConstPtr ortSessionPtr) + outPtr + handleOrtStatus ortApi ortStatusPtr $ do + CSize inputCount <- peek outPtr + pure inputCount + +foreign import capi unsafe + "Onnxruntime/CApi_hsc.h _wrap_OrtApi_SessionGetInputCount" + _wrap_OrtApi_SessionGetInputCount :: + ConstPtr OrtSession -> + Ptr CSize -> + IO (Ptr OrtStatus) + +#{def + OrtStatus* _wrap_OrtApi_SessionGetInputCount( + const HsOrtSession* ortSession, + size_t* out + ) { + return ortSession->ortApi->SessionGetInputCount( + ortSession->ortSession, + out + ); + } +} + +------------------------------------------------------------------------------- +-- OrtApi::SessionGetOutputCount + +{- | +> ORT_API2_STATUS(SessionGetOutputCount, +> _In_ const OrtSession* session, +> _Out_ size_t* out +> ); +-} +ortApiSessionGetOutputCount :: + OrtSession -> + IO Word64 +ortApiSessionGetOutputCount ortSession = do + ortApi <- getOrtApi ortSession + withOrtSessionPtr ortSession $ \ortSessionPtr -> + alloca $ \outPtr -> do + ortStatusPtr <- + _wrap_OrtApi_SessionGetOutputCount + (ConstPtr ortSessionPtr) + outPtr + handleOrtStatus ortApi ortStatusPtr $ do + CSize inputCount <- peek outPtr + pure inputCount + +foreign import capi unsafe + "Onnxruntime/CApi_hsc.h _wrap_OrtApi_SessionGetOutputCount" + _wrap_OrtApi_SessionGetOutputCount :: + ConstPtr OrtSession -> + Ptr CSize -> + IO (Ptr OrtStatus) + +#{def + OrtStatus* _wrap_OrtApi_SessionGetOutputCount( + const HsOrtSession* ortSession, + size_t* out + ) { + return ortSession->ortApi->SessionGetOutputCount( + ortSession->ortSession, + out + ); + } +} + +------------------------------------------------------------------------------- +-- OrtApi::SessionGetOverridableInitializerCount + +-- ORT_API2_STATUS(SessionGetOverridableInitializerCount, _In_ const OrtSession* session, _Out_ size_t* out); + +-- TODO: unimplemented + +------------------------------------------------------------------------------- +-- OrtApi::SessionGetInputTypeInfo + +{- | +> ORT_API2_STATUS(SessionGetInputTypeInfo, +> _In_ const OrtSession* session, +> size_t index, +> _Outptr_ OrtTypeInfo** type_info +> ); +-} +ortApiSessionGetInputTypeInfo :: + OrtSession -> + Word64 -> + IO OrtTypeInfo +ortApiSessionGetInputTypeInfo ortSession index = do + ortApi <- getOrtApi ortSession + withOrtSessionPtr ortSession $ \ortSessionPtr -> + alloca $ \outPtr -> do + ortStatusPtr <- + _wrap_OrtApi_SessionGetInputTypeInfo + (ConstPtr ortSessionPtr) + (CSize index) + outPtr + handleOrtStatus ortApi ortStatusPtr $ + wrapCOrtTypeInfo ortApi + =<< peek outPtr + +foreign import capi unsafe + "Onnxruntime/CApi_hsc.h _wrap_OrtApi_SessionGetInputTypeInfo" + _wrap_OrtApi_SessionGetInputTypeInfo :: + ConstPtr OrtSession -> + CSize -> + Ptr (Ptr COrtTypeInfo) -> + IO (Ptr OrtStatus) + +#{def + OrtStatus* _wrap_OrtApi_SessionGetInputTypeInfo( + const HsOrtSession* ortSession, + size_t index, + COrtTypeInfo** out + ) { + return ortSession->ortApi->SessionGetInputTypeInfo( + ortSession->ortSession, + index, + out + ); + } +} + +------------------------------------------------------------------------------- +-- OrtApi::SessionGetOutputTypeInfo + +{- | +> ORT_API2_STATUS(SessionGetOutputTypeInfo, +> _In_ const OrtSession* session, +> size_t index, +> _Outptr_ OrtTypeInfo** type_info +> ); +-} +ortApiSessionGetOutputTypeInfo :: + OrtSession -> + Word64 -> + IO OrtTypeInfo +ortApiSessionGetOutputTypeInfo ortSession index = do + ortApi <- getOrtApi ortSession + withOrtSessionPtr ortSession $ \ortSessionPtr -> + alloca $ \outPtr -> do + ortStatusPtr <- + _wrap_OrtApi_SessionGetOutputTypeInfo + (ConstPtr ortSessionPtr) + (CSize index) + outPtr + handleOrtStatus ortApi ortStatusPtr $ + wrapCOrtTypeInfo ortApi + =<< peek outPtr + +foreign import capi unsafe + "Onnxruntime/CApi_hsc.h _wrap_OrtApi_SessionGetOutputTypeInfo" + _wrap_OrtApi_SessionGetOutputTypeInfo :: + ConstPtr OrtSession -> + CSize -> + Ptr (Ptr COrtTypeInfo) -> + IO (Ptr OrtStatus) + +#{def + OrtStatus* _wrap_OrtApi_SessionGetOutputTypeInfo( + const HsOrtSession* ortSession, + size_t index, + COrtTypeInfo** out + ) { + return ortSession->ortApi->SessionGetOutputTypeInfo( + ortSession->ortSession, + index, + out + ); + } +} + +------------------------------------------------------------------------------- +-- OrtApi::SessionGetOverridableInitializerTypeInfo + +-- ORT_API2_STATUS(SessionGetOverridableInitializerTypeInfo, _In_ const OrtSession* session, size_t index, _Outptr_ OrtTypeInfo** type_info); + +-- TODO: unimplemented + +------------------------------------------------------------------------------- +-- OrtApi::SessionGetInputName + +-- ORT_API2_STATUS(SessionGetInputName, _In_ const OrtSession* session, size_t index, _Inout_ OrtAllocator* allocator, _Outptr_ char** value); + +-- TODO: unimplemented + +------------------------------------------------------------------------------- +-- OrtApi::SessionGetOutputName + +-- ORT_API2_STATUS(SessionGetOutputName, _In_ const OrtSession* session, size_t index, _Inout_ OrtAllocator* allocator, _Outptr_ char** value); + +-- TODO: unimplemented + +------------------------------------------------------------------------------- +-- OrtApi::SessionGetOverridableInitializerName + +-- ORT_API2_STATUS(SessionGetOverridableInitializerName, _In_ const OrtSession* session, size_t index, _Inout_ OrtAllocator* allocator, _Outptr_ char** value); + +-- TODO: unimplemented + +------------------------------------------------------------------------------- +-- OrtApi::CreateRunOptions + +{- | +> ORT_API2_STATUS(CreateRunOptions, +> _Outptr_ OrtRunOptions** options +> ); +-} +ortApiCreateRunOptions :: + OrtApi -> + IO OrtRunOptions +ortApiCreateRunOptions ortApi = do + alloca $ \outPtr -> do + ortStatusPtr <- + _wrap_OrtApi_CreateRunOptions + ortApi.ortApiConstPtr + outPtr + handleOrtStatus ortApi ortStatusPtr $ do + cOrtRunOptionsPtr <- peek outPtr + wrapCOrtRunOptions ortApi cOrtRunOptionsPtr + +foreign import capi unsafe + "Onnxruntime/CApi_hsc.h _wrap_OrtApi_CreateRunOptions" + _wrap_OrtApi_CreateRunOptions :: + ConstPtr OrtApi -> + Ptr (Ptr COrtRunOptions) -> + IO (Ptr OrtStatus) + +#{def + OrtStatus* _wrap_OrtApi_CreateRunOptions( + const OrtApi* ortApi, + COrtRunOptions** out + ) { + return ortApi->CreateRunOptions(out); + } +} + +------------------------------------------------------------------------------- +-- OrtApi::RunOptionsSetRunLogVerbosityLevel + +{- | +> ORT_API2_STATUS(RunOptionsSetRunLogVerbosityLevel, +> _Inout_ OrtRunOptions* options, +> int log_verbosity_level +> ); +-} +ortApiRunOptionsSetRunLogVerbosityLevel :: + OrtRunOptions -> + Int -> + IO () +ortApiRunOptionsSetRunLogVerbosityLevel options logVerbosityLevel = do + ortApi <- getOrtApi options + withOrtRunOptionsPtr options $ \optionsPtr -> do + ortStatusPtr <- + _wrap_OrtApi_RunOptionsSetRunLogVerbosityLevel + optionsPtr + (fromIntegral logVerbosityLevel) + handleOrtStatus ortApi ortStatusPtr $ do + pure () + +foreign import capi unsafe + "Onnxruntime/CApi_hsc.h _wrap_OrtApi_RunOptionsSetRunLogVerbosityLevel" + _wrap_OrtApi_RunOptionsSetRunLogVerbosityLevel :: + Ptr OrtRunOptions -> + CInt -> + IO (Ptr OrtStatus) + +#{def + OrtStatus* _wrap_OrtApi_RunOptionsSetRunLogVerbosityLevel( + HsOrtRunOptions* options, + int logVerbosityLevel + ) { + return options->ortApi->RunOptionsSetRunLogVerbosityLevel( + options->ortRunOptions, + logVerbosityLevel + ); + } +} + +------------------------------------------------------------------------------- +-- OrtApi::RunOptionsSetRunLogSeverityLevel + +{- | +> ORT_API2_STATUS(RunOptionsSetRunLogSeverityLevel, +> _Inout_ OrtRunOptions* options, +> int log_severity_level +> ); +-} +ortApiRunOptionsSetRunLogSeverityLevel :: + OrtRunOptions -> + Int -> + IO () +ortApiRunOptionsSetRunLogSeverityLevel options logSeverityLevel = do + ortApi <- getOrtApi options + withOrtRunOptionsPtr options $ \optionsPtr -> do + ortStatusPtr <- + _wrap_OrtApi_RunOptionsSetRunLogSeverityLevel + optionsPtr + (fromIntegral logSeverityLevel) + handleOrtStatus ortApi ortStatusPtr $ do + pure () + +foreign import capi unsafe + "Onnxruntime/CApi_hsc.h _wrap_OrtApi_RunOptionsSetRunLogSeverityLevel" + _wrap_OrtApi_RunOptionsSetRunLogSeverityLevel :: + Ptr OrtRunOptions -> + CInt -> + IO (Ptr OrtStatus) + +#{def + OrtStatus* _wrap_OrtApi_RunOptionsSetRunLogSeverityLevel( + HsOrtRunOptions* options, + int logSeverityLevel + ) { + return options->ortApi->RunOptionsSetRunLogSeverityLevel( + options->ortRunOptions, + logSeverityLevel + ); + } +} + +------------------------------------------------------------------------------- +-- OrtApi::RunOptionsSetRunTag + +{- | +> ORT_API2_STATUS(RunOptionsSetRunTag, +> _Inout_ OrtRunOptions* options, +> _In_ const char* run_tag +> ); +-} +ortApiRunOptionsSetRunTag :: + OrtRunOptions -> + String -> + IO () +ortApiRunOptionsSetRunTag options runTag = do + ortApi <- getOrtApi options + withCString runTag $ \runTagPtr -> do + withOrtRunOptionsPtr options $ \optionsPtr -> do + ortStatusPtr <- + _wrap_OrtApi_RunOptionsSetRunTag + optionsPtr + (ConstPtr runTagPtr) -- NOTE: This is unsafe. + handleOrtStatus ortApi ortStatusPtr $ do + pure () + +foreign import capi unsafe + "Onnxruntime/CApi_hsc.h _wrap_OrtApi_RunOptionsSetRunTag" + _wrap_OrtApi_RunOptionsSetRunTag :: + Ptr OrtRunOptions -> + ConstPtr CChar -> + IO (Ptr OrtStatus) + +#{def + OrtStatus* _wrap_OrtApi_RunOptionsSetRunTag( + HsOrtRunOptions* options, + const char* runTag + ) { + return options->ortApi->RunOptionsSetRunTag( + options->ortRunOptions, + runTag + ); + } +} + +------------------------------------------------------------------------------- +-- OrtApi::RunOptionsGetRunLogVerbosityLevel + +{- | +> ORT_API2_STATUS(RunOptionsGetRunLogVerbosityLevel, +> _In_ const OrtRunOptions* options, +> _Out_ int* log_verbosity_level +> ); +-} +ortApiRunOptionsGetRunLogVerbosityLevel :: + OrtRunOptions -> + IO Int +ortApiRunOptionsGetRunLogVerbosityLevel options = do + ortApi <- getOrtApi options + alloca $ \outPtr -> do + withOrtRunOptionsPtr options $ \optionsPtr -> do + ortStatusPtr <- + _wrap_OrtApi_RunOptionsGetRunLogVerbosityLevel + optionsPtr + outPtr + handleOrtStatus ortApi ortStatusPtr $ do + fromIntegral <$> peek outPtr + +foreign import capi unsafe + "Onnxruntime/CApi_hsc.h _wrap_OrtApi_RunOptionsGetRunLogVerbosityLevel" + _wrap_OrtApi_RunOptionsGetRunLogVerbosityLevel :: + Ptr OrtRunOptions -> + Ptr CInt -> + IO (Ptr OrtStatus) + +#{def + OrtStatus* _wrap_OrtApi_RunOptionsGetRunLogVerbosityLevel( + HsOrtRunOptions* options, + int* logVerbosityLevel + ) { + return options->ortApi->RunOptionsGetRunLogVerbosityLevel( + options->ortRunOptions, + logVerbosityLevel + ); + } +} + +------------------------------------------------------------------------------- +-- OrtApi::RunOptionsGetRunLogSeverityLevel + +{- | +> ORT_API2_STATUS(RunOptionsGetRunLogSeverityLevel, +> _In_ const OrtRunOptions* options, +> _Out_ int* log_severity_level +> ); +-} +ortApiRunOptionsGetRunLogSeverityLevel :: + OrtRunOptions -> + IO Int +ortApiRunOptionsGetRunLogSeverityLevel options = do + ortApi <- getOrtApi options + alloca $ \outPtr -> do + withOrtRunOptionsPtr options $ \optionsPtr -> do + ortStatusPtr <- + _wrap_OrtApi_RunOptionsGetRunLogSeverityLevel + optionsPtr + outPtr + handleOrtStatus ortApi ortStatusPtr $ do + fromIntegral <$> peek outPtr + +foreign import capi unsafe + "Onnxruntime/CApi_hsc.h _wrap_OrtApi_RunOptionsGetRunLogSeverityLevel" + _wrap_OrtApi_RunOptionsGetRunLogSeverityLevel :: + Ptr OrtRunOptions -> + Ptr CInt -> + IO (Ptr OrtStatus) + +#{def + OrtStatus* _wrap_OrtApi_RunOptionsGetRunLogSeverityLevel( + HsOrtRunOptions* options, + int* logSeverityLevel + ) { + return options->ortApi->RunOptionsGetRunLogSeverityLevel( + options->ortRunOptions, + logSeverityLevel + ); + } +} + +------------------------------------------------------------------------------- +-- OrtApi::RunOptionsGetRunTag + +{- | +> ORT_API2_STATUS(RunOptionsGetRunTag, +> _In_ const OrtRunOptions* options, +> _Out_ const char** run_tag +> ); +-} +ortApiRunOptionsGetRunTag :: + OrtRunOptions -> + IO String +ortApiRunOptionsGetRunTag options = do + ortApi <- getOrtApi options + alloca @(ConstPtr CChar) $ \outPtr -> do + withOrtRunOptionsPtr options $ \optionsPtr -> do + ortStatusPtr <- + _wrap_OrtApi_RunOptionsGetRunTag + optionsPtr + outPtr + handleOrtStatus ortApi ortStatusPtr $ do + peekCString . unConstPtr + =<< peek outPtr + +foreign import capi unsafe + "Onnxruntime/CApi_hsc.h _wrap_OrtApi_RunOptionsGetRunTag" + _wrap_OrtApi_RunOptionsGetRunTag :: + Ptr OrtRunOptions -> + Ptr (ConstPtr CChar) -> + IO (Ptr OrtStatus) + +#{def + OrtStatus* _wrap_OrtApi_RunOptionsGetRunTag( + HsOrtRunOptions* options, + const char** runTag + ) { + return options->ortApi->RunOptionsGetRunTag( + options->ortRunOptions, + runTag + ); + } +} + +------------------------------------------------------------------------------- +-- OrtApi::RunOptionsSetTerminate + +{- | +> ORT_API2_STATUS(RunOptionsSetTerminate, +> _Inout_ OrtRunOptions* options +> ); +-} +ortApiRunOptionsSetTerminate :: + OrtRunOptions -> + IO () +ortApiRunOptionsSetTerminate options = do + ortApi <- getOrtApi options + withOrtRunOptionsPtr options $ \optionsPtr -> do + ortStatusPtr <- + _wrap_OrtApi_RunOptionsSetTerminate + optionsPtr + handleOrtStatus ortApi ortStatusPtr $ do + pure () + +foreign import capi unsafe + "Onnxruntime/CApi_hsc.h _wrap_OrtApi_RunOptionsSetTerminate" + _wrap_OrtApi_RunOptionsSetTerminate :: + Ptr OrtRunOptions -> + IO (Ptr OrtStatus) + +#{def + OrtStatus* _wrap_OrtApi_RunOptionsSetTerminate( + HsOrtRunOptions* options + ) { + return options->ortApi->RunOptionsSetTerminate( + options->ortRunOptions + ); + } +} + +------------------------------------------------------------------------------- +-- OrtApi::RunOptionsUnsetTerminate + +{- | +> ORT_API2_STATUS(RunOptionsUnsetTerminate, +> _Inout_ OrtRunOptions* options +> ); +-} +ortApiRunOptionsUnsetTerminate :: + OrtRunOptions -> + IO () +ortApiRunOptionsUnsetTerminate options = do + ortApi <- getOrtApi options + withOrtRunOptionsPtr options $ \optionsPtr -> do + ortStatusPtr <- + _wrap_OrtApi_RunOptionsUnsetTerminate + optionsPtr + handleOrtStatus ortApi ortStatusPtr $ do + pure () + +foreign import capi unsafe + "Onnxruntime/CApi_hsc.h _wrap_OrtApi_RunOptionsUnsetTerminate" + _wrap_OrtApi_RunOptionsUnsetTerminate :: + Ptr OrtRunOptions -> + IO (Ptr OrtStatus) + +#{def + OrtStatus* _wrap_OrtApi_RunOptionsUnsetTerminate( + HsOrtRunOptions* options + ) { + return options->ortApi->RunOptionsUnsetTerminate( + options->ortRunOptions + ); + } +} + +------------------------------------------------------------------------------- +-- OrtApi::CreateTensorAsOrtValue + +-- TODO: Required to get OrtApi::Run working. + +{- +> ORT_API2_STATUS(CreateTensorAsOrtValue, +> _Inout_ OrtAllocator* allocator, +> _In_ const int64_t* shape, +> size_t shape_len, +> ONNXTensorElementDataType type, +> _Outptr_ OrtValue** out +> ); +-} +ortApiCreateTensorAsOrtValue :: + OrtAllocator -> + [Int64] -> + ONNXTensorElementDataType -> + IO OrtValue +ortApiCreateTensorAsOrtValue allocator shape dataType = do + ortApi <- getOrtApi allocator + withCTypePtr allocator $ \cOrtAllocatorPtr -> do + withArrayLen shape $ \shapeLen shapePtr -> do + alloca $ \outPtr -> do + ortStatusPtr <- + _wrap_OrtApi_CreateTensorAsOrtValue + ortApi.ortApiConstPtr + cOrtAllocatorPtr + shapePtr + (fromIntegral shapeLen) + dataType + outPtr + handleOrtStatus ortApi ortStatusPtr $ do + wrapCOrtValue ortApi + =<< peek outPtr + +foreign import capi unsafe + "Onnxruntime/CApi_hsc.h _wrap_OrtApi_CreateTensorAsOrtValue" + _wrap_OrtApi_CreateTensorAsOrtValue :: + ConstPtr OrtApi -> + Ptr COrtAllocator -> + Ptr ( #{type int64_t} ) -> + ( #{type size_t} ) -> + ONNXTensorElementDataType -> + Ptr (Ptr COrtValue) -> + IO (Ptr OrtStatus) + +#{def + OrtStatus* _wrap_OrtApi_CreateTensorAsOrtValue( + const OrtApi* ortApi, + COrtAllocator* allocator, + const int64_t* shape, + size_t shape_len, + ONNXTensorElementDataType type, + COrtValue** out + ) { + return ortApi->CreateTensorAsOrtValue( + allocator, + shape, + shape_len, + type, + out + ); + } +} + +------------------------------------------------------------------------------- +-- OrtApi::CreateTensorWithDataAsOrtValue + +{- +> ORT_API2_STATUS(CreateTensorWithDataAsOrtValue, +> _In_ const OrtMemoryInfo* info, +> _Inout_ void* p_data, +> size_t p_data_len, +> _In_ const int64_t* shape, +> size_t shape_len, +> ONNXTensorElementDataType type, +> _Outptr_ OrtValue** out +> ); +-} +ortApiWithTensorWithDataAsOrtValue :: + forall a b. + (IsONNXTensorElementDataType a) => + OrtMemoryInfo -> + Vector a -> + [Int64] -> + (OrtValue -> IO b) -> + IO b +ortApiWithTensorWithDataAsOrtValue memoryInfo values shape action = do + ortApi <- getOrtApi memoryInfo + withCTypePtr memoryInfo $ \cOrtMemoryInfoPtr -> do + let valueLen = VS.length values + VS.unsafeWith values $ \valuePtr -> do + withArrayLen shape $ \shapeLen shapePtr -> + alloca $ \outPtr -> do + ortStatusPtr <- + _wrap_OrtApi_CreateTensorWithDataAsOrtValue + ortApi + cOrtMemoryInfoPtr + (castPtr valuePtr) + (fromIntegral $ valueLen * sizeOf (undefined :: a)) + shapePtr + (fromIntegral shapeLen) + (getONNXTensorElementDataType (Proxy :: Proxy a)) + outPtr + ortValue <- + handleOrtStatus ortApi ortStatusPtr $ do + wrapCOrtValue ortApi + =<< peek outPtr + action ortValue + +foreign import capi unsafe + "Onnxruntime/CApi_hsc.h _wrap_OrtApi_CreateTensorWithDataAsOrtValue" + _wrap_OrtApi_CreateTensorWithDataAsOrtValue :: + OrtApi -> + Ptr COrtMemoryInfo -> + Ptr Void -> + ( #{type size_t} ) -> + Ptr ( #{type int64_t} ) -> + ( #{type size_t} ) -> + ONNXTensorElementDataType -> + Ptr (Ptr COrtValue) -> + IO (Ptr OrtStatus) + +#{def + OrtStatus* _wrap_OrtApi_CreateTensorWithDataAsOrtValue( + const OrtApi* ortApi, + const COrtMemoryInfo* info, + void* p_data, + size_t p_data_len, + const int64_t* shape, + size_t shapeLen, + ONNXTensorElementDataType type, + COrtValue** out + ) { + return ortApi->CreateTensorWithDataAsOrtValue( + info, + p_data, + p_data_len, + shape, + shapeLen, + type, + out + ); + } +} + +------------------------------------------------------------------------------- +-- OrtApi::IsTensor + +{- +> ORT_API2_STATUS(IsTensor, +> _In_ const OrtValue* value, +> _Out_ int* out +> ); +-} +ortApiIsTensor :: + OrtValue -> + IO Bool +ortApiIsTensor ortValue = do + ortApi <- getOrtApi ortValue + withOrtValuePtr ortValue $ \ortValuePtr -> + alloca $ \outPtr -> do + ortStatusPtr <- + _wrap_OrtApi_IsTensor + ortValuePtr + outPtr + handleOrtStatus ortApi ortStatusPtr $ do + (==1) <$> peek outPtr + +foreign import capi unsafe + "Onnxruntime/CApi_hsc.h _wrap_OrtApi_IsTensor" + _wrap_OrtApi_IsTensor :: + Ptr OrtValue -> + Ptr ( #{type int} ) -> + IO (Ptr OrtStatus) + +#{def + OrtStatus* _wrap_OrtApi_IsTensor( + const HsOrtValue* value, + int* out + ) { + return value->ortApi->IsTensor( + value->ortValue, + out + ); + } +} + +------------------------------------------------------------------------------- +-- OrtApi::GetTensorMutableData + +data ONNXTypeError + = ErrONNXTypeMismatch + -- | Expected type. + !ONNXType + -- | Actual type. + !ONNXType + | ErrONNXTensorElementDataTypeMismatch + -- | Expected element data type. + !ONNXTensorElementDataType + -- | Actual element data type. + !ONNXTensorElementDataType + deriving (Eq, Show) + +instance Exception ONNXTypeError + +ortApiCheckType :: + ONNXType -> + OrtValue -> + IO () +ortApiCheckType expectedType ortValue = do + actualType <- ortApiGetValueType ortValue + unless (expectedType == actualType) $ + throwIO (ErrONNXTypeMismatch expectedType actualType) + +ortApiCheckTensorElementDataType :: + ONNXTensorElementDataType -> + OrtValue -> + IO () +ortApiCheckTensorElementDataType expectedElementType ortValue = do + ortApiCheckType ONNXTypeTensor ortValue + tensorTypeAndShape <- ortApiGetTensorTypeAndShape ortValue + actualElementType <- ortApiGetTensorElementType tensorTypeAndShape + unless (expectedElementType == actualElementType) $ + throwIO (ErrONNXTensorElementDataTypeMismatch expectedElementType actualElementType) + +{- +> ORT_API2_STATUS(GetTensorMutableData, +> _In_ OrtValue* value, +> _Outptr_ void** out +> ); +-} +ortApiWithTensorData :: + forall a b. + (IsONNXTensorElementDataType a) => + OrtValue -> + (Vector a -> IO b) -> + IO b +ortApiWithTensorData ortValue action = do + ortApi <- getOrtApi ortValue + -- Check the tensor type + ortApiCheckTensorElementDataType (getONNXTensorElementDataType (Proxy :: Proxy a)) ortValue + -- Get the tensor dimensions + tensorTypeAndShape <- ortApiGetTensorTypeAndShape ortValue + tensorElementCount <- ortApiGetTensorShapeElementCount tensorTypeAndShape + -- Get the tensor data + withOrtValuePtr ortValue $ \ortValuePtr -> + alloca $ \outPtr -> do + ortStatusPtr <- + _wrap_OrtApi_GetTensorMutableData + ortValuePtr + outPtr + mutableDataPtr <- + handleOrtStatus ortApi ortStatusPtr $ do + castPtr + <$> peek outPtr + mutableDataForeignPtr <- + newForeignPtr_ mutableDataPtr + action $ + VS.unsafeFromForeignPtr0 mutableDataForeignPtr (fromIntegral tensorElementCount) + +foreign import capi unsafe + "Onnxruntime/CApi_hsc.h _wrap_OrtApi_GetTensorMutableData" + _wrap_OrtApi_GetTensorMutableData :: + Ptr OrtValue -> + Ptr (Ptr Void) -> + IO (Ptr OrtStatus) + +#{def + OrtStatus* _wrap_OrtApi_GetTensorMutableData( + const HsOrtValue* value, + void** out + ) { + return value->ortApi->GetTensorMutableData( + value->ortValue, + out + ); + } +} + +------------------------------------------------------------------------------- +-- OrtApi::FillStringTensor + +{- +> ORT_API2_STATUS(FillStringTensor, +> _Inout_ OrtValue* value, +> _In_ const char* const* s, +> size_t s_len +> ); +-} + +-- TODO: unimplemented + +------------------------------------------------------------------------------- +-- OrtApi::GetStringTensorDataLength + +{- +> ORT_API2_STATUS(GetStringTensorDataLength, +> _In_ const OrtValue* value, +> _Out_ size_t* len +> ); +-} + +-- TODO: unimplemented + +------------------------------------------------------------------------------- +-- OrtApi::GetStringTensorContent + +{- +> ORT_API2_STATUS(GetStringTensorContent, +> _In_ const OrtValue* value, +> _Out_writes_bytes_all_(s_len) void* s, +> size_t s_len, +> _Out_writes_all_(offsets_len) size_t* offsets, +> size_t offsets_len +> ); +-} + +-- TODO: unimplemented + +------------------------------------------------------------------------------- +-- OrtApi::CastTypeInfoToTensorInfo + +{- +> ORT_API2_STATUS(CastTypeInfoToTensorInfo, +> _In_ const OrtTypeInfo* type_info, +> _Outptr_result_maybenull_ const OrtTensorTypeAndShapeInfo** out +> ); +-} +ortApiCastTypeInfoToTensorInfo :: + OrtTypeInfo -> + IO OrtTensorTypeAndShapeInfo +ortApiCastTypeInfoToTensorInfo ortTypeInfo = do + ortApi <- getOrtApi ortTypeInfo + withOrtTypeInfoPtr ortTypeInfo $ \ortTypeInfoPtr -> + alloca $ \outPtr -> do + ortStatusPtr <- + _wrap_OrtApi_CastTypeInfoToTensorInfo + (ConstPtr ortTypeInfoPtr) + outPtr + handleOrtStatus ortApi ortStatusPtr $ do + ConstPtr ortTypeAndShapeInfoPtr <- peek outPtr + -- If the ortTypeInfo does not represent a tensor type, + -- then OrtApi::CastTypeInfoToTensorInfo returns NULL. + if ortTypeAndShapeInfoPtr == nullPtr + then do + actualType <- ortApiGetOnnxTypeFromTypeInfo ortTypeInfo + throwIO (ErrONNXTypeMismatch ONNXTypeTensor actualType) + else + wrapCOrtTensorTypeAndShapeInfoFromOrtTypeInfo ortApi ortTypeInfo ortTypeAndShapeInfoPtr + +foreign import capi unsafe + "Onnxruntime/CApi_hsc.h _wrap_OrtApi_CastTypeInfoToTensorInfo" + _wrap_OrtApi_CastTypeInfoToTensorInfo :: + ConstPtr OrtTypeInfo -> + Ptr (ConstPtr COrtTensorTypeAndShapeInfo) -> + IO (Ptr OrtStatus) + +#{def + OrtStatus* _wrap_OrtApi_CastTypeInfoToTensorInfo( + const HsOrtTypeInfo* value, + const COrtTensorTypeAndShapeInfo** out + ) { + return value->ortApi->CastTypeInfoToTensorInfo( + value->ortTypeInfo, + out + ); + } +} + +------------------------------------------------------------------------------- +-- OrtApi::GetOnnxTypeFromTypeInfo + +{- +> ORT_API2_STATUS(GetOnnxTypeFromTypeInfo, +> _In_ const OrtTypeInfo* type_info, +> _Out_ enum ONNXType* out +> ); +-} +ortApiGetOnnxTypeFromTypeInfo :: + OrtTypeInfo -> + IO ONNXType +ortApiGetOnnxTypeFromTypeInfo ortTypeInfo = do + ortApi <- getOrtApi ortTypeInfo + withOrtTypeInfoPtr ortTypeInfo $ \ortTypeInfoPtr -> + alloca $ \outPtr -> do + ortStatusPtr <- + _wrap_OrtApi_GetOnnxTypeFromTypeInfo + (ConstPtr ortTypeInfoPtr) + outPtr + handleOrtStatus ortApi ortStatusPtr $ + ONNXType + <$> peek outPtr + +foreign import capi unsafe + "Onnxruntime/CApi_hsc.h _wrap_OrtApi_GetOnnxTypeFromTypeInfo" + _wrap_OrtApi_GetOnnxTypeFromTypeInfo :: + ConstPtr OrtTypeInfo -> + Ptr ( #{type ONNXType} ) -> + IO (Ptr OrtStatus) + +#{def + OrtStatus* _wrap_OrtApi_GetOnnxTypeFromTypeInfo( + const HsOrtTypeInfo* value, + ONNXType* out + ) { + return value->ortApi->GetOnnxTypeFromTypeInfo( + value->ortTypeInfo, + out + ); + } +} + +------------------------------------------------------------------------------- +-- OrtApi::CreateTensorTypeAndShapeInfo + +{- +> ORT_API2_STATUS(CreateTensorTypeAndShapeInfo, +> _Outptr_ OrtTensorTypeAndShapeInfo** out +> ); +-} + +-- TODO: unimplemented + +------------------------------------------------------------------------------- +-- OrtApi::SetTensorElementType + +{- +> ORT_API2_STATUS(SetTensorElementType, +> _Inout_ OrtTensorTypeAndShapeInfo* info, +> enum ONNXTensorElementDataType type +> ); +-} + +-- TODO: unimplemented + +------------------------------------------------------------------------------- +-- OrtApi::SetDimensions + +{- +> ORT_API2_STATUS(SetDimensions, +> OrtTensorTypeAndShapeInfo* info, +> _In_ const int64_t* dim_values, +> size_t dim_count +> ); +-} + +-- TODO: unimplemented + +------------------------------------------------------------------------------- +-- OrtApi::GetTensorElementType + +{- +> ORT_API2_STATUS(GetTensorElementType, +> _In_ const OrtTensorTypeAndShapeInfo* info, +> _Out_ enum ONNXTensorElementDataType* out +> ); +-} +ortApiGetTensorElementType :: + OrtTensorTypeAndShapeInfo -> + IO ONNXTensorElementDataType +ortApiGetTensorElementType ortTensortTypeAndShapeInfo = do + ortApi <- getOrtApi ortTensortTypeAndShapeInfo + withOrtTensorTypeAndShapeInfoPtr ortTensortTypeAndShapeInfo $ \ortTensortTypeAndShapeInfoPtr -> + alloca $ \outPtr -> do + ortStatusPtr <- + _wrap_OrtApi_GetTensorElementType + (ConstPtr ortTensortTypeAndShapeInfoPtr) + outPtr + handleOrtStatus ortApi ortStatusPtr $ + ONNXTensorElementDataType <$> + peek outPtr + +foreign import capi unsafe + "Onnxruntime/CApi_hsc.h _wrap_OrtApi_GetTensorElementType" + _wrap_OrtApi_GetTensorElementType :: + ConstPtr OrtTensorTypeAndShapeInfo -> + Ptr ( #{type ONNXTensorElementDataType} ) -> + IO (Ptr OrtStatus) + +#{def + OrtStatus* _wrap_OrtApi_GetTensorElementType( + const HsOrtTensorTypeAndShapeInfo* value, + ONNXTensorElementDataType* out + ) { + return value->ortApi->GetTensorElementType( + value->ortTensorTypeAndShapeInfo, + out + ); + } +} + +------------------------------------------------------------------------------- +-- OrtApi::GetDimensionsCount + +{- +> ORT_API2_STATUS(GetDimensionsCount, +> _In_ const OrtTensorTypeAndShapeInfo* info, +> _Out_ size_t* out +> ); +-} +ortApiGetDimensionsCount :: + OrtTensorTypeAndShapeInfo -> + IO Word64 +ortApiGetDimensionsCount ortTensortTypeAndShapeInfo = do + ortApi <- getOrtApi ortTensortTypeAndShapeInfo + withOrtTensorTypeAndShapeInfoPtr ortTensortTypeAndShapeInfo $ \ortTensortTypeAndShapeInfoPtr -> + alloca $ \outPtr -> do + ortStatusPtr <- + _wrap_OrtApi_GetDimensionsCount + (ConstPtr ortTensortTypeAndShapeInfoPtr) + outPtr + handleOrtStatus ortApi ortStatusPtr $ do + CSize dimValuesLen <- peek outPtr + pure dimValuesLen + +foreign import capi unsafe + "Onnxruntime/CApi_hsc.h _wrap_OrtApi_GetDimensionsCount" + _wrap_OrtApi_GetDimensionsCount :: + ConstPtr OrtTensorTypeAndShapeInfo -> + Ptr CSize -> + IO (Ptr OrtStatus) + +#{def + OrtStatus* _wrap_OrtApi_GetDimensionsCount( + const HsOrtTensorTypeAndShapeInfo* value, + size_t* out + ) { + return value->ortApi->GetDimensionsCount( + value->ortTensorTypeAndShapeInfo, + out + ); + } +} + +------------------------------------------------------------------------------- +-- OrtApi::GetDimensions + +{- +> ORT_API2_STATUS(GetDimensions, +> _In_ const OrtTensorTypeAndShapeInfo* info, +> _Out_ int64_t* dim_values, +> size_t dim_values_length +> ); +-} +ortApiGetDimensions :: + OrtTensorTypeAndShapeInfo -> + IO [Int64] +ortApiGetDimensions ortTensortTypeAndShapeInfo = do + dimValuesLen <- ortApiGetDimensionsCount ortTensortTypeAndShapeInfo + ortApi <- getOrtApi ortTensortTypeAndShapeInfo + withOrtTensorTypeAndShapeInfoPtr ortTensortTypeAndShapeInfo $ \ortTensortTypeAndShapeInfoPtr -> + alloca $ \dimValuesPtr -> do + ortStatusPtr <- + _wrap_OrtApi_GetDimensions + (ConstPtr ortTensortTypeAndShapeInfoPtr) + dimValuesPtr + (CSize dimValuesLen) + handleOrtStatus ortApi ortStatusPtr $ + peekArray (fromIntegral dimValuesLen) dimValuesPtr + +foreign import capi unsafe + "Onnxruntime/CApi_hsc.h _wrap_OrtApi_GetDimensions" + _wrap_OrtApi_GetDimensions :: + ConstPtr OrtTensorTypeAndShapeInfo -> + Ptr ( #{type int64_t} ) -> + CSize -> + IO (Ptr OrtStatus) + +#{def + OrtStatus* _wrap_OrtApi_GetDimensions( + const HsOrtTensorTypeAndShapeInfo* value, + int64_t* dim_values, + size_t dim_values_length + ) { + return value->ortApi->GetDimensions( + value->ortTensorTypeAndShapeInfo, + dim_values, + dim_values_length + ); + } +} + +------------------------------------------------------------------------------- +-- OrtApi::GetSymbolicDimensions + +{- +> ORT_API2_STATUS(GetSymbolicDimensions, +> _In_ const OrtTensorTypeAndShapeInfo* info, +> _Out_writes_all_(dim_params_length) const char* dim_params[], +> size_t dim_params_length +> ); +-} + +-- TODO: unimplemented + +------------------------------------------------------------------------------- +-- OrtApi::GetTensorShapeElementCount + +{- +> ORT_API2_STATUS(GetTensorShapeElementCount, +> _In_ const OrtTensorTypeAndShapeInfo* info, +> _Out_ size_t* out +> ); +-} +ortApiGetTensorShapeElementCount :: + OrtTensorTypeAndShapeInfo -> + IO Word64 +ortApiGetTensorShapeElementCount ortTensortTypeAndShapeInfo = do + ortApi <- getOrtApi ortTensortTypeAndShapeInfo + withOrtTensorTypeAndShapeInfoPtr ortTensortTypeAndShapeInfo $ \ortTensortTypeAndShapeInfoPtr -> + alloca $ \outPtr -> do + ortStatusPtr <- + _wrap_OrtApi_GetTensorShapeElementCount + (ConstPtr ortTensortTypeAndShapeInfoPtr) + outPtr + handleOrtStatus ortApi ortStatusPtr $ + fromIntegral + <$> peek outPtr + +foreign import capi unsafe + "Onnxruntime/CApi_hsc.h _wrap_OrtApi_GetTensorShapeElementCount" + _wrap_OrtApi_GetTensorShapeElementCount :: + ConstPtr OrtTensorTypeAndShapeInfo -> + Ptr CSize -> + IO (Ptr OrtStatus) + +#{def + OrtStatus* _wrap_OrtApi_GetTensorShapeElementCount( + const HsOrtTensorTypeAndShapeInfo* value, + size_t* out + ) { + return value->ortApi->GetTensorShapeElementCount( + value->ortTensorTypeAndShapeInfo, + out + ); + } +} + +------------------------------------------------------------------------------- +-- OrtApi::GetTensorTypeAndShape + +{- +> ORT_API2_STATUS(GetTensorTypeAndShape, +> _In_ const OrtValue* value, +> _Outptr_ OrtTensorTypeAndShapeInfo** out +> ); +-} +ortApiGetTensorTypeAndShape :: + OrtValue -> + IO OrtTensorTypeAndShapeInfo +ortApiGetTensorTypeAndShape ortValue = do + ortApi <- getOrtApi ortValue + withOrtValuePtr ortValue $ \ortValuePtr -> + alloca $ \outPtr -> do + ortStatusPtr <- + _wrap_OrtApi_GetTensorTypeAndShape + (ConstPtr ortValuePtr) + outPtr + handleOrtStatus ortApi ortStatusPtr $ + wrapCOrtTensorTypeAndShapeInfo ortApi + =<< peek outPtr + +foreign import capi unsafe + "Onnxruntime/CApi_hsc.h _wrap_OrtApi_GetTensorTypeAndShape" + _wrap_OrtApi_GetTensorTypeAndShape :: + ConstPtr OrtValue -> + Ptr (Ptr COrtTensorTypeAndShapeInfo) -> + IO (Ptr OrtStatus) + +#{def + OrtStatus* _wrap_OrtApi_GetTensorTypeAndShape( + const HsOrtValue* value, + COrtTensorTypeAndShapeInfo** out + ) { + return value->ortApi->GetTensorTypeAndShape( + value->ortValue, + out + ); + } +} + +------------------------------------------------------------------------------- +-- OrtApi::GetTypeInfo + +{- +> ORT_API2_STATUS(GetTypeInfo, +> _In_ const OrtValue* value, +> _Outptr_result_maybenull_ OrtTypeInfo** out +> ); +-} +ortApiGetTypeInfo :: + OrtValue -> + IO OrtTypeInfo +ortApiGetTypeInfo ortValue = do + ortApi <- getOrtApi ortValue + withOrtValuePtr ortValue $ \ortValuePtr -> + alloca $ \outPtr -> do + ortStatusPtr <- + _wrap_OrtApi_GetTypeInfo + (ConstPtr ortValuePtr) + outPtr + handleOrtStatus ortApi ortStatusPtr $ + wrapCOrtTypeInfo ortApi + =<< peek outPtr + +foreign import capi unsafe + "Onnxruntime/CApi_hsc.h _wrap_OrtApi_GetTypeInfo" + _wrap_OrtApi_GetTypeInfo :: + ConstPtr OrtValue -> + Ptr (Ptr COrtTypeInfo) -> + IO (Ptr OrtStatus) + +#{def + OrtStatus* _wrap_OrtApi_GetTypeInfo( + const HsOrtValue* value, + COrtTypeInfo** out + ) { + return value->ortApi->GetTypeInfo( + value->ortValue, + out + ); + } +} + +------------------------------------------------------------------------------- +-- OrtApi::GetValueType + +{- +> ORT_API2_STATUS(GetValueType, +> _In_ const OrtValue* value, +> _Out_ enum ONNXType* out +> ); +-} +ortApiGetValueType :: + OrtValue -> + IO ONNXType +ortApiGetValueType ortValue = do + ortApi <- getOrtApi ortValue + withOrtValuePtr ortValue $ \ortValuePtr -> + alloca $ \outPtr -> do + ortStatusPtr <- + _wrap_OrtApi_GetValueType + (ConstPtr ortValuePtr) + outPtr + handleOrtStatus ortApi ortStatusPtr $ + ONNXType + <$> peek outPtr + +foreign import capi unsafe + "Onnxruntime/CApi_hsc.h _wrap_OrtApi_GetValueType" + _wrap_OrtApi_GetValueType :: + ConstPtr OrtValue -> + Ptr ( #{type ONNXType} ) -> + IO (Ptr OrtStatus) + +#{def + OrtStatus* _wrap_OrtApi_GetValueType( + const HsOrtValue* value, + ONNXType* out + ) { + return value->ortApi->GetValueType( + value->ortValue, + out + ); + } +} + +------------------------------------------------------------------------------- +-- OrtApi::CreateMemoryInfo + +{- +> ORT_API2_STATUS(CreateMemoryInfo, +> _In_ const char* name, +> enum OrtAllocatorType type, +> int id, +> enum OrtMemType mem_type, +> _Outptr_ OrtMemoryInfo** out +> ); +-} +ortApiCreateMemoryInfo :: + OrtApi -> + String -> + OrtAllocatorType -> + Int -> + OrtMemType -> + IO OrtMemoryInfo +ortApiCreateMemoryInfo ortApi allocatorName allocatorType allocatorId memoryType = do + withCString allocatorName $ \allocatorNamePtr -> do + alloca $ \outPtr -> do + ortStatusPtr <- + _wrap_OrtApi_CreateMemoryInfo + ortApi.ortApiConstPtr + (ConstPtr allocatorNamePtr) -- NOTE: This is unsafe. + allocatorType + (fromIntegral allocatorId) + memoryType + outPtr + handleOrtStatus ortApi ortStatusPtr $ do + wrapCOrtMemoryInfo ortApi + =<< peek outPtr + +foreign import capi unsafe + "Onnxruntime/CApi_hsc.h _wrap_OrtApi_CreateMemoryInfo" + _wrap_OrtApi_CreateMemoryInfo :: + ConstPtr OrtApi -> + ConstPtr CChar -> + OrtAllocatorType -> + ( #{type int} ) -> + OrtMemType -> + Ptr (Ptr COrtMemoryInfo) -> + IO (Ptr OrtStatus) + +#{def + OrtStatus* _wrap_OrtApi_CreateMemoryInfo( + const OrtApi* ortApi, + const char* name, + enum OrtAllocatorType type, + int id, + enum OrtMemType mem_type, + COrtMemoryInfo** out + ) { + return ortApi->CreateMemoryInfo(name, type, id, mem_type, out); + } +} + +------------------------------------------------------------------------------- +-- OrtApi::CreateCpuMemoryInfo + +{- +> ORT_API2_STATUS(CreateCpuMemoryInfo, +> enum OrtAllocatorType type, +> enum OrtMemType mem_type, +> _Outptr_ OrtMemoryInfo** out +> ); +-} +ortApiCreateCpuMemoryInfo :: + OrtApi -> + OrtAllocatorType -> + OrtMemType -> + IO OrtMemoryInfo +ortApiCreateCpuMemoryInfo ortApi allocatorType memoryType = do + alloca $ \outPtr -> do + ortStatusPtr <- + _wrap_OrtApi_CreateCpuMemoryInfo + ortApi.ortApiConstPtr + allocatorType + memoryType + outPtr + handleOrtStatus ortApi ortStatusPtr $ do + wrapCOrtMemoryInfo ortApi + =<< peek outPtr + +foreign import capi unsafe + "Onnxruntime/CApi_hsc.h _wrap_OrtApi_CreateCpuMemoryInfo" + _wrap_OrtApi_CreateCpuMemoryInfo :: + ConstPtr OrtApi -> + OrtAllocatorType -> + OrtMemType -> + Ptr (Ptr COrtMemoryInfo) -> + IO (Ptr OrtStatus) + +#{def + OrtStatus* _wrap_OrtApi_CreateCpuMemoryInfo( + const OrtApi* ortApi, + enum OrtAllocatorType type, + enum OrtMemType mem_type, + COrtMemoryInfo** out + ) { + return ortApi->CreateCpuMemoryInfo(type, mem_type, out); + } +} + +------------------------------------------------------------------------------- +-- OrtApi::CompareMemoryInfo + +{- +> ORT_API2_STATUS(CompareMemoryInfo, +> _In_ const OrtMemoryInfo* info1, +> _In_ const OrtMemoryInfo* info2, +> _Out_ int* out +> ); +-} + +-- TODO: unimplemented + +------------------------------------------------------------------------------- +-- OrtApi::MemoryInfoGetName + +{- +> ORT_API2_STATUS(MemoryInfoGetName, +> _In_ const OrtMemoryInfo* ptr, +> _Out_ const char** out +> ); +-} + +-- TODO: unimplemented + +------------------------------------------------------------------------------- +-- OrtApi::MemoryInfoGetId + +{- +> ORT_API2_STATUS(MemoryInfoGetId, +> _In_ const OrtMemoryInfo* ptr, +> _Out_ int* out +> ); +-} + +-- TODO: unimplemented + +------------------------------------------------------------------------------- +-- OrtApi::MemoryInfoGetMemType + +{- +> ORT_API2_STATUS(MemoryInfoGetMemType, +> _In_ const OrtMemoryInfo* ptr, +> _Out_ OrtMemType* out +> ); +-} + +-- TODO: unimplemented + +------------------------------------------------------------------------------- +-- OrtApi::MemoryInfoGetType + +{- +> ORT_API2_STATUS(MemoryInfoGetType, +> _In_ const OrtMemoryInfo* ptr, +> _Out_ OrtAllocatorType* out +> ); +-} + +-- TODO: unimplemented + +------------------------------------------------------------------------------- +-- OrtApi::AllocatorAlloc + +{- +> ORT_API2_STATUS(AllocatorAlloc, +> _Inout_ OrtAllocator* ort_allocator, +> size_t size, +> _Outptr_ void** out +> ); +-} + +-- TODO: unimplemented + +------------------------------------------------------------------------------- +-- OrtApi::AllocatorFree + +{- +> ORT_API2_STATUS(AllocatorFree, +> _Inout_ OrtAllocator* ort_allocator, +> void* p +> ); +-} + +-- TODO: unimplemented + +------------------------------------------------------------------------------- +-- OrtApi::AllocatorGetInfo + +{- +> ORT_API2_STATUS(AllocatorGetInfo, +> _In_ const OrtAllocator* ort_allocator, +> _Outptr_ const struct OrtMemoryInfo** out +> ); +-} + +-- TODO: unimplemented + +------------------------------------------------------------------------------- +-- OrtApi::GetAllocatorWithDefaultOptions + +{- | +> ORT_API2_STATUS(GetAllocatorWithDefaultOptions, +> _Outptr_ OrtAllocator** out +> ); +-} +ortApiGetAllocatorWithDefaultOptions :: + OrtApi -> + IO OrtAllocator +ortApiGetAllocatorWithDefaultOptions ortApi = do + alloca $ \outPtr -> do + ortStatusPtr <- + _wrap_OrtApi_GetAllocatorWithDefaultOptions + ortApi.ortApiConstPtr + outPtr + handleOrtStatus ortApi ortStatusPtr $ do + wrapCOrtAllocator ortApi + =<< peek outPtr + +foreign import capi unsafe + "Onnxruntime/CApi_hsc.h _wrap_OrtApi_GetAllocatorWithDefaultOptions" + _wrap_OrtApi_GetAllocatorWithDefaultOptions :: + ConstPtr OrtApi -> + Ptr (Ptr COrtAllocator) -> + IO (Ptr OrtStatus) + +#{def + OrtStatus* _wrap_OrtApi_GetAllocatorWithDefaultOptions( + const OrtApi* ortApi, + OrtAllocator** out + ) { + return ortApi->GetAllocatorWithDefaultOptions(out); + } +} + +------------------------------------------------------------------------------- +-- OrtApi::AddFreeDimensionOverride + +{- | +> ORT_API2_STATUS(AddFreeDimensionOverride, +> _Inout_ OrtSessionOptions* options, +> _In_ const char* dim_denotation, +> _In_ int64_t dim_value +> ); +-} +ortApiAddFreeDimensionOverride :: + OrtSessionOptions -> + String -> + Int -> + IO () +ortApiAddFreeDimensionOverride options dimDenotation dimValue = do + ortApi <- getOrtApi options + withCString dimDenotation $ \dimDenotationPtr -> + withOrtSessionOptionsPtr options $ \optionsPtr -> do + ortStatusPtr <- + _wrap_OrtApi_AddFreeDimensionOverride + optionsPtr + (ConstPtr dimDenotationPtr) -- NOTE: This is unsafe. + (fromIntegral dimValue) + handleOrtStatus ortApi ortStatusPtr $ do + pure () + +foreign import capi unsafe + "Onnxruntime/CApi_hsc.h _wrap_OrtApi_AddFreeDimensionOverride" + _wrap_OrtApi_AddFreeDimensionOverride :: + Ptr OrtSessionOptions -> + ConstPtr CChar -> + ( #{type int64_t} ) -> + IO (Ptr OrtStatus) + +#{def + OrtStatus* _wrap_OrtApi_AddFreeDimensionOverride( + HsOrtSessionOptions* options, + const char* dimDenotation, + int64_t dimValue + ) { + return options->ortApi->AddFreeDimensionOverride( + options->ortSessionOptions, + dimDenotation, + dimValue + ); + } +} + +------------------------------------------------------------------------------- +-- OrtApi::GetValue + +{- +> ORT_API2_STATUS(GetValue, +> _In_ const OrtValue* value, +> int index, +> _Inout_ OrtAllocator* allocator, +> _Outptr_ OrtValue** out +> ); +-} + +-- TODO: unimplemented + +------------------------------------------------------------------------------- +-- OrtApi::GetValueCount + +{- +> ORT_API2_STATUS(GetValueCount, +> _In_ const OrtValue* value, +> _Out_ size_t* out +> ); +-} + +-- TODO: unimplemented + +------------------------------------------------------------------------------- +-- OrtApi::CreateValue + +{- +> ORT_API2_STATUS(CreateValue, +> _In_reads_(num_values) const OrtValue* const* in, +> size_t num_values, +> enum ONNXType value_type, +> _Outptr_ OrtValue** out +> ); +-} + +-- TODO: unimplemented + +------------------------------------------------------------------------------- +-- OrtApi::CreateOpaqueValue + +{- +> ORT_API2_STATUS(CreateOpaqueValue, +> _In_z_ const char* domain_name, +> _In_z_ const char* type_name, +> _In_ const void* data_container, +> size_t data_container_size, +> _Outptr_ OrtValue** out +> ); +-} + +-- TODO: unimplemented + +------------------------------------------------------------------------------- +-- OrtApi::GetOpaqueValue + +{- +> ORT_API2_STATUS(GetOpaqueValue, +> _In_ const char* domain_name, +> _In_ const char* type_name, +> _In_ const OrtValue* in, +> _Out_ void* data_container, +> size_t data_container_size +> ); +-} + +-- TODO: unimplemented + +------------------------------------------------------------------------------- +-- OrtApi::KernelInfoGetAttribute_float + +-- ORT_API2_STATUS(KernelInfoGetAttribute_float, _In_ const OrtKernelInfo* info, _In_ const char* name, _Out_ float* out); + +-- TODO: unimplemented + +------------------------------------------------------------------------------- +-- OrtApi::KernelInfoGetAttribute_int64 + +-- ORT_API2_STATUS(KernelInfoGetAttribute_int64, _In_ const OrtKernelInfo* info, _In_ const char* name, _Out_ int64_t* out); + +-- TODO: unimplemented + +------------------------------------------------------------------------------- +-- OrtApi::KernelInfoGetAttribute_string + +-- ORT_API2_STATUS(KernelInfoGetAttribute_string, _In_ const OrtKernelInfo* info, _In_ const char* name, _Out_ char* out, _Inout_ size_t* size); + +-- TODO: unimplemented + +------------------------------------------------------------------------------- +-- OrtApi::KernelContext_GetInputCount + +-- ORT_API2_STATUS(KernelContext_GetInputCount, _In_ const OrtKernelContext* context, _Out_ size_t* out); + +-- TODO: unimplemented + +------------------------------------------------------------------------------- +-- OrtApi::KernelContext_GetOutputCount + +-- ORT_API2_STATUS(KernelContext_GetOutputCount, _In_ const OrtKernelContext* context, _Out_ size_t* out); + +-- TODO: unimplemented + +------------------------------------------------------------------------------- +-- OrtApi::KernelContext_GetInput + +-- ORT_API2_STATUS(KernelContext_GetInput, _In_ const OrtKernelContext* context, _In_ size_t index, _Out_ const OrtValue** out); + +-- TODO: unimplemented + +------------------------------------------------------------------------------- +-- OrtApi::KernelContext_GetOutput + +-- ORT_API2_STATUS(KernelContext_GetOutput, _Inout_ OrtKernelContext* context, _In_ size_t index, _In_ const int64_t* dim_values, size_t dim_count, _Outptr_ OrtValue** out); + +-- TODO: unimplemented + +------------------------------------------------------------------------------- +-- OrtApi::GetDenotationFromTypeInfo + +{- +> ORT_API2_STATUS(GetDenotationFromTypeInfo, +> _In_ const OrtTypeInfo* type_info, +> _Out_ const char** const denotation, +> _Out_ size_t* len +> ); +-} + +-- TODO: unimplemented + +------------------------------------------------------------------------------- +-- OrtApi::CastTypeInfoToMapTypeInfo + +{- +> ORT_API2_STATUS(CastTypeInfoToMapTypeInfo, +> _In_ const OrtTypeInfo* type_info, +> _Outptr_result_maybenull_ const OrtMapTypeInfo** out +> ); +-} + +-- TODO: unimplemented + +------------------------------------------------------------------------------- +-- OrtApi::CastTypeInfoToSequenceTypeInfo + +{- +> ORT_API2_STATUS(CastTypeInfoToSequenceTypeInfo, +> _In_ const OrtTypeInfo* type_info, +> _Outptr_result_maybenull_ const OrtSequenceTypeInfo** out +> ); +-} + +-- TODO: unimplemented + + +-------------------------------------------------------------------------------- +-- OrtApi::GetMapKeyType + +-- > ORT_API2_STATUS(GetMapKeyType, _In_ const OrtMapTypeInfo* map_type_info, _Out_ enum ONNXTensorElementDataType* out); + +-- TODO: unimplemented + +-------------------------------------------------------------------------------- +-- OrtApi::GetMapValueType + +-- > ORT_API2_STATUS(GetMapValueType, _In_ const OrtMapTypeInfo* map_type_info, _Outptr_ OrtTypeInfo** type_info); + +-- TODO: unimplemented + +-------------------------------------------------------------------------------- +-- OrtApi::GetSequenceElementType + +-- > ORT_API2_STATUS(GetSequenceElementType, _In_ const OrtSequenceTypeInfo* sequence_type_info, _Outptr_ OrtTypeInfo** type_info); + +-- TODO: unimplemented + +-------------------------------------------------------------------------------- +-- OrtApi::SessionEndProfiling + +-- > ORT_API2_STATUS(SessionEndProfiling, _In_ OrtSession* session, _Inout_ OrtAllocator* allocator, _Outptr_ char** out); + +-- TODO: unimplemented + +-------------------------------------------------------------------------------- +-- OrtApi::SessionGetModelMetadata + +-- > ORT_API2_STATUS(SessionGetModelMetadata, _In_ const OrtSession* session, _Outptr_ OrtModelMetadata** out); + +-- TODO: unimplemented + +-------------------------------------------------------------------------------- +-- OrtApi::ModelMetadataGetProducerName + +-- > ORT_API2_STATUS(ModelMetadataGetProducerName, _In_ const OrtModelMetadata* model_metadata, _Inout_ OrtAllocator* allocator, _Outptr_ char** value); + +-- TODO: unimplemented + +-------------------------------------------------------------------------------- +-- OrtApi::ModelMetadataGetGraphName + +-- > ORT_API2_STATUS(ModelMetadataGetGraphName, _In_ const OrtModelMetadata* model_metadata, _Inout_ OrtAllocator* allocator, _Outptr_ char** value); + +-- TODO: unimplemented + +-------------------------------------------------------------------------------- +-- OrtApi::ModelMetadataGetDomain + +-- > ORT_API2_STATUS(ModelMetadataGetDomain, _In_ const OrtModelMetadata* model_metadata, _Inout_ OrtAllocator* allocator, _Outptr_ char** value); + +-- TODO: unimplemented + +-------------------------------------------------------------------------------- +-- OrtApi::ModelMetadataGetDescription + +-- > ORT_API2_STATUS(ModelMetadataGetDescription, _In_ const OrtModelMetadata* model_metadata, _Inout_ OrtAllocator* allocator, _Outptr_ char** value); + +-- TODO: unimplemented + +-------------------------------------------------------------------------------- +-- OrtApi::ModelMetadataLookupCustomMetadataMap + +-- > ORT_API2_STATUS(ModelMetadataLookupCustomMetadataMap, _In_ const OrtModelMetadata* model_metadata, _Inout_ OrtAllocator* allocator, _In_ const char* key, _Outptr_result_maybenull_ char** value); + +-- TODO: unimplemented + +-------------------------------------------------------------------------------- +-- OrtApi::ModelMetadataGetVersion + +-- > ORT_API2_STATUS(ModelMetadataGetVersion, _In_ const OrtModelMetadata* model_metadata, _Out_ int64_t* value); + +-- TODO: unimplemented + +-------------------------------------------------------------------------------- +-- OrtApi::CreateEnvWithGlobalThreadPools + +-- > ORT_API2_STATUS(CreateEnvWithGlobalThreadPools, OrtLoggingLevel log_severity_level, _In_ const char* logid, _In_ const OrtThreadingOptions* tp_options, _Outptr_ OrtEnv** out); + +-- TODO: unimplemented + +-------------------------------------------------------------------------------- +-- OrtApi::DisablePerSessionThreads + +-- > ORT_API2_STATUS(DisablePerSessionThreads, _Inout_ OrtSessionOptions* options); + +-- TODO: unimplemented + +-------------------------------------------------------------------------------- +-- OrtApi::CreateThreadingOptions + +-- > ORT_API2_STATUS(CreateThreadingOptions, _Outptr_ OrtThreadingOptions** out); + +-- TODO: unimplemented + +-------------------------------------------------------------------------------- +-- OrtApi::ModelMetadataGetCustomMetadataMapKeys + +-- > ORT_API2_STATUS(ModelMetadataGetCustomMetadataMapKeys, _In_ const OrtModelMetadata* model_metadata, _Inout_ OrtAllocator* allocator, _Outptr_result_buffer_maybenull_(*num_keys) char*** keys, _Out_ int64_t* num_keys); + +-- TODO: unimplemented + +-------------------------------------------------------------------------------- +-- OrtApi::AddFreeDimensionOverrideByName + +-- > ORT_API2_STATUS(AddFreeDimensionOverrideByName, _Inout_ OrtSessionOptions* options, _In_ const char* dim_name, _In_ int64_t dim_value); + +-- TODO: unimplemented + +-------------------------------------------------------------------------------- +-- OrtApi::GetAvailableProviders + +-- > ORT_API2_STATUS(GetAvailableProviders, _Outptr_ char*** out_ptr, _Out_ int* provider_length); + +-- TODO: unimplemented + +-------------------------------------------------------------------------------- +-- OrtApi::ReleaseAvailableProviders + +-- > ORT_API2_STATUS(ReleaseAvailableProviders, _In_ char** ptr, _In_ int providers_length); + +-- TODO: unimplemented + +-------------------------------------------------------------------------------- +-- OrtApi::GetStringTensorElementLength + +-- > ORT_API2_STATUS(GetStringTensorElementLength, _In_ const OrtValue* value, size_t index, _Out_ size_t* out); + +-- TODO: unimplemented + +-------------------------------------------------------------------------------- +-- OrtApi::GetStringTensorElement + +-- > ORT_API2_STATUS(GetStringTensorElement, _In_ const OrtValue* value, size_t s_len, size_t index, _Out_writes_bytes_all_(s_len) void* s); + +-- TODO: unimplemented + +-------------------------------------------------------------------------------- +-- OrtApi::FillStringTensorElement + +-- > ORT_API2_STATUS(FillStringTensorElement, _Inout_ OrtValue* value, _In_ const char* s, size_t index); + +-- TODO: unimplemented + +-------------------------------------------------------------------------------- +-- OrtApi::AddSessionConfigEntry + +-- > ORT_API2_STATUS(AddSessionConfigEntry, _Inout_ OrtSessionOptions* options, _In_z_ const char* config_key, _In_z_ const char* config_value); + +-- TODO: unimplemented + +-------------------------------------------------------------------------------- +-- OrtApi::CreateAllocator + +-- > ORT_API2_STATUS(CreateAllocator, _In_ const OrtSession* session, _In_ const OrtMemoryInfo* mem_info, _Outptr_ OrtAllocator** out); + +-- TODO: unimplemented + +-------------------------------------------------------------------------------- +-- OrtApi::RunWithBinding + +-- > ORT_API2_STATUS(RunWithBinding, _Inout_ OrtSession* session, _In_ const OrtRunOptions* run_options, _In_ const OrtIoBinding* binding_ptr); + +-- TODO: unimplemented + +-------------------------------------------------------------------------------- +-- OrtApi::CreateIoBinding + +-- > ORT_API2_STATUS(CreateIoBinding, _Inout_ OrtSession* session, _Outptr_ OrtIoBinding** out); + +-- TODO: unimplemented + +-------------------------------------------------------------------------------- +-- OrtApi::BindInput + +-- > ORT_API2_STATUS(BindInput, _Inout_ OrtIoBinding* binding_ptr, _In_ const char* name, _In_ const OrtValue* val_ptr); + +-- TODO: unimplemented + +-------------------------------------------------------------------------------- +-- OrtApi::BindOutput + +-- > ORT_API2_STATUS(BindOutput, _Inout_ OrtIoBinding* binding_ptr, _In_ const char* name, _In_ const OrtValue* val_ptr); + +-- TODO: unimplemented + +-------------------------------------------------------------------------------- +-- OrtApi::BindOutputToDevice + +-- > ORT_API2_STATUS(BindOutputToDevice, _Inout_ OrtIoBinding* binding_ptr, _In_ const char* name, _In_ const OrtMemoryInfo* mem_info_ptr); + +-- TODO: unimplemented + +-------------------------------------------------------------------------------- +-- OrtApi::GetBoundOutputNames + +-- > ORT_API2_STATUS(GetBoundOutputNames, _In_ const OrtIoBinding* binding_ptr, _In_ OrtAllocator* allocator, _Out_ char** buffer, _Out_writes_all_(count) size_t** lengths, _Out_ size_t* count); + +-- TODO: unimplemented + +-------------------------------------------------------------------------------- +-- OrtApi::GetBoundOutputValues + +-- > ORT_API2_STATUS(GetBoundOutputValues, _In_ const OrtIoBinding* binding_ptr, _In_ OrtAllocator* allocator, _Out_writes_all_(output_count) OrtValue*** output, _Out_ size_t* output_count); + +-- TODO: unimplemented + +-------------------------------------------------------------------------------- +-- OrtApi::ClearBoundInputs + +-- > void(ORT_API_CALL* ClearBoundInputs)(_Inout_ OrtIoBinding* binding_ptr) NO_EXCEPTION ORT_ALL_ARGS_NONNULL; + +-- TODO: unimplemented + +-------------------------------------------------------------------------------- +-- OrtApi::ClearBoundOutputs + +-- > void(ORT_API_CALL* ClearBoundOutputs)(_Inout_ OrtIoBinding* binding_ptr) NO_EXCEPTION ORT_ALL_ARGS_NONNULL; + +-- TODO: unimplemented + +-------------------------------------------------------------------------------- +-- OrtApi::TensorAt + +-- > ORT_API2_STATUS(TensorAt, _Inout_ OrtValue* value, const int64_t* location_values, size_t location_values_count, _Outptr_ void** out); + +-- TODO: unimplemented + +-------------------------------------------------------------------------------- +-- OrtApi::CreateAndRegisterAllocator + +-- > ORT_API2_STATUS(CreateAndRegisterAllocator, _Inout_ OrtEnv* env, _In_ const OrtMemoryInfo* mem_info, _In_ const OrtArenaCfg* arena_cfg); + +-- TODO: unimplemented + +-------------------------------------------------------------------------------- +-- OrtApi::SetLanguageProjection + +-- > ORT_API2_STATUS(SetLanguageProjection, _In_ const OrtEnv* ort_env, _In_ OrtLanguageProjection projection); + +-- TODO: unimplemented + +-------------------------------------------------------------------------------- +-- OrtApi::SessionGetProfilingStartTimeNs + +-- > ORT_API2_STATUS(SessionGetProfilingStartTimeNs, _In_ const OrtSession* session, _Outptr_ uint64_t* out); + +-- TODO: unimplemented + +-------------------------------------------------------------------------------- +-- OrtApi::SetGlobalIntraOpNumThreads + +-- > ORT_API2_STATUS(SetGlobalIntraOpNumThreads, _Inout_ OrtThreadingOptions* tp_options, int intra_op_num_threads); + +-- TODO: unimplemented + +-------------------------------------------------------------------------------- +-- OrtApi::SetGlobalInterOpNumThreads + +-- > ORT_API2_STATUS(SetGlobalInterOpNumThreads, _Inout_ OrtThreadingOptions* tp_options, int inter_op_num_threads); + +-- TODO: unimplemented + +-------------------------------------------------------------------------------- +-- OrtApi::SetGlobalSpinControl + +-- > ORT_API2_STATUS(SetGlobalSpinControl, _Inout_ OrtThreadingOptions* tp_options, int allow_spinning); + +-- TODO: unimplemented + +-------------------------------------------------------------------------------- +-- OrtApi::AddInitializer + +-- > ORT_API2_STATUS(AddInitializer, _Inout_ OrtSessionOptions* options, _In_z_ const char* name, _In_ const OrtValue* val); + +-- TODO: unimplemented + +-------------------------------------------------------------------------------- +-- OrtApi::CreateEnvWithCustomLoggerAndGlobalThreadPools + +-- > ORT_API2_STATUS(CreateEnvWithCustomLoggerAndGlobalThreadPools, OrtLoggingFunction logging_function, _In_opt_ void* logger_param, OrtLoggingLevel log_severity_level, _In_ const char* logid, _In_ const struct OrtThreadingOptions* tp_options, _Outptr_ OrtEnv** out); + +-- TODO: unimplemented + +-------------------------------------------------------------------------------- +-- OrtApi::SessionOptionsAppendExecutionProvider_CUDA + +-- > ORT_API2_STATUS(SessionOptionsAppendExecutionProvider_CUDA, _In_ OrtSessionOptions* options, _In_ const OrtCUDAProviderOptions* cuda_options); + +-- TODO: unimplemented + +-------------------------------------------------------------------------------- +-- OrtApi::SessionOptionsAppendExecutionProvider_ROCM + +-- > ORT_API2_STATUS(SessionOptionsAppendExecutionProvider_ROCM, _In_ OrtSessionOptions* options, _In_ const OrtROCMProviderOptions* rocm_options); + +-- TODO: unimplemented + +-------------------------------------------------------------------------------- +-- OrtApi::SessionOptionsAppendExecutionProvider_OpenVINO + +-- > ORT_API2_STATUS(SessionOptionsAppendExecutionProvider_OpenVINO, _In_ OrtSessionOptions* options, _In_ const OrtOpenVINOProviderOptions* provider_options); + +-- TODO: unimplemented + +-------------------------------------------------------------------------------- +-- OrtApi::SetGlobalDenormalAsZero + +-- > ORT_API2_STATUS(SetGlobalDenormalAsZero, _Inout_ OrtThreadingOptions* tp_options); + +-- TODO: unimplemented + +-------------------------------------------------------------------------------- +-- OrtApi::CreateArenaCfg + +-- > ORT_API2_STATUS(CreateArenaCfg, _In_ size_t max_mem, int arena_extend_strategy, int initial_chunk_size_bytes, int max_dead_bytes_per_chunk, _Outptr_ OrtArenaCfg** out); + +-- TODO: unimplemented + +-------------------------------------------------------------------------------- +-- OrtApi::ModelMetadataGetGraphDescription + +-- > ORT_API2_STATUS(ModelMetadataGetGraphDescription, _In_ const OrtModelMetadata* model_metadata, _Inout_ OrtAllocator* allocator, _Outptr_ char** value); + +-- TODO: unimplemented + +-------------------------------------------------------------------------------- +-- OrtApi::SessionOptionsAppendExecutionProvider_TensorRT + +-- > ORT_API2_STATUS(SessionOptionsAppendExecutionProvider_TensorRT, _In_ OrtSessionOptions* options, _In_ const OrtTensorRTProviderOptions* tensorrt_options); + +-- TODO: unimplemented + +-------------------------------------------------------------------------------- +-- OrtApi::SetCurrentGpuDeviceId + +-- > ORT_API2_STATUS(SetCurrentGpuDeviceId, _In_ int device_id); + +-- TODO: unimplemented + +-------------------------------------------------------------------------------- +-- OrtApi::GetCurrentGpuDeviceId + +-- > ORT_API2_STATUS(GetCurrentGpuDeviceId, _In_ int* device_id); + +-- TODO: unimplemented + +-------------------------------------------------------------------------------- +-- OrtApi::KernelInfoGetAttributeArray_float + +-- > ORT_API2_STATUS(KernelInfoGetAttributeArray_float, _In_ const OrtKernelInfo* info, _In_ const char* name, _Out_ float* out, _Inout_ size_t* size); + +-- TODO: unimplemented + +-------------------------------------------------------------------------------- +-- OrtApi::KernelInfoGetAttributeArray_int64 + +-- > ORT_API2_STATUS(KernelInfoGetAttributeArray_int64, _In_ const OrtKernelInfo* info, _In_ const char* name, _Out_ int64_t* out, _Inout_ size_t* size); + +-- TODO: unimplemented + +-------------------------------------------------------------------------------- +-- OrtApi::CreateArenaCfgV2 + +-- > ORT_API2_STATUS(CreateArenaCfgV2, _In_reads_(num_keys) const char* const* arena_config_keys, _In_reads_(num_keys) const size_t* arena_config_values, _In_ size_t num_keys, _Outptr_ OrtArenaCfg** out); + +-- TODO: unimplemented + +-------------------------------------------------------------------------------- +-- OrtApi::AddRunConfigEntry + +-- > ORT_API2_STATUS(AddRunConfigEntry, _Inout_ OrtRunOptions* options, _In_z_ const char* config_key, _In_z_ const char* config_value); + +-- TODO: unimplemented + +-------------------------------------------------------------------------------- +-- OrtApi::CreatePrepackedWeightsContainer + +-- > ORT_API2_STATUS(CreatePrepackedWeightsContainer, _Outptr_ OrtPrepackedWeightsContainer** out); + +-- TODO: unimplemented + +-------------------------------------------------------------------------------- +-- OrtApi::CreateSessionWithPrepackedWeightsContainer + +-- > ORT_API2_STATUS(CreateSessionWithPrepackedWeightsContainer, _In_ const OrtEnv* env, _In_ const ORTCHAR_T* model_path, _In_ const OrtSessionOptions* options, _Inout_ OrtPrepackedWeightsContainer* prepacked_weights_container, _Outptr_ OrtSession** out); + +-- TODO: unimplemented + +-------------------------------------------------------------------------------- +-- OrtApi::CreateSessionFromArrayWithPrepackedWeightsContainer + +-- > ORT_API2_STATUS(CreateSessionFromArrayWithPrepackedWeightsContainer, _In_ const OrtEnv* env, _In_ const void* model_data, size_t model_data_length, _In_ const OrtSessionOptions* options, _Inout_ OrtPrepackedWeightsContainer* prepacked_weights_container, _Outptr_ OrtSession** out); + +-- TODO: unimplemented + +-------------------------------------------------------------------------------- +-- OrtApi::SessionOptionsAppendExecutionProvider_TensorRT_V2 + +-- > ORT_API2_STATUS(SessionOptionsAppendExecutionProvider_TensorRT_V2, _In_ OrtSessionOptions* options, _In_ const OrtTensorRTProviderOptionsV2* tensorrt_options); + +-- TODO: unimplemented + +-------------------------------------------------------------------------------- +-- OrtApi::CreateTensorRTProviderOptions + +-- > ORT_API2_STATUS(CreateTensorRTProviderOptions, _Outptr_ OrtTensorRTProviderOptionsV2** out); + +-- TODO: unimplemented + +-------------------------------------------------------------------------------- +-- OrtApi::UpdateTensorRTProviderOptions + +-- > ORT_API2_STATUS(UpdateTensorRTProviderOptions, _Inout_ OrtTensorRTProviderOptionsV2* tensorrt_options, _In_reads_(num_keys) const char* const* provider_options_keys, _In_reads_(num_keys) const char* const* provider_options_values, _In_ size_t num_keys); + +-- TODO: unimplemented + +-------------------------------------------------------------------------------- +-- OrtApi::GetTensorRTProviderOptionsAsString + +-- > ORT_API2_STATUS(GetTensorRTProviderOptionsAsString, _In_ const OrtTensorRTProviderOptionsV2* tensorrt_options, _Inout_ OrtAllocator* allocator, _Outptr_ char** ptr); + +-- TODO: unimplemented + +-------------------------------------------------------------------------------- +-- OrtApi::ReleaseTensorRTProviderOptions + +-- > void(ORT_API_CALL* ReleaseTensorRTProviderOptions)(_Frees_ptr_opt_ OrtTensorRTProviderOptionsV2* input); + +-- TODO: unimplemented + +-------------------------------------------------------------------------------- +-- OrtApi::EnableOrtCustomOps + +-- > ORT_API2_STATUS(EnableOrtCustomOps, _Inout_ OrtSessionOptions* options); + +-- TODO: unimplemented + +-------------------------------------------------------------------------------- +-- OrtApi::RegisterAllocator + +-- > ORT_API2_STATUS(RegisterAllocator, _Inout_ OrtEnv* env, _In_ OrtAllocator* allocator); + +-- TODO: unimplemented + +-------------------------------------------------------------------------------- +-- OrtApi::UnregisterAllocator + +-- > ORT_API2_STATUS(UnregisterAllocator, _Inout_ OrtEnv* env, _In_ const OrtMemoryInfo* mem_info); + +-- TODO: unimplemented + +-------------------------------------------------------------------------------- +-- OrtApi::IsSparseTensor + +-- > ORT_API2_STATUS(IsSparseTensor, _In_ const OrtValue* value, _Out_ int* out); + +-- TODO: unimplemented + +-------------------------------------------------------------------------------- +-- OrtApi::CreateSparseTensorAsOrtValue + +-- > ORT_API2_STATUS(CreateSparseTensorAsOrtValue, _Inout_ OrtAllocator* allocator, _In_ const int64_t* dense_shape, size_t dense_shape_len, ONNXTensorElementDataType type, _Outptr_ OrtValue** out); + +-- TODO: unimplemented + +-------------------------------------------------------------------------------- +-- OrtApi::FillSparseTensorCoo + +-- > ORT_API2_STATUS(FillSparseTensorCoo, _Inout_ OrtValue* ort_value, _In_ const OrtMemoryInfo* data_mem_info, _In_ const int64_t* values_shape, size_t values_shape_len, _In_ const void* values, _In_ const int64_t* indices_data, size_t indices_num); + +-- TODO: unimplemented + +-------------------------------------------------------------------------------- +-- OrtApi::FillSparseTensorCsr + +-- > ORT_API2_STATUS(FillSparseTensorCsr, _Inout_ OrtValue* ort_value, _In_ const OrtMemoryInfo* data_mem_info, _In_ const int64_t* values_shape, size_t values_shape_len, _In_ const void* values, _In_ const int64_t* inner_indices_data, size_t inner_indices_num, _In_ const int64_t* outer_indices_data, size_t outer_indices_num); + +-- TODO: unimplemented + +-------------------------------------------------------------------------------- +-- OrtApi::FillSparseTensorBlockSparse + +-- > ORT_API2_STATUS(FillSparseTensorBlockSparse, _Inout_ OrtValue* ort_value, _In_ const OrtMemoryInfo* data_mem_info, _In_ const int64_t* values_shape, size_t values_shape_len, _In_ const void* values, _In_ const int64_t* indices_shape_data, size_t indices_shape_len, _In_ const int32_t* indices_data); + +-- TODO: unimplemented + +-------------------------------------------------------------------------------- +-- OrtApi::CreateSparseTensorWithValuesAsOrtValue + +-- > ORT_API2_STATUS(CreateSparseTensorWithValuesAsOrtValue, _In_ const OrtMemoryInfo* info, _Inout_ void* p_data, _In_ const int64_t* dense_shape, size_t dense_shape_len, _In_ const int64_t* values_shape, size_t values_shape_len, ONNXTensorElementDataType type, _Outptr_ OrtValue** out); + +-- TODO: unimplemented + +-------------------------------------------------------------------------------- +-- OrtApi::UseCooIndices + +-- > ORT_API2_STATUS(UseCooIndices, _Inout_ OrtValue* ort_value, _Inout_ int64_t* indices_data, size_t indices_num); + +-- TODO: unimplemented + +-------------------------------------------------------------------------------- +-- OrtApi::UseCsrIndices + +-- > ORT_API2_STATUS(UseCsrIndices, _Inout_ OrtValue* ort_value, _Inout_ int64_t* inner_data, size_t inner_num, _Inout_ int64_t* outer_data, size_t outer_num); + +-- TODO: unimplemented + +-------------------------------------------------------------------------------- +-- OrtApi::UseBlockSparseIndices + +-- > ORT_API2_STATUS(UseBlockSparseIndices, _Inout_ OrtValue* ort_value, const int64_t* indices_shape, size_t indices_shape_len, _Inout_ int32_t* indices_data); + +-- TODO: unimplemented + +-------------------------------------------------------------------------------- +-- OrtApi::GetSparseTensorFormat + +-- > ORT_API2_STATUS(GetSparseTensorFormat, _In_ const OrtValue* ort_value, _Out_ enum OrtSparseFormat* out); + +-- TODO: unimplemented + +-------------------------------------------------------------------------------- +-- OrtApi::GetSparseTensorValuesTypeAndShape + +-- > ORT_API2_STATUS(GetSparseTensorValuesTypeAndShape, _In_ const OrtValue* ort_value, _Outptr_ OrtTensorTypeAndShapeInfo** out); + +-- TODO: unimplemented + +-------------------------------------------------------------------------------- +-- OrtApi::GetSparseTensorValues + +-- > ORT_API2_STATUS(GetSparseTensorValues, _In_ const OrtValue* ort_value, _Outptr_ const void** out); + +-- TODO: unimplemented + +-------------------------------------------------------------------------------- +-- OrtApi::GetSparseTensorIndicesTypeShape + +-- > ORT_API2_STATUS(GetSparseTensorIndicesTypeShape, _In_ const OrtValue* ort_value, enum OrtSparseIndicesFormat indices_format, _Outptr_ OrtTensorTypeAndShapeInfo** out); + +-- TODO: unimplemented + +-------------------------------------------------------------------------------- +-- OrtApi::GetSparseTensorIndices + +-- > ORT_API2_STATUS(GetSparseTensorIndices, _In_ const OrtValue* ort_value, enum OrtSparseIndicesFormat indices_format, _Out_ size_t* num_indices, _Outptr_ const void** indices); + +-- TODO: unimplemented + +-------------------------------------------------------------------------------- +-- OrtApi::HasValue + +-- > ORT_API2_STATUS(HasValue, _In_ const OrtValue* value, _Out_ int* out); + +-- TODO: unimplemented + +-------------------------------------------------------------------------------- +-- OrtApi::KernelContext_GetGPUComputeStream + +-- > ORT_API2_STATUS(KernelContext_GetGPUComputeStream, _In_ const OrtKernelContext* context, _Outptr_ void** out); + +-- TODO: unimplemented + +-------------------------------------------------------------------------------- +-- OrtApi::GetTensorMemoryInfo + +-- > ORT_API2_STATUS(GetTensorMemoryInfo, _In_ const OrtValue* value, _Out_ const OrtMemoryInfo** mem_info); + +-- TODO: unimplemented + +-------------------------------------------------------------------------------- +-- OrtApi::GetExecutionProviderApi + +-- > ORT_API2_STATUS(GetExecutionProviderApi, _In_ const char* provider_name, _In_ uint32_t version, _Outptr_ const void** provider_api); + +-- TODO: unimplemented + +-------------------------------------------------------------------------------- +-- OrtApi::SessionOptionsSetCustomCreateThreadFn + +-- > ORT_API2_STATUS(SessionOptionsSetCustomCreateThreadFn, _Inout_ OrtSessionOptions* options, _In_ OrtCustomCreateThreadFn ort_custom_create_thread_fn); + +-- TODO: unimplemented + +-------------------------------------------------------------------------------- +-- OrtApi::SessionOptionsSetCustomThreadCreationOptions + +-- > ORT_API2_STATUS(SessionOptionsSetCustomThreadCreationOptions, _Inout_ OrtSessionOptions* options, _In_ void* ort_custom_thread_creation_options); + +-- TODO: unimplemented + +-------------------------------------------------------------------------------- +-- OrtApi::SessionOptionsSetCustomJoinThreadFn + +-- > ORT_API2_STATUS(SessionOptionsSetCustomJoinThreadFn, _Inout_ OrtSessionOptions* options, _In_ OrtCustomJoinThreadFn ort_custom_join_thread_fn); + +-- TODO: unimplemented + +-------------------------------------------------------------------------------- +-- OrtApi::SetGlobalCustomCreateThreadFn + +-- > ORT_API2_STATUS(SetGlobalCustomCreateThreadFn, _Inout_ OrtThreadingOptions* tp_options, _In_ OrtCustomCreateThreadFn ort_custom_create_thread_fn); + +-- TODO: unimplemented + +-------------------------------------------------------------------------------- +-- OrtApi::SetGlobalCustomThreadCreationOptions + +-- > ORT_API2_STATUS(SetGlobalCustomThreadCreationOptions, _Inout_ OrtThreadingOptions* tp_options, _In_ void* ort_custom_thread_creation_options); + +-- TODO: unimplemented + +-------------------------------------------------------------------------------- +-- OrtApi::SetGlobalCustomJoinThreadFn + +-- > ORT_API2_STATUS(SetGlobalCustomJoinThreadFn, _Inout_ OrtThreadingOptions* tp_options, _In_ OrtCustomJoinThreadFn ort_custom_join_thread_fn); + +-- TODO: unimplemented + +-------------------------------------------------------------------------------- +-- OrtApi::SynchronizeBoundInputs + +-- > ORT_API2_STATUS(SynchronizeBoundInputs, _Inout_ OrtIoBinding* binding_ptr); + +-- TODO: unimplemented + +-------------------------------------------------------------------------------- +-- OrtApi::SynchronizeBoundOutputs + +-- > ORT_API2_STATUS(SynchronizeBoundOutputs, _Inout_ OrtIoBinding* binding_ptr); + +-- TODO: unimplemented + +-------------------------------------------------------------------------------- +-- OrtApi::SessionOptionsAppendExecutionProvider_CUDA_V2 + +-- > ORT_API2_STATUS(SessionOptionsAppendExecutionProvider_CUDA_V2, _In_ OrtSessionOptions* options, _In_ const OrtCUDAProviderOptionsV2* cuda_options); + +-- TODO: unimplemented + +-------------------------------------------------------------------------------- +-- OrtApi::CreateCUDAProviderOptions + +-- > ORT_API2_STATUS(CreateCUDAProviderOptions, _Outptr_ OrtCUDAProviderOptionsV2** out); + +-- TODO: unimplemented + +-------------------------------------------------------------------------------- +-- OrtApi::UpdateCUDAProviderOptions + +-- > ORT_API2_STATUS(UpdateCUDAProviderOptions, _Inout_ OrtCUDAProviderOptionsV2* cuda_options, _In_reads_(num_keys) const char* const* provider_options_keys, _In_reads_(num_keys) const char* const* provider_options_values, _In_ size_t num_keys); + +-- TODO: unimplemented + +-------------------------------------------------------------------------------- +-- OrtApi::GetCUDAProviderOptionsAsString + +-- > ORT_API2_STATUS(GetCUDAProviderOptionsAsString, _In_ const OrtCUDAProviderOptionsV2* cuda_options, _Inout_ OrtAllocator* allocator, _Outptr_ char** ptr); + +-- TODO: unimplemented + +-------------------------------------------------------------------------------- +-- OrtApi::ReleaseCUDAProviderOptions + +-- > void(ORT_API_CALL* ReleaseCUDAProviderOptions)(_Frees_ptr_opt_ OrtCUDAProviderOptionsV2* input); + +-- TODO: unimplemented + +-------------------------------------------------------------------------------- +-- OrtApi::SessionOptionsAppendExecutionProvider_MIGraphX + +-- > ORT_API2_STATUS(SessionOptionsAppendExecutionProvider_MIGraphX, _In_ OrtSessionOptions* options, _In_ const OrtMIGraphXProviderOptions* migraphx_options); + +-- TODO: unimplemented + +-------------------------------------------------------------------------------- +-- OrtApi::AddExternalInitializers + +-- > ORT_API2_STATUS(AddExternalInitializers, _In_ OrtSessionOptions* options, _In_reads_(num_initializers) const char* const* initializer_names, _In_reads_(num_initializers) const OrtValue* const* initializers, size_t num_initializers); + +-- TODO: unimplemented + +-------------------------------------------------------------------------------- +-- OrtApi::CreateOpAttr + +-- > ORT_API2_STATUS(CreateOpAttr, _In_ const char* name, _In_ const void* data, _In_ int len, _In_ OrtOpAttrType type, _Outptr_ OrtOpAttr** op_attr); + +-- TODO: unimplemented + +-------------------------------------------------------------------------------- +-- OrtApi::CreateOp + +-- > ORT_API2_STATUS(CreateOp, _In_ const OrtKernelInfo* info, _In_z_ const char* op_name, _In_z_ const char* domain, int version, _In_reads_(type_constraint_count) const char** type_constraint_names, _In_reads_(type_constraint_count) const ONNXTensorElementDataType* type_constraint_values, int type_constraint_count, _In_reads_(attr_count) const OrtOpAttr* const* attr_values, int attr_count, int input_count, int output_count, _Outptr_ OrtOp** ort_op); + +-- TODO: unimplemented + +-------------------------------------------------------------------------------- +-- OrtApi::InvokeOp + +-- > ORT_API2_STATUS(InvokeOp, _In_ const OrtKernelContext* context, _In_ const OrtOp* ort_op, _In_ const OrtValue* const* input_values, _In_ int input_count, _Inout_ OrtValue* const* output_values, _In_ int output_count); + +-- TODO: unimplemented + +-------------------------------------------------------------------------------- +-- OrtApi::SessionOptionsAppendExecutionProvider + +-- > ORT_API2_STATUS(SessionOptionsAppendExecutionProvider, _In_ OrtSessionOptions* options, _In_ const char* provider_name, _In_reads_(num_keys) const char* const* provider_options_keys, _In_reads_(num_keys) const char* const* provider_options_values, _In_ size_t num_keys); + +-- TODO: unimplemented + +-------------------------------------------------------------------------------- +-- OrtApi::CopyKernelInfo + +-- > ORT_API2_STATUS(CopyKernelInfo, _In_ const OrtKernelInfo* info, _Outptr_ OrtKernelInfo** info_copy); + +-- TODO: unimplemented + +-- ... and many more +-- ... genuinely, this is only about halfway through the header file