MindSpore Lite provides lightweight AI inference acceleration capabilities for different hardware devices, enabling intelligent applications and providing end-to-end solutions for developers. It offers development friendly, efficient, and flexible deployment experiences for algorithm engineers and data scientists, helping the AI software and hardware application ecosystem thrive. In the future, MindSpore Lite will work with the MindSpore AI community to enrich the AI software and hardware application ecosystem.
For more details please check out our MindSpore Lite Architecture Guide.
MindSpore Lite achieves double the inference performance for AIGC, speech algorithms, and CV model inference, and has been deployed in Huawei's flagship smartphones for commercial use. As shown in the figure below, MindSpore Lite supports image style transfer and image segmentation for CV algorithms.
-
Compile
MindSpore Lite has multiple different hardware backends, including:
-
For service side devices, users can compile dynamic libraries and Python wheel packages by setting compilation options such as
MSLITE_ENABLE_CLOUD_INFERENCEfor inference of upgrade and CPU hardware. For detailed compilation tutorials, please refer to the official website of MindSpore Lite. -
For end and edge devices, different dynamic libraries can be compiled through different cross compilation toolchains. For detailed compilation tutorials, please refer to the official website of MindSpore Lite.
-
-
Model conversion
MindSpore Lite supports the conversion of models serialized from various AI frameworks such as MindSpore, ONNX, TF, etc. into MindSpore Lite format IR. In order to achieve more efficient model inference, MindSpore Lite supports the conversion of models into
.msformat or.mindirformat, where:-
The
.mindirmodel is used for inference on service side devices and is more compatible with the model structure exported by the MindSpore training framework. It is mainly suitable for Ascend cards and X86/Arm architecture CPU hardware. For detailed conversion methods, please refer to the Conversion Tool Tutorial. -
The
.msmodel is mainly used for inference of end and edge devices, and is mainly suitable for terminal hardware such as Kirin NPU and Arm architecture CPU. In order to better reduce the size of the model file, the.msmodel is serialized and deserialized through protobuffer. For detailed instructions on how to use the conversion tool, please refer to the Conversion Tool
-
-
Model inference
MindSpore Lite provides three APIs: Python, C++, and Java, and complete usage cases for the corresponding APIs:
-
Python API Interface Use Case
-
C/C++ Complete Use Cases
-
Complete Java Use Cases
-
-
Terminal and Cloud one-stop inference deployment
-
Provide end-to-end processes for model transformation optimization, deployment, and inference.
-
The unified IR realizes the device-cloud AI application integration.
-
-
Lightweight
-
Provides model compression, which could help to improve performance as well.
-
Provides the ultra-lightweight reasoning solution MindSpore Lite Micro to meet the deployment requirements in extreme environments such as smart watches and headphones.
-
-
High-performance
-
The built-in high-performance kernel computing library NNACL supports high-performance inference for dedicated chips such as CPU, NNRt, and Ascend, maximizing hardware computing power while minimizing inference latency and power consumption.
-
Assembly code to improve performance of kernel operators. Supports CPU, GPU, and NPU.
-
-
Versatility
-
Supports deployment of multiple hardware such as server-side Ascend and CPU.
-
Supports HarmonyOS and Android mobile operating systems.
-
If you wish to further learn and use MindSpore Lite, please refer to the following content:
-
API documentation:
-
Welcome to Gitee Issues: submit questions, reports, and suggestions;
-
Welcome to Community Forum: engage in technical and problem-solving exchanges;
-
Welcome to Sig: to manage and improve workflow, participate in discussions and exchanges;