🛠Lite.Ai.ToolKit: A lite C++ toolkit of awesome AI models, such as Object Detection, Face Detection, Face Recognition, Segmentation, Matting, etc. See Model Zoo and ONNX Hub, MNN Hub, TNN Hub, NCNN Hub.
News 👇👇
Most of my time now is focused on LLM/VLM Inference. Please check 📖Awesome-LLM-Inference , 📖Awesome-SD-Inference and 📖CUDA-Learn-Notes for more details.
Features 👏👋
- Simply and User friendly. Simply and Consistent syntax like lite::cv::Type::Class, see examples.
- Minimum Dependencies. Only OpenCV and ONNXRuntime are required by default, see build.
- Many Models Supported. 300+ C++ implementations and 500+ weights 👉 Supported-Matrix.
Build 👇👇
Download prebuilt lite.ai.toolkit library from tag/v0.2.0, or just build it from source:
git clone --depth=1 https://github.com/DefTruth/lite.ai.toolkit.git # latest
cd lite.ai.toolkit && sh ./build.sh # >= 0.2.0, support Linux only, tested on Ubuntu 20.04.6 LTS
Quick Start 🌟🌟
Example0: Object Detection using YOLOv5. Download model from Model-Zoo2.
#include "lite/lite.h"
int main(int argc, char *argv[]) {
std::string onnx_path = "yolov5s.onnx";
std::string test_img_path = "test_yolov5.jpg";
std::string save_img_path = "test_results.jpg";
auto *yolov5 = new lite::cv::detection::YoloV5(onnx_path);
std::vector<lite::types::Boxf> detected_boxes;
cv::Mat img_bgr = cv::imread(test_img_path);
yolov5->detect(img_bgr, detected_boxes);
lite::utils::draw_boxes_inplace(img_bgr, detected_boxes);
cv::imwrite(save_img_path, img_bgr);
delete yolov5;
return 0;
}
You can download the prebuilt lite.ai.tooklit library and test resources from tag/v0.2.0.
export LITE_AI_TAG_URL=https://github.com/DefTruth/lite.ai.toolkit/releases/download/v0.2.0
wget ${LITE_AI_TAG_URL}/lite-ort1.17.1+ocv4.9.0+ffmpeg4.2.2-linux-x86_64.tgz
wget ${LITE_AI_TAG_URL}/yolov5s.onnx && wget ${LITE_AI_TAG_URL}/test_yolov5.jpg
Quick Setup 👀
To quickly setup lite.ai.toolkit
, you can follow the CMakeLists.txt
listed as belows. 👇👀
set(lite.ai.toolkit_DIR YOUR-PATH-TO-LITE-INSTALL)
find_package(lite.ai.toolkit REQUIRED PATHS ${lite.ai.toolkit_DIR})
add_executable(lite_yolov5 test_lite_yolov5.cpp)
target_link_libraries(lite_yolov5 ${lite.ai.toolkit_LIBS})
Mixed with MNN or ONNXRuntime 👇👇
The goal of lite.ai.toolkit is not to abstract on top of MNN and ONNXRuntime. So, you can use lite.ai.toolkit mixed with MNN(-DENABLE_MNN=ON, default OFF
) or ONNXRuntime(-DENABLE_ONNXRUNTIME=ON, default ON
). The lite.ai.toolkit installation package contains complete MNN and ONNXRuntime. The workflow may looks like:
#include "lite/lite.h"
// 0. use yolov5 from lite.ai.toolkit to detect objs.
auto *yolov5 = new lite::cv::detection::YoloV5(onnx_path);
// 1. use OnnxRuntime or MNN to implement your own classfier.
interpreter = std::shared_ptr<MNN::Interpreter>(MNN::Interpreter::createFromFile(mnn_path));
// or: session = new Ort::Session(ort_env, onnx_path, session_options);
classfier = interpreter->createSession(schedule_config);
// 2. then, classify the detected objs use your own classfier ...
The included headers of MNN and ONNXRuntime can be found at mnn_config.h and ort_config.h.
🔑️ Check the detailed Quick Start!Click here!
Download resources
You can download the prebuilt lite.ai.tooklit library and test resources from tag/v0.2.0.
export LITE_AI_TAG_URL=https://github.com/DefTruth/lite.ai.toolkit/releases/download/v0.2.0
wget ${LITE_AI_TAG_URL}/lite-ort1.17.1+ocv4.9.0+ffmpeg4.2.2-linux-x86_64.tgz
wget ${LITE_AI_TAG_URL}/yolov5s.onnx && wget ${LITE_AI_TAG_URL}/test_yolov5.jpg
tar -zxvf lite-ort1.17.1+ocv4.9.0+ffmpeg4.2.2-linux-x86_64.tgz
Write test code
write YOLOv5 example codes and name it test_lite_yolov5.cpp
:
#include "lite/lite.h"
int main(int argc, char *argv[]) {
std::string onnx_path = "yolov5s.onnx";
std::string test_img_path = "test_yolov5.jpg";
std::string save_img_path = "test_results.jpg";
auto *yolov5 = new lite::cv::detection::YoloV5(onnx_path);
std::vector<lite::types::Boxf> detected_boxes;
cv::Mat img_bgr = cv::imread(test_img_path);
yolov5->detect(img_bgr, detected_boxes);
lite::utils::draw_boxes_inplace(img_bgr, detected_boxes);
cv::imwrite(save_img_path, img_bgr);
delete yolov5;
return 0;
}
Setup CMakeLists.txt
cmake_minimum_required(VERSION 3.10)
project(lite_yolov5)
set(CMAKE_CXX_STANDARD 17)
set(lite.ai.toolkit_DIR YOUR-PATH-TO-LITE-INSTALL)
find_package(lite.ai.toolkit REQUIRED PATHS ${lite.ai.toolkit_DIR})
if (lite.ai.toolkit_Found)
message(STATUS "lite.ai.toolkit_INCLUDE_DIRS: ${lite.ai.toolkit_INCLUDE_DIRS}")
message(STATUS " lite.ai.toolkit_LIBS: ${lite.ai.toolkit_LIBS}")
message(STATUS " lite.ai.toolkit_LIBS_DIRS: ${lite.ai.toolkit_LIBS_DIRS}")
endif()
add_executable(lite_yolov5 test_lite_yolov5.cpp)
target_link_libraries(lite_yolov5 ${lite.ai.toolkit_LIBS})
Build example
mkdir build && cd build && cmake .. && make -j1
Then, export the lib paths to LD_LIBRARY_PATH
which listed by lite.ai.toolkit_LIBS_DIRS
.
export LD_LIBRARY_PATH=YOUR-PATH-TO-LITE-INSTALL/lib:$LD_LIBRARY_PATH
export LD_LIBRARY_PATH=YOUR-PATH-TO-LITE-INSTALL/third_party/opencv/lib:$LD_LIBRARY_PATH
export LD_LIBRARY_PATH=YOUR-PATH-TO-LITE-INSTALL/third_party/onnxruntime/lib:$LD_LIBRARY_PATH
export LD_LIBRARY_PATH=YOUR-PATH-TO-LITE-INSTALL/third_party/MNN/lib:$LD_LIBRARY_PATH # if -DENABLE_MNN=ON
Run binary:
cp ../yolov5s.onnx ../test_yolov.jpg .
./lite_yolov5
The output logs:
LITEORT_DEBUG LogId: ../examples/hub/onnx/cv/yolov5s.onnx
===============