Skip to content

Latest commit

 

History

History
239 lines (191 loc) · 24.9 KB

README.md

File metadata and controls

239 lines (191 loc) · 24.9 KB
Raw

LibMP: MediaPipe as a Shared Library

Integrate MediaPipe into Your C++ Application

A new MediaPipe build target has been added under /mediapipe/examples/desktop/libmp. Using Bazel, you can build this target to create a shared library (.so on Linux, .dll on Windows), which you can then integrate - along with its .h header file - into any C++ application you wish.

Compilation has been tested on Windows (VS 2022 17.6.16, MSVC 19.36.32546) and Ubuntu 20.04 (GCC 9.4.0).

Enormous thanks to @asprecic for sharing libexample, which made LibMP possible!

Prerequisites

Linux

  • Bazel
  • OpenCV
    • Download pre-compiled binaries: sudo apt-get install libopencv-core-dev libopencv-imgproc-dev libopencv-imgcodecs-dev
    • Alternatively, build from source
  • By default, this repository's Bazel build files expect OpenCV to be version 4 and installed using the apt package manager (i.e., at /usr). To match build settings to your own OpenCV version and path, modify the following Bazel files:
    • /third_party/opencv_linux.BUILD (search for "OPENCV_VERSION")
    • /WORKSPACE (search for "linux_opencv")

Windows

  • NOTE: If using Visual Studio, please use VS 2022 17.6 or above. There is a bug in VS 2022 17.4 and 17.5 that causes MediaPipe to fail to link/compile. See #1 for more details.
  • Bazel
  • OpenCV
  • By default, this repository's Bazel build files expect OpenCV to be version 4.6.0 and installed at C:\opencv. To match build settings to your own OpenCV version and path, modify the following Bazel files:
    • /third_party/opencv_windows.BUILD (search for "OPENCV_VERSION")
    • /WORKSPACE (search for "windows_opencv")

Building

Linux

  1. git clone this repository and cd into it
  2. Run bazel build -c opt --define MEDIAPIPE_DISABLE_GPU=1 mediapipe/examples/desktop/libmp:libmp.so

Windows

  1. git clone this repository and cd into it
  2. Run build_libmp_win.sh (Bash) or build_libmp_win.bat (CMD, Powershell)
    • Edit the Bazel-related environmental variables defined by the script(s) as necessary
    • Bazel requires a Python executable with access to NumPy
      • E.g., create a new conda environment, run conda install numpy, then point Bazel to the environment's Python executable

Usage

  1. The header file that goes with libmp can be found at /mediapipe/examples/desktop/libmp/libmp.h
  2. The library files can be found under /bazel-bin/mediapipe/examples/desktop/libmp as follows:
    • On Linux:
      1. Use the libmp.so shared library in your project
    • On Windows:
      1. Copy libmp.dll to your binary's output location (or otherwise ensure that it is in your PATH)
      2. Use the libmp.dll.if.lib import library in your project
      3. Also copy opencv_world###.dll to your binary's output location (or otherwise ensure it is in your PATH)
  3. In your binary's output location, create a mediapipe directory. Copy the modules and models folders from /bazel-bin/mediapipe into this new directory. (Be sure to copy from /bazel-bin/mediapipe, NOT /mediapipe, as the latter contains empty source directories.)

Notes

  • Unused OpenCV libraries are disabled in /third_party/opencv_linux.BUILD. If you need them, you can re-enable them to get them linked in the final binary too.
  • One recent breaking change to MediaPipe is that models (e.g., .tflite files) are no longer included in the repository itself. They are instead hosted on Google Cloud Storage (GCS). If your code is not working for some reason, you can manually clone an older version of google/mediapipe and copy the source /mediapipe/models and /mediapipe/modules folders over to your binary's location. See cc6a2f7 for the last versions of these folders before deletion.
  • Bazel version 7.0.0 removes apple_common.multi_arch_split, resulting in the error 'apple_common' value has no field or method 'multi_arch_split'. Using Bazel version 6 fixes the issue.
  • An error related to local_execution_config_python may be encountered when building on Windows in CMD using build_libmp_win.bat. This has been fixed by adding BAZEL_SH to the environmental variables and pointing it to MSYS2's bash.exe.

Contributions

Questions, suggestions, issues, pull requests, etc. are all welcome!

layout title nav_order
default home 1

MediaPipe

Attention: Thanks for your interest in MediaPipe! We have moved to https://developers.google.com/mediapipe as the primary developer documentation site for MediaPipe as of April 3, 2023.

This notice and web page will be removed on June 1, 2023.































Live ML anywhere

MediaPipe offers cross-platform, customizable ML solutions for live and streaming media.

accelerated.png cross_platform.png
End-to-End acceleration: Built-in fast ML inference and processing accelerated even on common hardware Build once, deploy anywhere: Unified solution works across Android, iOS, desktop/cloud, web and IoT
ready_to_use.png open_source.png
Ready-to-use solutions: Cutting-edge ML solutions demonstrating full power of the framework Free and open source: Framework and solutions both under Apache 2.0, fully extensible and customizable

ML solutions in MediaPipe

Face Detection Face Mesh Iris Hands Pose Holistic
face_detection face_mesh iris hand pose hair_segmentation
Hair Segmentation Object Detection Box Tracking Instant Motion Tracking Objectron KNIFT
hair_segmentation object_detection box_tracking instant_motion_tracking objectron knift
Android iOS C++ Python JS Coral
Face Detection ✅ ✅ ✅ ✅ ✅ ✅
Face Mesh ✅ ✅ ✅ ✅ ✅
Iris ✅ ✅ ✅
Hands ✅ ✅ ✅ ✅ ✅
Pose ✅ ✅ ✅ ✅ ✅
Holistic ✅ ✅ ✅ ✅ ✅
Selfie Segmentation ✅ ✅ ✅ ✅ ✅
Hair Segmentation ✅ ✅
Object Detection ✅ ✅ ✅ ✅
Box Tracking ✅ ✅ ✅
Instant Motion Tracking ✅
Objectron ✅ ✅ ✅ ✅
KNIFT ✅
AutoFlip ✅
MediaSequence ✅
YouTube 8M ✅

See also MediaPipe Models and Model Cards for ML models released in MediaPipe.

Getting started

To start using MediaPipe solutions with only a few lines code, see example code and demos in MediaPipe in Python and MediaPipe in JavaScript.

To use MediaPipe in C++, Android and iOS, which allow further customization of the solutions as well as building your own, learn how to install MediaPipe and start building example applications in C++, Android and iOS.

The source code is hosted in the MediaPipe Github repository, and you can run code search using Google Open Source Code Search.

Publications

Videos

Events

Community

Alpha disclaimer

MediaPipe is currently in alpha at v0.7. We may be still making breaking API changes and expect to get to stable APIs by v1.0.

Contributing

We welcome contributions. Please follow these guidelines.

We use GitHub issues for tracking requests and bugs. Please post questions to the MediaPipe Stack Overflow with a mediapipe tag.