Audio Player

[HidenoriMatsubayashi]

Create AudioPlayer plugin

[Taha-Firoz]

I'm trying to get libwinmedia to work, it compiles and works fine on flutter desktop for windows but it doesn't build the so for the elinux builds. Any idea why that's happening

[HidenoriMatsubayashi]

We need to create elinux directory under flutter projects like flutter desktops. video player for elinux will help you.

[Taha-Firoz]

@HidenoriMatsubayashi Yup thats exactly how I started and what I did but the problem is a little different. This project depends on gtk and something called flutter_linux . I don't know why exactly they're required but I don't think they should be for what flutter-elinux is doing. The problem comes in the main CMakeLists.txt file where it checks if the platform is Linux then it should compile with the linux directory, adding a elinux directory isn't helping at all since it never gets referenced because of this.

I was thinking maybe just Port Audio miniaudio as it already provides the multiplatform functionality. This might be unrelated but do you think Canonicals Dbus package will compile with flutter-elinux or does that need a elinux directory too.

[HidenoriMatsubayashi]

Only about flutter support in libwinmedia, it supports flutter desktops for Linux and Windows only. The communication/plugin APIs between Flutter and each platform (e.g. MethodChannel) are different implementations for the same functions. Thus, we need to implement platform porting code to add platform-specific plugins. In this case, we need to add elinux directory under libwinmedia/flutter/ for flutter-elinux and some some plugin interface implementations. Of Course, if you want to use CMakeLists.txt, you need to modify it and other files, too.

About miniaudio. If we use this library in flutter (flutter-elinux), we also need to a plugin interface implementation like video player.

About Canonicals Dbus. I guess we can use it as it is with flutter-elinux because it's just Dart source project (not platform specific code files).

[Taha-Firoz]

@HidenoriMatsubayashi The reason why I asked about Canonicals Dbus was that they're doing ffi calls in their dart code at certain points, like here

  final dylib = DynamicLibrary.open('libc.so.6');
  final getuidP = dylib.lookupFunction<_getuidC, _getuidDart>('getuid');

Would doing these ffi calls provide performance related problems? If you could link to some documentation that helped guide you in writing the video player plugin that would really help. I've written a few Native Node Addons for nodejs to replace FFI calls to an .so file so I do have some idea of what I'm doing however I'm confused about the approach in dart. Canonical has added FFI calls here and there in their code base but other libraries have separate folders for each platform and a lot of C++ code.

Why I pitched the idea of miniaudio is that it'll provide a single header file (though it is 2.8 mb) with no external dependencies while providing compatibility on almost all desktop platforms.

[Taha-Firoz]

I followed the official tutorial for making native bindings but with flutter elinux and platform as elinux

flutter-elinux create --platforms=elinux --template=plugin <pluginname>

[Taha-Firoz]

Any advice on why this is coming up on a clean generated template?

[build] CMake Error at CMakeLists.txt:13 (apply_standard_settings):
[build]   Unknown CMake command "apply_standard_settings".

I saw that you're using a standard template for the video player plugin, except for adding the requirements for pkg_check_modules(GLIB REQUIRED glib-2.0) & pkg_check_modules(GSTREAMER REQUIRED gstreamer-1.0) . I don't have to do that right? I can just link the miniaudio.c directly from a git submodule I hope.

add_library(${PLUGIN_NAME} SHARED
  "ping_plugin.cc"
  "miniaudio/extras/miniaudio_split/miniaudio.c"
)

[HidenoriMatsubayashi]

Would doing these ffi calls provide performance related problems?

No, I don't think so. ffi is a common way to bind a target programming language to other one. Generally, in terms of flutter app performance, the plugins used ffi are better than plugins use Flutter plugin interfaces (APIs).

Any advice on why this is coming up on a clean generated template?

I'm not sure. Is it possible to see your project somewhre?

I don't have to do that right?

Exactly.

[Taha-Firoz]

@HidenoriMatsubayashi yup the clean generated template is here. I haven't added anything custom yet, so I shouldn't be getting any errors?

[Taha-Firoz]

Actually I just checked, the generated template wasn't putting in the appriopriate platform inside of the pubspec.yaml for the plugin. It was

some_platform:
        pluginClass: somePluginClass

instead of

elinux:
        pluginClass: PingPlugin

After changing that the template has been built successfully with the plugin!

[HidenoriMatsubayashi]

That's good. I'm` happy to hear that!

[Taha-Firoz]

@HidenoriMatsubayashi Can you link a guide or reference you used to build the video plugin, I'm having a little trouble figuring out what goes into which file. I'm basically just changing the miniaudio python bindings word for word into dart. Also how do you import the c types like enum values and all when using method channels?

[HidenoriMatsubayashi]

Unfortunetlly, I have no good guides or references, but you need to decide the APIs between Dart (flutter) and platform (miniaudio) using MethodChannel and/or EventChannel. It's just simply async message passing.

Also how do you import the c types like enum values and all when using method channels?

Probably, you need to use map.

e.g. camera plugin:

• dart side: https://github.com/flutter/plugins/blob/master/packages/camera/camera_platform_interface/lib/src/method_channel/method_channel_camera.dart
• platform side: https://github.com/sony/flutter-elinux-plugins/blob/main/packages/camera/elinux/camera_elinux_plugin.cc

[Taha-Firoz]

I see, I've gone through them and from what I can understand is that the generated example is using method calls but these implementations are doing some things differently.

Like here

 //Dart implementation
    static const MethodChannel _channel = MethodChannel('ping');
   ...
  static Future<String?> get platformVersion async {
    final String? version = await _channel.invokeMethod('getPlatformVersion');
    return version;
  }

 //CPP implementation
  if (method_call.method_name().compare("getPlatformVersion") == 0) {
    std::ostringstream version_stream;
    version_stream << "eLinux";
    result->Success(flutter::EncodableValue(version_stream.str()));
  } else {
    result->NotImplemented();
  }

The dart side of the implementation is invoking a specific method on the CPP side and then it gets matched and responded too on the CPP side.

However in the code for the video player things are being done very differently.

//Dart implementation
Future<TextureMessage> create(CreateMessage arg) async {
    final Object encoded = arg.encode();
    const BasicMessageChannel<Object?> channel = BasicMessageChannel<Object?>(
        'dev.flutter.pigeon.VideoPlayerApi.create', StandardMessageCodec());
    final Map<Object?, Object?>? replyMap =
        await channel.send(encoded) as Map<Object?, Object?>?;
    if (replyMap == null) {
      throw PlatformException(
        code: 'channel-error',
        message: 'Unable to establish connection on channel.',
        details: null,
      );
    } else if (replyMap['error'] != null) {
      final Map<Object?, Object?> error =
          replyMap['error'] as Map<Object?, Object?>;
      throw PlatformException(
        code: error['code'] as String,
        message: error['message'] as String?,
        details: error['details'],
      );
    } else {
      return TextureMessage.decode(replyMap['result']!);
    }
  }

Every thing is configured according to different method channels and instead of invocation, encoded data is sent directly to that handler.

There is no matching done since no method was invoked and the logic is implementation on the sent data.

//CPP Implementation
void VideoPlayerPlugin::HandleCreateMethodCall(
    const flutter::EncodableValue& message,
    flutter::MessageReply<flutter::EncodableValue> reply) {
  auto meta = CreateMessage::FromMap(message);
  std::string uri;
  if (!meta.GetAsset().empty()) {
    // todo: gets propery path of the Flutter project.
    std::string flutter_project_path = GetExecutableDirectory() + "/data/";
    uri = flutter_project_path + "flutter_assets/" + meta.GetAsset();
  } else {
    uri = meta.GetUri();
  }
  ....

Can you explain why is the generated template so different from the video player plugin? Were you constrained by however the platform api was coded for the flutter plugin for compatibility reasons? Also which method should I go with, because neither really has any documentation what so ever however this method at least has a couple of examples in this repository.

[HidenoriMatsubayashi]

Can you explain why is the generated template so different from the video player plugin?

Because the generated template is just simply example/templete. Also, it's completely the same with the official Flutter desktop for Linux's one.

Were you constrained by however the platform api was coded for the flutter plugin for compatibility reasons?

What does it mean? The plugin interface/APIs are completely the same as the official flutter desktop for Windows. It means developers can implement their plugins with the same ways of flutter windows. Flutter plugins for Windows platform will help you.

You can use the following cpp plugin APIs.
https://github.com/sony/flutter-embedded-linux/tree/master/src/flutter/shell/platform/common/client_wrapper/include/flutter

[Taha-Firoz]

@HidenoriMatsubayashi okay so I've completed some very rough bindings for now and called it Ping because it's meant to only play simple sounds for notifications etc., not really meant to be a whole audio player. My only question is that the code should be the same for Windows or even Linux due to minaudio but since it's restricted to the elinux directory what am I supposed to do? Right now the only way for me to test it is to run it on an embedded flutter device. The example it broken I'll update it after I've done some testing and fixed up any bugs.

[Taha-Firoz]

Do you know any I'm having this issue, I've added a fPIC flag into the plugins CMake, the plugin compiles fine on x86 but cross compiling gives this error.

'dlopen' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
           /usr/bin/aarch64-linux-gnu-ld:
.../ubuntu18-arm64-sysroot/usr/lib/aarch64-linux-gnu/libm.a(s_sin.o):
           relocation R_AARCH64_ADR_PREL_PG_HI21 against symbol `__stack_chk_guard@@GLIBC_2.17' which may bind externally can not be used when making a
shared
           object; recompile with -fPIC

[makotosato-at]

Hello.

I tried porting audioplayers for elinux.
https://github.com/makotosato-at/flutter-elinux-plugins/tree/add_audioplayers/packages/audioplayers
It seems to be working fine in my environment(i.MX8MP board and RPi4).

[HidenoriMatsubayashi]

Great! If you send the pull request, I'll review it and then submit it.

[HidenoriMatsubayashi]

Closing. Thanks a lot.