Using FFI to control VLC from Pharo

VLC is the best-known software for playing audio and video files. More than files, it also allows to play streams, and therefore to play, for example, the stream broadcasted by security cameras. We present here how to use Pharo to interface with the library developed for VLC and thus control VLC from Pharo.

Let's dream

When I'm working, I often want to listen to music. But my music is stored in different places: locally on my laptop, on my personal server, on YouTube, and on premium online music accounts. While I can switch between systems easily, I was dreaming of a music manager where everything would be combined in one place. Let's not lie, it was also the perfect argument to do a little of Pharo.

Playing song with Pharo

Pharo already allows one to play music directly via a plugin integrated into the virtual machine (VM). Unfortunately, this plugin is old and does not guarantee to play the hundreds of music formats that exist today. On the other hand, VLC can play all these audio and video formats. In order to be able to play music controlled by Pharo, you just need to manipulate VLC. And this is what uFFI will allow us to do! Indeed, the VLC project is supported on the three most used platforms (Linux, Windows, and OSX) and also provides on all these platforms the shared libraries libvlc and libvlccore.

A shared library is intended to be used by other programs. They define an API allowing a program to interact with another. And this is precisely what uFFI allows to do: call from Pharo the methods defined in a shared library.

Set-up FFI

Setting up FFI has never been so easy. There are only a few pre-requisites:

Install the project you want to use (for me it's VLC),
Make sure that the shared libraries are in the PATH of the system,
And that's it for the first step!

Once the project is installed, we move on to development. First of all, we create a class that serves as a link between Pharo and VLC. Pharo comes with a complete framework to set up FFI, so we just have to create a subclass of FFILibrary (VLCLibrary) and to indicate the name of the external libraries according to the system we use (.so on Linux, .dll on Windows and .dylib on OSX). We have finished setting up FFI!

1VLCLibrary>>#unix64LibraryName
2    #('/usr/lib/i386-linux-gnu' '/usr/lib32' '/usr/lib') , ((OSEnvironment current at: 'LD_LIBRARY_PATH' ifAbsent: [ '' ]) substrings: ':')
3    do: [ :path |
4        | libraryPath |
5        libraryPath := path asFileReference / 'libvlc.so'.
6        libraryPath exists ifTrue: [ ^ libraryPath fullName ] ].
7    self error: 'Cannot locate vlc library. Please check if it installed on your system'

Functions mapping

Once the interfacing mechanism between Pharo and VLC is in place, we will now be able to map the VLC functions. To be able to use a function defined in the library, we declare in our VLCLibrary class the methods we are going to use. To know the functions, we have two options, use tools that will extract the available functions from the libraries or... use the documentation! Fortunately for us, the VLC libraries are well documented, which greatly simplifies the work. By accessing the documentation of the libraries we have access to the signature of the available functions. We just have to copy and paste the signature of the functions in Pharo to be able to use them. Primitive types are understood by Pharo while we will cleverly replace structures by void * in order to manipulate them as opaque objects.

1VLCLibrary>>#createVLCInstance
2    ^ self ffiCall: 'void * libvlc_new()'

A first test

Now we present the first methods we had to implement to use VLC. First of all, libvlc_new allows one to create an instance of VLC. We will then use this instance to script VLC.

Call to method libvlc_new()

Next, we developed the code necessary to play a music hosted on our computer. For this, we need three methods: creation of a media, creation of the corresponding media player and the play method of the media player. The three methods are described in the documentation as: libvlc_media_new_path, libvlc_media_player_new_from_media and libvlc_media_player_play. For their implementation, we just have to copy their signatures and create methods in the VLCLibrary class as we did for the new.

1VLCLibrary>>#mediaNew: aVLCInstance path: aStringPath
2    "Create a media for a certain file path."
3    ^ self ffiCall: void * libvlc_media_new_path(void * aVLCInstance, String aStringPath)'

1VLCLibrary>>#mediaPlayerPlay: aMediaPlayer
2    "Play"
3    ^ self ffiCall: 'int libvlc_media_player_play(void * aMediaPlayer)'

Finally, some lines in the playground allow us to script VLC to play music.

1vlc := VLCLibrary uniqueInstance createVLCInstance.
2"do not use accentuated characters for the path"
3media := VLCLibrary uniqueInstance mediaNew: vlc path: '/my/file/path.mp3'.
4mediaPlayer := VLCLibrary uniqueInstance mediaPlayerNewFromMedia: media.
5VLCLibrary uniqueInstance mediaPlayerPlay: mediaPlayer.

Structures mapping - Let's dev in Pharo

By writing all the public methods of the VLC libraries in Pharo, it is possible to script VLC completely. However, we get a single abstraction with lots of methods when it would be much better to create objects that represent each aspect of VLC.

It is thus possible to improve our mapping to a VLC library by mapping C structures to Pharo objects. Once again, we will be able to do this thanks to VLC. Structures can be divided into four categories: enums, callbacks, C structures, and opaque structures. FFI offers a solution for mapping each of them.

For enumerations, we need to extend the Pharo class FFIExternalEnumeration, then we implement the enumDecl method by writing an array that will contain the name of the enumeration element followed by its value, etc. We execute the method rebuildEnumAccessors on the class, this step will generate accessors to all the values of the enumeration. Finally, we initialize the class. It is now possible to use the enumeration by using: enumeration name + name of one of the elements of the enumeration.

1VLCMediaType class>>#enumDecl
2"self rebuildEnumAccessors"
3    
4    ^ #(libvlc_media_type_unknown 1
5libvlc_media_type_file 2
6libvlc_media_type_directory 3
7libvlc_media_type_disc 4
8libvlc_media_type_stream 5
9libvlc_media_type_playlist 6)

For callbacks, uFFI also comes with an out-of-the-box implementation. This time we need to extend the FFICallback class. We will then define the function signature to which our callback corresponds in C and the Pharo block that will be executed when the callback is called. For the function signature, we define two methods: the first one fnSpec which returns the function signature as we did when mapping the API, and the second one on: which takes a block as a parameter that will be executed when the callback is called.

Callback definition

It is for opaque structures that it is easiest to define an FFI mapping. You just have to extend the Pharo class FFIOpaqueObject. This step allows one to use this structure by considering only its methods. This is a strategy used in C to hide the internal functioning of the structure.

Finally, there remains the case of C structures. As for the previous cases, we extend this time the FFIExternalStructure class. Then, we define on the class side the fieldsDesc method which returns an array containing all the variables of the structure and their types. By executing the rebuildFieldAccessors method on the class, we also create the accessors of these attributes automatically.

1VLCTrackDescription>>#fieldsDesc
2    "self rebuildFieldAccessors"
3    ^ #(int i_id;
4    String psz_name;
5    VLCTrackDescription * p_next;)

C structure mapping

That's it! We can now completely use the VLC library as if it was a Pharo library. We will see in the next part what it allows us to do quickly.

What about the graphical aspect

We will now look at how to quickly create an interface in Pharo that will allow us to control VLC. To do this, we will use the new version of Spec with Pharo 9. The final goal is to create a usable audio player interface for the user. More precisely, we will add an extension to the Pharo inspector in order to be able to watch and control the state of our VLC players. To do this, we will do two things: add an extension and create the user interface of the extension. To add an extension we just need to create a method with the pragma inspectorPresentationOrder:title: which returns the interface we want to create. To define the user interface we use Spec2 (the reference graphical framework since Pharo 8+). We defined for the interface a progress bar that shows us the progress of a piece of music and two buttons start and pause to control the playback. The complete example is available on the GitHub repository of Pharo-LibVLC by loading the "inspector" group of the baseline.

1Metacello new
2  baseline: 'VLC';
3  repository: 'github://badetitou/Pharo-LibVLC';
4  load: 'inspector'

Then it is possible to launch a music and obtain the following interface by inspecting:

1vlc := VLCLibrary uniqueInstance createVLCInstance.
2media := vlc createMediaFromPath: '/path/to/file.mp3'.
3mediaPlayer := vlc createMediaPlayerFromMedia: media.
4mediaPlayer play

Conclusion

We have seen that it is possible to control VLC from Pharo. This way we can play songs from our computer, but also from other services (like YouTube) using VLC's ability to play a remotely read audio stream. We have presented a first interface to control the audio player graphically. So, why not continue in this direction by creating a complete media center in Pharo, either as a desktop application or as a web application with the Seaside web framework.