FSU: Gstreamer made simple!

On August 4, 2010, in Development, Farsight, by kakaroto

Hi everyone!

Many of you probably know how to use GStreamer, and many others probably know how complicated it can be to use GStreamer.

When you want to build complex pipelines and mostly dynamic pipelines, it can get pretty complicated and will usually require a huge amount of code to properly build or modify your pipeline without killing yourself in the process.

I’ve been working in the last few months on a subproject of Farsight that I called “Farsight-utils” (for now) which consists of a library that will help you build complex GStreamer pipelines very very easily.

It was mainly written for VoIP applications that want to concentrate on the UI rather than the GStreamer API, but it can be used for pretty much anything else (Cheese or PiTiVi for example). The API is really simple and extensible. FSU (Farsight-Utils) is made of three layers, the first layer is what this post is all about, it’s a set of classes (FsuFilter(s) and FsuFilterManager) which build the pipelines for you while giving you a simple API to do it. The second layer is a set of classes which simply wrap the Farsight objects and take care of the GStreamer part of things (linking the Farsight element with the source/sink, etc..), and provide access to a filter manager for the session and streams. The third and last part of FSU is a set of higher level classes that take care of the signaling part of Farsight for you and allow you to do simple audio/video calls with a very minimalistic API. But let’s talk about the FilterManager and the filters and the best way to explain it is to start with a simple use case :

The story is simple, I was talking to Olivier Crete (Farsight developer)  about FSU, then we talked about how iChat has this video effects grid that allows you to preview different effects and choose an effect to apply to your video, so I thought “how hard can it be”, and I started hacking on a simple example to do that. It took me 10 minutes to write (literally) and less than 150 lines of code (no, not python, yes, C code). Let’s show you a screencast of it first :

Or for people without flash, you can download the video here.

So, as you can see, it works! and it’s really nice! Also, thanks to the latest git version of GStreamer, the dynamic changes to the pipeline are quite fast as you probably noticed! I won’t bore you with the main (which is about 25 lines of C code) which builds the whole pipeline, but I’d like to show you how the magic is done when you click/choose an effect :

  1. static void
  2. button_clicked (GtkButton *button, gpointer user_data)
  3. {
  4.   static FsuFilterId *effect_id = NULL;
  5.   static gchar *previous_filename = NULL;
  6.   gchar *filename = user_data;
  7.   FsuFilterManager *preview_manager = NULL;
  8.   FsuFilter *filter = FSU_FILTER (fsu_gnome_effect_filter_new (filename, NULL));
  9.  
  10.   g_object_get (preview, "filter-manager", &preview_manager, NULL);
  11.  
  12.   if (effect_id == NULL) {
  13.     /* Add the filter if there are none */
  14.     effect_id = fsu_filter_manager_append_filter (preview_manager, filter);
  15.     previous_filename = filename;
  16.   } else if (!strcmp (filename, previous_filename)) {
  17.     /* Remove the existing effect if it was clicked twice */
  18.     fsu_filter_manager_remove_filter (preview_manager, effect_id);
  19.     effect_id = NULL;
  20.     filename = NULL;
  21.   } else {
  22.     /* Replace the existing effect with the new one*/
  23.     effect_id = fsu_filter_manager_replace_filter (preview_manager, filter,
  24.         effect_id);
  25.   }
  26.  
  27.   previous_filename = filename;
  28.   g_object_unref (filter);
  29.   g_object_unref (preview_manager);
  30. }

I think this is pretty straightforward so I won’t try to explain it to you, however you probably notice that there is no error checking, or pad blocking, etc.. because the FilterManager takes care of everything for us, it will do a pad_block, then find where in the pipeline it should do the change, unlink, apply the filter and relink. All of this asynchronously! And if a filter fails to apply (a non-existing element or some caps issue), then the filter manager will send you a signal to let you know what happened, and it will just ignore that specific filter and revert the change it did (if any) and keep your pipeline in a coherent/working state.

As you might have noticed in the code, the filter I used is called FsuGnomeEffectFilter. This filter takes a filename to a .effect file from the recently created gnome-video-effects package and creates the proper pipeline from it, as well as providing us with the name and categories of the effect. It also has an API to list all the effects :

  1.   GList *effects = fsu_gnome_effect_list_effects ("/usr/share/gnome-video-effects");

Using the filter manager is also quite easy, all you have to do is to ‘apply’ it between the source and the sink, then we can add all sorts of FsuFilters to the FilterManager, like the resolution filter, the gnome-effect filter or the preview filter (to which you give a window x-id as argument in the ‘new’ and you don’t have to bother with listening to the proper GstMessage on the Bus, etc..). To ‘apply’ the filter manager, just do :

  1.   sink_pad = fsu_filter_manager_apply (filters, GST_BIN (pipeline), sink_pad);
  2.   gst_pad_link (src_pad, sink_pad);

You can also just append a level and volume filter to your audio pipeline and get an API to control the volume and receive a gobject signal (not a GstMessage on the bus) whenever the sound level changes, so you can really start concentrating on making a beautiful UI that uses/shows all these features to your users!

I think that’s it for now, I will let everyone digest everything, and I’ll post something soon to show the progress and explain a bit more on the API!

In the meantime, you can try that demo app (from the screencast) from my ‘fsu’ branch of Farsight git. The example is in examples/filter-manager/grid-effects2.c.

Enjoy!

Tagged with:  

4 Responses to FSU: Gstreamer made simple!

  1. Brett says:

    Is this being integrated into Cheese by any chance?

    • kakaroto says:

      Not at the moment as far as I know. Simply because Cheese already does everything I do, only it does it with a lot more code. The good news though is that it will probably soon be merged into GStreamer itself (and out of Farsight since it’s completely independent) so it will have a lot more visibility, and people can use like (like Cheese) without depending on Farsight.

  2. […] you remember my last post about Fsu, I talked about how you could use the FsuFilterManager and FsuFilter  classes to create some […]