FluidSynth on embedded platforms, zero boot time?

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

FluidSynth on embedded platforms, zero boot time?

Geoff Plitt
I'm working on a musical instrument that uses FluidSynth for playing SoundFonts, works great. But I'm using Raspberry Pi (Raspbian) and there's a 30-60 second boot time, so I'm looking at other platforms with sub-5-second boot time. I'm really looking to replicate the experience of hardware synths, where you turn it on and it's ready to make sounds in a few seconds at most.

Has anyone had success building/running FluidSynth on an embedded platform with minimal OS, like Arduino, Teensy, Adafruit Feather, etc? Let's say the system has an SD card and DAC/amp to output audio to little speakers, but not a full POSIX environment.

I found https://github.com/divideconcept/FluidLite but it doesn't include audio output and doesn't look maintained.

My other approach might be to strip out unnecessary parts of Raspbian to get a fast boot time, but the documentation I've found still indicates it's hard to get under 10 seconds.

_______________________________________________
fluid-dev mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/fluid-dev
Reply | Threaded
Open this post in threaded view
|

Re: FluidSynth on embedded platforms, zero boot time?

Ceresa Jean-Jacques

Hi,

>so I'm looking at other platforms with sub-5-second boot time. I'm really looking to replicate the experience of hardware synths, where you turn it on and it's ready to make sounds in a few >seconds at most.

One best approch to get very fast boot time would be to search platforms equipped with on board Flash memory, and a serious documentation that explains how to put a custom OS into this memory Flash.

Typically at boot time the loader boot the OS from memory Flash to RAM. The very low access time of memory Flash allows a very fast boot time.

jjc.

 


_______________________________________________
fluid-dev mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/fluid-dev
Reply | Threaded
Open this post in threaded view
|

Re: FluidSynth on embedded platforms, zero boot time?

Carlo Bramini
In reply to this post by Geoff Plitt
Hello!
yes, I succeded to run FluidSynth on an embedded platform.
I have built a digital piano and I used FluidSynth as a rendering engine.

First of all, I would like to suggest you to read this message:

http://lists.nongnu.org/archive/html/fluid-dev/2018-11/msg00022.html

with some results from my experiments.

I suggest you to abandon the idea to use things like Arduino and similar, they are simply not powerful enough. Technically, something like the Raspberry Pi, even the Zero version, and other single board computers are the best and the cheapest solution for running FluidSynth.
But as you already noticed, their boot time is very slow, if compared to an usual microcontroller which is up and running in few milliseconds. I know that there are some ports of RTOS for the Raspberry Pi, like FreeRTOS, ChibiOS and eCos, but I never tried them.

It depends on what you want to do: in my project, I did a small board with a PIC18F25K50 for scanning the keyboard matrix of the piano and provides an USB port and the legacy MIDI-IN/MIDI-OUT. Then, I could connect to the USB port with several hosts like a PC or a phone.
I also tried to do some all-in-one solutions, with an embedded ARM Cortex M3 or a PDA, but I was not able to get decent results, as you could read in the report that I linked in this email.

So, if the all-in-one is what you want, my suggestion is to try again with a single board computer (Raspberry Pi is good, but you must also remember to add an I2S DAC because PWM audio is bad and slow) and try one of the RTOS solutions currently available. Porting is not difficult, I did some of them (Windows, DJGPP and others) without much problems. However, if you have some doubts, just ask.

Sincerely.


> Il 30 dicembre 2018 alle 20.55 Geoff Plitt <[hidden email]> ha scritto:
>
> I'm working on a musical instrument that uses FluidSynth for playing SoundFonts, works great. But I'm using Raspberry Pi (Raspbian) and there's a 30-60 second boot time, so I'm looking at other platforms with sub-5-second boot time. I'm really looking to replicate the experience of hardware synths, where you turn it on and it's ready to make sounds in a few seconds at most.
>
> Has anyone had success building/running FluidSynth on an embedded platform with minimal OS, like Arduino, Teensy, Adafruit Feather, etc? Let's say the system has an SD card and DAC/amp to output audio to little speakers, but not a full POSIX environment.
>
> I found  https://github.com/divideconcept/FluidLite but it doesn't include audio output and doesn't look maintained.
>
> My other approach might be to strip out unnecessary parts of Raspbian to get a fast boot time, but the documentation I've found still indicates it's hard to get under 10 seconds.
>
> _______________________________________________
> fluid-dev mailing list
> [hidden email]
> https://lists.nongnu.org/mailman/listinfo/fluid-dev

_______________________________________________
fluid-dev mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/fluid-dev
Reply | Threaded
Open this post in threaded view
|

Re: FluidSynth on embedded platforms, zero boot time?

Geoff Plitt
Thanks! Yes I'm getting great results with RPi Zero, using an I2S DAC. I will look into FreeRTOS, ChibiOS, eCos.

On Mon, Dec 31, 2018 at 5:50 AM Carlo Bramini <[hidden email]> wrote:
Hello!
yes, I succeded to run FluidSynth on an embedded platform.
I have built a digital piano and I used FluidSynth as a rendering engine.

First of all, I would like to suggest you to read this message:

http://lists.nongnu.org/archive/html/fluid-dev/2018-11/msg00022.html

with some results from my experiments.

I suggest you to abandon the idea to use things like Arduino and similar, they are simply not powerful enough. Technically, something like the Raspberry Pi, even the Zero version, and other single board computers are the best and the cheapest solution for running FluidSynth.
But as you already noticed, their boot time is very slow, if compared to an usual microcontroller which is up and running in few milliseconds. I know that there are some ports of RTOS for the Raspberry Pi, like FreeRTOS, ChibiOS and eCos, but I never tried them.

It depends on what you want to do: in my project, I did a small board with a PIC18F25K50 for scanning the keyboard matrix of the piano and provides an USB port and the legacy MIDI-IN/MIDI-OUT. Then, I could connect to the USB port with several hosts like a PC or a phone.
I also tried to do some all-in-one solutions, with an embedded ARM Cortex M3 or a PDA, but I was not able to get decent results, as you could read in the report that I linked in this email.

So, if the all-in-one is what you want, my suggestion is to try again with a single board computer (Raspberry Pi is good, but you must also remember to add an I2S DAC because PWM audio is bad and slow) and try one of the RTOS solutions currently available. Porting is not difficult, I did some of them (Windows, DJGPP and others) without much problems. However, if you have some doubts, just ask.

Sincerely.


> Il 30 dicembre 2018 alle 20.55 Geoff Plitt <[hidden email]> ha scritto:
>
> I'm working on a musical instrument that uses FluidSynth for playing SoundFonts, works great. But I'm using Raspberry Pi (Raspbian) and there's a 30-60 second boot time, so I'm looking at other platforms with sub-5-second boot time. I'm really looking to replicate the experience of hardware synths, where you turn it on and it's ready to make sounds in a few seconds at most.
>
> Has anyone had success building/running FluidSynth on an embedded platform with minimal OS, like Arduino, Teensy, Adafruit Feather, etc? Let's say the system has an SD card and DAC/amp to output audio to little speakers, but not a full POSIX environment.
>
> I found  https://github.com/divideconcept/FluidLite but it doesn't include audio output and doesn't look maintained.
>
> My other approach might be to strip out unnecessary parts of Raspbian to get a fast boot time, but the documentation I've found still indicates it's hard to get under 10 seconds.
>
> _______________________________________________
> fluid-dev mailing list
> [hidden email]
> https://lists.nongnu.org/mailman/listinfo/fluid-dev

_______________________________________________
fluid-dev mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/fluid-dev


_______________________________________________
fluid-dev mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/fluid-dev
Reply | Threaded
Open this post in threaded view
|

Re: FluidSynth on embedded platforms, zero boot time?

Carlo Bramini
Hello,
since you are using Raspberry Pi Zero, I also suggest you to:

1) compile FluidSynth with "enable-threads" optionset to "off", or remove definition of ENABLE_MIXER_THREADS from your config.h file for disabling parallel rendering. Since the BCM2835 is a single core ARM11, having multithreading is unnecessary.

2) If you will decide to not use threading, when you will port to your RTOS, probably you can leave all these functions just empty:

fluid_curtime
fluid_utime
fluid_thread_self_set_prio
fluid_mutex_init
fluid_mutex_destroy
fluid_mutex_lock
fluid_mutex_unlock
fluid_cond_mutex_lock
fluid_cond_mutex_unlock
new_fluid_cond_mutex
delete_fluid_cond_mutex
fluid_rec_mutex_init
fluid_rec_mutex_destroy
fluid_rec_mutex_lock
fluid_rec_mutex_unlock
fluid_private_init
fluid_private_free
fluid_private_get
fluid_private_set
fluid_cond_signal
fluid_cond_broadcast
fluid_cond_wait
new_fluid_cond
delete_fluid_cond
new_fluid_thread
delete_fluid_thread
fluid_thread_join

This function is not stricly necessary, but it will be useful for not locking the CPU at 100% usage when you will wait that the playback of a MIDI file is finished:

fluid_msleep

it has to just call the function for leaving the control to the IDLE process, so it should not be a big problem. Finally, write the atomic functions in a very simple manner (this is my implementation):

void fluid_atomic_int_inc(volatile int *_pi) { *_pi++; }
int fluid_atomic_int_add(volatile int *_pi, int _add)
{
    int prev = *_pi;
    *_pi = prev + _add;
    return prev;
}
int  fluid_atomic_int_get(volatile int *_pi) { return *_pi; }
void fluid_atomic_int_set(volatile int *_pi, int data) { *_pi = data; }
int  fluid_atomic_int_dec_and_test(volatile int *_pi)
{
    int data = *_pi - 1;
    *_pi = data;
    return data == 0;
}

int  fluid_atomic_int_compare_and_exchange(volatile int *_pi, int _old, int _new)
{
    if (*_pi == _old)
    {
        *_pi = _new;
        return 1;
    } else
        return 0;
}

int  fluid_atomic_int_exchange_and_add(volatile int *_pi, int _add)
{
    int ret = *_pi;
    *_pi += _add;
    return ret;
}

void fluid_atomic_float_set(volatile float *fptr, float val)
{
    fluid_atomic_int_set((volatile int *)fptr, val);
}

float fluid_atomic_float_get(volatile float *fptr)
{
    union {
        int32_t i;
        float   f;
    } val;

    val.i = fluid_atomic_int_get((volatile int *)fptr);

    return val.f;
}

Of course, you must be sure that the functions of FluidSynth are called from a single thread and this depends how you will write your final application.

Sincerely.


> Il 31 dicembre 2018 alle 23.57 Geoff Plitt <[hidden email]> ha scritto:
>
> Thanks! Yes I'm getting great results with RPi Zero, using an I2S DAC. I will look into FreeRTOS, ChibiOS, eCos.
>
> On Mon, Dec 31, 2018 at 5:50 AM Carlo Bramini < [hidden email]> wrote:
> > Hello!
> >  yes, I succeded to run FluidSynth on an embedded platform.
> >  I have built a digital piano and I used FluidSynth as a rendering engine.
> >
> >  First of all, I would like to suggest you to read this message:
> >
> >  http://lists.nongnu.org/archive/html/fluid-dev/2018-11/msg00022.html 
> >
> >  with some results from my experiments.
> >
> >  I suggest you to abandon the idea to use things like Arduino and similar, they are simply not powerful enough. Technically, something like the Raspberry Pi, even the Zero version, and other single board computers are the best and the cheapest solution for running FluidSynth.
> >  But as you already noticed, their boot time is very slow, if compared to an usual microcontroller which is up and running in few milliseconds. I know that there are some ports of RTOS for the Raspberry Pi, like FreeRTOS, ChibiOS and eCos, but I never tried them.
> >
> >  It depends on what you want to do: in my project, I did a small board with a PIC18F25K50 for scanning the keyboard matrix of the piano and provides an USB port and the legacy MIDI-IN/MIDI-OUT. Then, I could connect to the USB port with several hosts like a PC or a phone.
> >  I also tried to do some all-in-one solutions, with an embedded ARM Cortex M3 or a PDA, but I was not able to get decent results, as you could read in the report that I linked in this email.
> >
> >  So, if the all-in-one is what you want, my suggestion is to try again with a single board computer (Raspberry Pi is good, but you must also remember to add an I2S DAC because PWM audio is bad and slow) and try one of the RTOS solutions currently available. Porting is not difficult, I did some of them (Windows, DJGPP and others) without much problems. However, if you have some doubts, just ask.
> >
> >  Sincerely.
> >
> > > Il 30 dicembre 2018 alle 20.55 Geoff Plitt < [hidden email]> ha scritto:
> > >
> > > I'm working on a musical instrument that uses FluidSynth for playing SoundFonts, works great. But I'm using Raspberry Pi (Raspbian) and there's a 30-60 second boot time, so I'm looking at other platforms with sub-5-second boot time. I'm really looking to replicate the experience of hardware synths, where you turn it on and it's ready to make sounds in a few seconds at most.
> > >
> > > Has anyone had success building/running FluidSynth on an embedded platform with minimal OS, like Arduino, Teensy, Adafruit Feather, etc? Let's say the system has an SD card and DAC/amp to output audio to little speakers, but not a full POSIX environment.
> > >
> > > I found  https://github.com/divideconcept/FluidLite but it doesn't include audio output and doesn't look maintained.
> > >
> > > My other approach might be to strip out unnecessary parts of Raspbian to get a fast boot time, but the documentation I've found still indicates it's hard to get under 10 seconds.
> > >
> > > _______________________________________________
> > > fluid-dev mailing list
> > > [hidden email]
> > > https://lists.nongnu.org/mailman/listinfo/fluid-dev 
> >
> >  _______________________________________________
> >  fluid-dev mailing list
> >  [hidden email]
> >  https://lists.nongnu.org/mailman/listinfo/fluid-dev
>
> _______________________________________________
> fluid-dev mailing list
> [hidden email]
> https://lists.nongnu.org/mailman/listinfo/fluid-dev

_______________________________________________
fluid-dev mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/fluid-dev
Reply | Threaded
Open this post in threaded view
|

Re: FluidSynth on embedded platforms, zero boot time?

Marcus Weseloh
In reply to this post by Geoff Plitt
Hi,

Am Mo., 31. Dez. 2018 um 23:57 Uhr schrieb Geoff Plitt <[hidden email]>:
Thanks! Yes I'm getting great results with RPi Zero, using an I2S DAC. I will look into FreeRTOS, ChibiOS, eCos.

I'm running FluidSynth on similar but slightly more powerful hardware. The plain linux boot time (via SD-card) from power on to start of my custom kernel drivers, fluidsynth and my main program takes about 4-5 seconds. And I haven't taken some of the more invasive steps to reduce the Linux boot time yet. And I'm not really sure it's even relevant... as most of the startup time of my embedded system comes from loading the Soundfonts from disk and fluidsynth parsing them into memory. But the dynamic-sample-loading option has helped in reduced that load time considerably.

I'm not using a ready-made ARM distribution, but build everything from scratch using buildroot. It's a fantastic tool that lets you build extremely streamlined Linux systems that also boot very quickly. I think the key is: remove everything you don't need. Both in the system and in the kernel. The guys from bootlin (formerly Free Electrons) who developed buildroot also have good information about boot time reduction: https://bootlin.com/doc/training/boot-time/

And maybe as an additional note: on my embedded platform, I'm using the preempt-rt realtime patches for the Linux kernel. They give you a real-time capable Linux system with bounded latencies, configurable IRQ priorities etc via the standard Linux API. No need for a separate RTOS, no need for new or distinct APIs. I haven't tried other RTOS solutions though, so can't really compare the different approaches.

Cheers,

    Marcus 

_______________________________________________
fluid-dev mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/fluid-dev
Reply | Threaded
Open this post in threaded view
|

Re: FluidSynth on embedded platforms, zero boot time?

Ceresa Jean-Jacques

Hi,

>I'm running FluidSynth on similar but slightly more powerful hardware. The plain linux boot time (via SD-card) from power on to start of my custom kernel drivers, fluidsynth and my main program takes about 4-5 seconds.

 

I like very much this kind of news. This beats by far my old Win XP system that take 20 seconds from an IDE hard disk (100Mb/s only) :).

Please could you recall the hardware used and the maximum voices it can play (if you remember this).

Also i am convinced that boot time could be seriously reduced using hardware with on board flash memory. Anyway 4-5 s is really comfortable.

All the best

jjc.

 


_______________________________________________
fluid-dev mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/fluid-dev
Reply | Threaded
Open this post in threaded view
|

Re: FluidSynth on embedded platforms, zero boot time?

Ben Gonzales
Yes, That's really good. 5 sec is the boot time for my Roland Aerophone, a commercial product with synth inside.

Ben

On 5/1/19 3:28 am, Ceresa Jean-Jacques wrote:

Hi,

>I'm running FluidSynth on similar but slightly more powerful hardware. The plain linux boot time (via SD-card) from power on to start of my custom kernel drivers, fluidsynth and my main program takes about 4-5 seconds.

 

I like very much this kind of news. This beats by far my old Win XP system that take 20 seconds from an IDE hard disk (100Mb/s only) :).

Please could you recall the hardware used and the maximum voices it can play (if you remember this).

Also i am convinced that boot time could be seriously reduced using hardware with on board flash memory. Anyway 4-5 s is really comfortable.

All the best

jjc.

 


_______________________________________________
fluid-dev mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/fluid-dev


_______________________________________________
fluid-dev mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/fluid-dev
Reply | Threaded
Open this post in threaded view
|

Re: FluidSynth on embedded platforms, zero boot time?

Marcus Weseloh
Hi,

Am Fr., 4. Jan. 2019 um 20:45 Uhr schrieb Ben Gonzales <[hidden email]>:
Yes, That's really good. 5 sec is the boot time for my Roland Aerophone, a commercial product with synth inside.

Sorry to disappoint... it seems like I wasn't clear in my previous mail. The 5 seconds boot time is Linux only, not the complete system. So it only includes uboot (the boot loader), Linux kernel boot, init process. So everything up to (but excluding) the start of my application.

After Linux start I run my Python main program (which in turn starts a custom C-extension which handles all time-critical aspects, including fluidsynth). Then the system gets configured via stored presets, which in turn trigger the Soundfont loading. So if we take out the soundfont loading, the system is responsive in about 10 seconds. Depending on the size of the samples in the selected presets, another few seconds to load the sample data from the SD-card.

The platform is an Allwinner A20, which is a dual-core ARMv7 running at 960Mhz. I can't really remember the actual measured max voices count (i've posted performance metrics some time ago on this list)... but I've set max polyphony to 64 currently. Fluidsynth only runs on a single core, the other core takes care of all sensor readings, USB communication, webserver, websockets etc. Latency using the build-in DAC on the SOM is around 12-15ms.

The code is all open source at https://github.com/midigurdy (distributed across multiple repositories in that organization).

But like I said to Geoff in a private mail: getting to 5 seconds Linux boot is really not that difficult. At least not if you are willing to let go of precompiled distributions and all the niceties that modern distributions bring along, like udev, systemd, networkmanager etc. With sytems like buildroot, you get an extremly streamlined Linux system with only minimal infrastructure to run your main application. Choose mdev instead of udev, compile your own kernel and strip out all you don't need. And even sub-second Linux boot is possible.... it's just a matter of how much infrastructure you can live without. I think if I were to strip out kernel print messages and get rid of some other modules I included for easier development and testing, remove the boot splash logo,  I could probably shave off another second or more of Linux boot. But I don't think it's really that relevant considering the time it takes to load the samples from the SD-card.

All the best,

    Marcus



_______________________________________________
fluid-dev mailing list
[hidden email]
https://lists.nongnu.org/mailman/listinfo/fluid-dev