Format Fitness – iOS music makers need a plugin standard to stick

finger on a tablet computer screenWhen Apple first launched iOS on the world, one of its key ‘features’ was the nature of the software (apps) that were intended to run on it. These were intended to be focused, ‘single-function’, tasks – a weather app, a stocks app, a memo recording app, a casual game app, a book-reading app, an email app, a simple photo editing app….. well, you get the idea. The software might well have been quite polished and slick but the vast majority of the apps offered a simple workflow and a tightly focused (and not overly deep) feature set.

This was (and still is) a great format for mobile computing devices. It offers most users as much functionality as they need most of the time and, if you are a really ‘deep’ users of a specific type of computer application (photo editing, text editing, gaming), then you went off to your desktop.

It was, however, a bit of a ‘standalone’ ecosystem where you used one app at a time. Yes, you could ‘share’ stuff (that is, move it between users or apps) via email or social media or a Dropbox-style app or via iTunes File Sharing, but this wasn’t always easy and the whole ‘cut ‘n’ paste’ process that is a familiar part of the desktop workflow – where data, text, images, etc. can easily be moved between different software applications – wasn’t (and still isn’t) something that was a strong suit of the OS.

Once upon a time, iOS music apps just ran as standalone instruments…. Animoog is an example but it demonstrated the considerable potential iOS had for music technology.

And the same was true of the first music apps that appeared, whether that was a synth, a piano, a drum machine or some other sort of virtual instrument. There were some brilliant examples amongst this first flush of music apps but they were also standalone tools. This first ‘format’ – the ‘standalone/run on its own’ format – was great when used individually in a ‘performance’ context, but it was not easy to use these various apps together in any serious/simple fashion (at least not with just a single iPhone/iPad).

Learn from the past

By the time iOS arrived, on the desktop, music software had solved this ‘between software’ communication issue. Indeed, it had been through several generations of solutions but perhaps the game-changing event was Steinberg’s Virtual Studio Technology – VST, in 1996 – that attempted, in software, to replicate the way in which music technology hardware in a studio-like environment could be connected, allowing audio to pass from one device (for example, a synth or effects processor) to another (for example, a mixer).

VST was (is) a software protocol that follows a set format for between software communication. In essence, it allows a VST ‘host’ (for example, a DAW/sequencer), could provide a home for a VST ‘plugin’ (for example, a software synth or software compressor) to be inserted into the host and used as if it was simply an extension of the features of that host.

The advantages of this ‘host + plugin’ approach were obvious and, in particular, the fact you could run multiple instances of a plugin within the host and that all the plugin settings were stored, and instantly recalled, as part of the host project, would be top of that list.

This is now the dominant paradigm on the desktop and perhaps the only thing to note is that VST is not the only plugin format. OSX also offers the Audio Units format (although VST also works under OSX), while Pro Tools users also have access to Avid Audio eXtension (AAX) and Real-Time Audio Suite (RTAS) plugin formats.

On the desktop, the plugin concept is now just part of the expected workflow…. and powerful virtual instruments such as Native Instrument’s Kontakt can run in an identical fashion in almost any desktop DAW/sequencer as well as standalone.

You might think that multiple formats could confuse matters and, perhaps early on in the process, that might have been the case. However, these are now long-established formats and, as a consequence, all the major software developers are well tuned in; when a new virtual instrument or virtual effects processor appears, it will generally be delivered with all of these formats available…. and if it is not, then the odds are it will not cut it in a commercial context. And if you are a new developer trying to break into the desktop music software marketplace, you better deliver in VST/AU as a bare minimum if you are going to give it a serious shot.

I suspect dealing with these various formats can be a bit of a pain in the *** for developers and they might all wish that there was a single ‘standard’ to work to. That said, these are formats that have now all been around for a long time so the expertise required to deal with them is well established.

However, perhaps the more important factor is that, to the user, the format is largely immaterial. Whether you are using VST, AU or one of the other plugin standards, within a suitable host, the plugin will both operate and look identical. And, with some hosts – for example, Harrison’s excellent MixBus DAW under OSX, you can freely mix and match between the LV2 plugin format Harrison use for their own plugins, VST and AU within the same project. The bottom line here is that, to the user, the plugin format doesn’t change the workflow.

Crawl, walk, run, sprint

Moving beyond the ‘just crawling’ stage offered by standalone-only mode under iOS has involved a number of different technologies. Aside from being able to move audio files created in one application to another (for example, via AudioShare, that first appeared in July 2012) but it was Audiobus (a few months later) that offered the first technical solution for getting iOS music apps to pass audio between each other in real-time.

This ability did require that the individual apps that were to be ‘hosted’ within Audiobus had some Audiobus-specific code added to them and it took quite a while before Audiobus compatibility became the norm for new apps (although it soon became something every user considered when making purchase decisions on new apps). It wasn’t, of course, quite the same as VST/AU on the desktop in that it was a format designed for a single host – Audiobus itself – rather than something that could then slot into any of a number of hosts… but it was a huge step forward in terms of workflow and we should not underestimate the impact Audiobus had in demonstrating the potential of iOS music making.

The initial launch of Audiobus suddenly opened up new possibilities for iOS music makers….

Apple took a little while but they did eventually cotton on the one group of their client base – music makers – required iOS to do something other than just run individual standalone apps. WE wanted the option to run multiple apps side-by-side and we wanted them to communicate with each other. There initial response was Inter App Audio (IAA) as part of iOS7 back in September 2013. Again, it required code adding to individual music apps, both to virtual instruments, etc. (so they could act as ‘IAA plugins’) and to potential hosts such as DAW/sequencer (so they could act as IAA hosts).

Again, it took time, but it did gradually happen and, yet again, iOS music makers were soon keen to see ‘IAA compatible’ in the feature list of any new iOS music app. And because it encouraged a range of IAA hosts – and DAWs such as Cubasis and Auria (for example) soon offered IAA hosting support – for recording junkies at least, it perhaps was a step forward compared to Audiobus (although, for some time, not always as technically solid).

IAA provided the next technological step forwards as a ‘plugin’ format…. but never really delivered the slick desktop workflow experience for the user.

On paper, IAA could have been the first step to a ‘proper’ plugin format for iOS and perhaps that’s what Apple initially planned. As it turns out, that potential was never quite realized and the format, while getting some modest tweaks (and certainly becoming more reliable), didn’t really expand dramatically on its core features set.

So both Audiobus and IAA were remarkable steps forward and embraced fully by the user community. But everyone also knew that both technologies had their limitations compared to the more mature world of the desktop. We were certainly walking… but perhaps not running.

Top of that list would be the inability to run more than a single instance of any app at the same time. This could be worked around but it did make for a less slick workflow under iOS than folks were used to on the desktop. Equally, however, saving a DAW/sequencer project on the desktop generally saved all your plugin settings and fully recalled those settings when you re-opened the project. That’s still not something that happens with 100% reliability within Audiobus or via an IAA host depending upon the combination of apps you happen to have ‘plugged in’.

In September 2015, Apple perhaps gave us a push towards the ‘run’ phase; iOS9 launched and, while IAA remained (and perhaps already with a ‘legacy technology’ status), a version of their Audio Units desktop plugin standard, appeared for iOS. This was greeted with some considerable excitement by iOS music makers. The potential was obvious; finally, iOS was going to get a plugin standard that could deliver all the same benefits – multiple instances and settings retained by the host – that desktop users had been experiencing in a mature fashion for 10-15+ years.

MultitrackStudio for iPad was the first AU host…. and iSEM the first virtual instrument to appear as an iOS AU plugin….

However, while we might have wanted to run, we were not running very quickly. AU was, in its first iteration at least, not the most robust technology not did it offer the full feature set of the desktop protocol. The consequence was that developers had two reasons to hesitate in re-working their apps to support AU. First, the technology didn’t feel complete (so AU R&D might be time wasted if Apple then shifted the technical goalposts). Second, with App Store economics so tight for many developers, funding the development work required to implement AU in an existing app without an easy way to generate a financial return, was a risk many were not able to take.

The result was, for 12 months at least, AU made something of a stuttering start under iOS. A modest number of apps added AU and an equally modest number of new apps appeared that offered AU support. All the obvious hosts – mostly the DAW/sequencers – gradually added AU hosting support, but it did take time. AU adoption was therefore slow.

Now we have a good range of synths and audio effects available with AU plugin support….

To their credit, Apple did – albeit slowly – improve the AU spec and, in its ‘v.3’ format, it is undoubtedly more robust, offers more features and an easier environment for developers to deal with technically. That doesn’t remove the financial ‘return of investment’ barrier for some developers…. but we are in a better place than we once were.

And the consequence of that is, that over the last 12 months (through the life cycle of iOS10), we have seen a significant increase in the number of iOS music apps that offer AU support. There are still plenty that don’t (including some complete categories of apps) but – just about – you could now put a pretty good music production system together on your iPad using just an AU host or two and a collection of AU-based virtual instrument/effects apps. If you are prepared to forego a few old favourites that have not yet added AU, and workaround (or don’t need) those ‘still missing in action’ categories, you are pretty much in the ‘running’ stage.

Time to sprint?

If you inhabit any of the usual iOS music making hangouts, when new apps are now released, as in the past with Audiobus and IAA support, perhaps the key technology folks are now commenting upon is AU support. If an app doesn’t offer it, then some potential sales are, I suspect, now being lost. Audiobus and IAA support are generally still included in most new apps…. but not all. That’s also a bone of contention for some long-standing iOS music makers with established workflows based upon those technologies.

That said, all the major ‘hosts’ now offer AU support. This includes all the most popular DAW/sequencers and the two most popular platforms that are perhaps closer in concept to a ‘virtual mixer’ – Audiobus itself and AUM – so the question is perhaps now ‘how long with IAA and Audiobus (the plugin protocol side of it, not the host itself) be required?’. Both represent important steps in the development of iOS music technology, but surly, as ‘plugin’ formats, the days of both are numbered?

Almost all the major DAW/sequencers – as well as Audiobus 3 and AUM – now offer hosting for AU plugins.

As I write this post, iOS11 is almost upon us (Apple’s iPhone 8 Keynote is later today!). I’ve no insider knowledge but I can only imagine iOS11 will bring some further refinements to the AU support. Surely, the next 12 months is going to see us in a place where all new iOS music apps really need is a standalone version and an AU version? Personally, I’d welcome that, although I’m sure some will have genuine reasons for disagreeing with me. It would, however, represent a position under iOS that mirrors exactly what happens on the desktop. Finally, from a workflow perspective, we might be sprinting?

Out with the old?

There are consequences though. I’d be surprised if there were not a lot of well-established ‘favourite’ apps that, mainly for financial reasons, don’t ever get AU support added. This is understandable if a developer can’t see how to generate the return on investment.

It is also perhaps understandable if an app features a particularly complex UI that is perhaps difficult to compress into the more limited screen real-estate offered by the AU plugin sub-windows that are currently offered by most hosts. This last point is something that might change in time though if Apple and/or the host developers can get on the same page and offer a ‘full-screen’ plugin window option for AU apps.

Thor is a personal favourite with Audiobus and IAA support…. but will we ever see it as an AU plugin under iOS?

I’ve some personal go-to apps that I’d love to see make the AU transition (and I’m sure you can think of your own). This would include some classic iOS synth apps such as Thor or Nave, and it would certainly include the various Korg synth apps (despite Korg’s own plugin format that is offered by Gadget) but, as we already have lots of AU-ready synths, this is a category I could easily already find some able replacements within. Equally, when it comes to audio effects, we have an impressive set of AU-ready plugins.

However, as mentioned earlier, there are some ‘missing’ categories where very few AU-based apps have appeared. Guitar rig sims (e.g. BIAS FX), acoustic drum virtual instruments (e.g. Drum Session), multi-timbral sample-based instruments (e.g. SampleTank) and mastering apps (e.g. Final Touch) are all notable by their absence. Until developers of these types of apps find a way into the world of iOS AU, that full-on sprint mode might have a few limps within it.

It would also be good to see one of the major iOS guitar app rigs take the AU plunge.

From sprint to fly?

But it is going to happen…. whether it is these favourites that get AU support, or new apps (perhaps by new developers?) that pop up to plug a gap in the iOS AU marketplace, AU is likely to be the plugin format that delivers the super-slick desktop-style workflow to your iPad and iPhone. And, if AU is the plugin format that eventually sticks, in making that workflow mirror the established desktop workflow, we might – finally – see a significant barrier between the two worlds disappear.

Those desktop-based musicians who have perhaps dismissed iOS as something of a toy might gradually find that it can get some serious stuff done and, what’s more, the learning curve to get all these excellent iOS music apps to work together in a coherent system – similar in workflow to the one they experience on the desktop – becomes non-existent.

Are we there yet? Well, when it comes to an AU-only iOS music app selection, perhaps not quite…. but we should also not ignore the progress that has been made. An AU-only approach is getting closer….

In terms of the long-term health of iOS as a music-making platform, pulling in established desktop-based musicians has got to be a significant step forward. And an AU-based workflow can be an important part of that process.

Bring it on…. While I love the unique elements that can be found on both the desktop and under iOS, to fully capitalise of both, I want these two worlds to offer a seamless experience. We are perhaps still a little way off that as yet….   Yes, perhaps we are only just about sprinting (and not all the time)….   but it would be even better if we could fly :-)

Oh, and as a final thought, if you want to share your thoughts on a single app that you most sincerely hope makes it across the AU divide, then please leave a comment below. It will be interesting to see if a few favourites appear….

Be Sociable; share this post....

    Comments

    1. I do wonder if there is a small case for still keeping on the old non-plugin formats. One thing that occurred to me for example is that there is at least one app on OSX/PC that isnt a plugin – Band In A Box – that i’d love to be able to operate in an IAA fashion on OSX – kind of like ReWire does when communicating with a DAW.
      Ableton also operates like this when “hosted” inside a DAW. Both BIAB and Ableton are of course – much more than simple audio generators or MIDI “filters” so arguably dont fit the AudioUnit paradigm…. but i’m open to persuasion on that.

      Another reason for maybe retaining IAA etc ( apart from backward compatibility) might simply be the desire to keep “big” apps that normally take a while to launch and “load up” – continually in RAM – simply getting rerouted to different DAW type hosts ( or AUM ) as needed. Since a plugin i’m pretty sure – gets its process killed – it quits – once its host is exited.

      Dunno.

      On OSX it has only ever been Band In A Box that I dreamt would eventually get something like IAA support for IAA hosting inside a DAW… so maybe just going full-on for AU and deprecating IAA/AudioBus in the long run is the wiser route.

      DUNNO.

      Oh – and as for that single app – I vote for SampleTank – redesigned so that instead of being multitimbral – each AU instance just has a single ST instrument – picked out of the ST arsenal.

      Multitibral plugins ( ie multichannel ) to me are always a right pain to set up in DAWs.

      • Hi Daniel…. thanks for that… and, yes, a mono-timbral version of SampleTank for AU iOS would be a cool idea…. and undoubtedly easier to manage with multiple instances rather than a single instance running multiple sounds. IKM are obviously familiar with the desktop plugin formats so you would imagine that the expertise is there…. More a question of the resources and the likely returns they think might see I guess…. best wishes, John

    2. AU *host* support in BeatHawk. Primarily for sampling of single AU sources (cf. BeatMaker 3 but keeping it simple(r)?), especially considering that its current input modes (i.e. mic/external pretty much) have always felt quite limiting. And as a bonus, just being able to bypass/replace the built-in delay+reverb FX with 1+1 hosted AUs would be extremely useful, even if no other major changes were made to the workflow or related UI. My 20 cents, considering that I love the streamlined and fast workflow of BeatHawk, while feeling a bit constrained by the walled gardens present in the app and OS. Keeping my fingers crossed wrt Files in iOS 11 opening the latter up a bit though… (?)

    3. Loran Pittman says:

      Nice article! Come join the AU group over on Facebook where I regularly update the iOS AU compatible app list. https://www.facebook.com/groups/1704410769839550/?multi_permalinks=1969567116657246&notif_t=group_activity&notif_id=1505285564326537

    4. Passivemixer says:

      AU support for Layr would be great.

    5. Nathan Cipperly says:

      AU support for Model 15 and Korg Mono/Poly. The gadget closed system means I find I either write things exclusively with Korg synths in gadget, or completely exclude Korg while using AU in Auria Pro with Zeeon and Bram Bos AU plugins. A meeting point on all of those would be amazing.

    6. It is this nonconformity, not only in Format but also in unidentifiable and non-standard symbols which has meant that ever since the iPad one, I haven’t managed to compose any music whatsoever with iOS. It’s just become to difficult for my old brain.

    7. Patterning, Model 15, and AU support on the GarageBand auxbp/send bus would do it for me

    Speak Your Mind

    *