It is said the road to hell is paved with good intentions.  I’m starting to appreciate the truth of that at the moment.  The music app that I developed worked reliably for a number of years.  Over the last couple some unexpected issues where songs could not be found or would not play started to appear.  This post explains what is causing it and, more importantly, how to fix it.

I’ll go through the entire situation to provide context but sections will be highlighted for app (how the app is affected), background (influences on the situation), and technical (coding stuff).  Feel free to skip the ones you have no interest in.  The section at the bottom of the page will show you how to fix it if you’re in a hurry.

App

The app tries to match against Les Mills quarterly releases by looking through the music library maintained by iOS.  It reads the details of the songs that are present and tries to match them against a list of albums the app stores internally.  This list is updated regularly.  When you tap on the ‘releases’ tab and download an update it’s storing a new list of albums.

 

Technical

There are two approaches Apple provides for playing items from the music library.  In the early days the app used the MPMusicPlayerController class.  It’s a very simple one which is great for playing a single track.  However, it didn’t allow for playing multiple songs in the background and it would have slight pauses between tracks (and let me tell you the Steppers were not happy about that).  What with the various scheduling issues required to add pauses it did not produce the most stable app.  So I moved to using the AVMutableComposition class.  This allows songs to be loaded as a AVURLAsset and then inserted into the AVMutableComposition instance.  In effect it adds the raw data of the song files into one large song file.  This allows the pre-song silence to be trimmed and the scheduled pauses before your tracks to be added.  It’s also the same class which allows the cross-fading between songs when playing pre-class snippets.

So this class has lots of advantages.  The greatest one is stability because when a class starts all it needs to do is play from the start with the minimal amount of effort.  It also left the door open for future development including audio levelling where all the songs in a playlist could be brought up to the same apparent volume.

 

App

People transferred their music from iTunes to their device by plugging it in directly via a cable.  The device ended up with an identical version of the music on their computer.  The world was a happy place.

 

Background

Then August 2015 arrived.

Apple Music.

The iCloud Music Library was debuted with this.  It uploaded all of the music on your desktop machine to the cloud and allowed you to download to stream it to your mobile devices.  Which is wonderful.  In theory.

In practice it was rather bad.  Apple decided that since they had 30 million songs on their servers then they didn’t need to upload all the songs from your desktop, only the ones they didn’t have.  For all the others they could simply give you a copy on your mobile device of the song they already had.  That saved you uploading a lot of songs.  Sounds great.  Except…

…the way they determined that they already had one of your songs was to compare the song title and artist name.  And that was it.  As instructors who work with remixes that are specifically designed to fit choreography you’ll immediately spot the problem.  Many people with live albums suddenly realised this when half the tracks were replaced with studio versions.  This is an issue still not resolved today.

 

App

When the app is looking through the music library it only considers songs that have been downloaded to the device.  Why do this when all the songs can exist in the cloud and be available for streaming?  The design decision I made was that you cannot guarantee a network connection will always be there and if you were trying to stream songs during class there was a significant chance the music would stop at some point.  Not great for participants, embarrassing for the instructors, and a lot of explaining for me to do.  So in order to reduce the chance of disruption in class all songs must be downloaded to the device.

 

Background

But there was another fly in the Apple Music ointment.  When you bought and downloaded a Les Mills release you get MP3 files.  These can be played on any device.  They are sold to you on the basis that the large majority of people will buy their own copy and not share them with other people.  People, generally, are honest.  Some music providers are a lot more suspicious of their customers.  Anyone who has taken note of record company behaviour over the last few decades will realise they fall into this category.  When Napster exploded early last decade the companies started to implement DRM.  This is ‘digital rights management’ and it restricts what can be done with the music.  In most cases this meant it was tied to a specific user account so only they could play it.  If a copy was given to another person that song would not play.  The record companies loved it; consumers didn’t as they would find they couldn’t use their music as they thought they should be able to.

The iTunes Music Store came along – with DRM – but eventually came popular enough to be able to push the record companies into dropping DRM.  Everyone was happy again.  The came streaming.  For the cost of buying a single album you could instead stream almost any music you wanted for a month.  But people also wanted to download those streaming songs to their mobile devices so they could listen to them while they were outside.  Downloading them at home meant not using up their 3G/4G data when they were out.  And you know there’s a ‘but’ coming up here…

…but when one of the 30 million songs on the Apple servers are downloaded to your device it is wrapped in DRM.  Even if you have purchased the Les Mills DRM-free version.

 

App

When the app is faced with a DRM-encumbered song it is unable to open it.  It cannot play it.  If you have recently had the situation where you have a track in a playlist and it has skipped over it in class this is the reason why.

 

Technical

When loading an item to add it to the AVMutableComposition entity the following code is used:

NSURL * workingUrl = [librarySong valueForProperty:MPMediaItemPropertyAssetURL];
NSDictionary *options = [NSDictionary dictionaryWithObject:[NSNumber numberWithBool:YES] forKey:AVURLAssetPreferPreciseDurationAndTimingKey];
AVURLAsset * asset = [AVURLAsset URLAssetWithURL:workingUrl options:options];

For all DRM-encumbered songs the last line always returns nil.  This means it cannot read the contents of that song in order for them to be inserted into the aggregated track.

I can see the reasoning for it.  If you can open the file to get access to the raw data then it’s trivial to store it in another file.  That would mean anyone could subscribe to a streaming service for one month and take a perfect copy of everything they can download.  But why not allow me to access and manipulate the data but prevent it from being exported?

App

The problem with Apple’s approach is that when they implemented these features of Apple Music it changed the underlying existing process and caused problems to manifest in the app.  It’s only when updating the app for iOS 10 and looking more closely at Apple Music that I realised the issues it was causing.

Originally the app would simply look at a downloaded song file and match it against a release.  This caused some problems because it would imply that the song would be played but if it was DRM-encumbered then it would not be played.  The first time you’d discover this would be in class.  The new version ignores DRM-encumbered files when doing the matching.  This is why a number of people are suddenly finding a lot less of their music present on their device.

What is happening is that for each Les Mills quarterly release there are fairly good odds that at least one of the tracks in it can be found on Apple’s servers.  So when you use iCloud to download it to your device it will give you the DRM-encumbered version.  That means the app will not find that release.

 

The Solution

Wait until you are home before doing this.  You need your mobile device and your desktop with iTunes.

Open iTunes on your desktop, create a playlist – I call mine “les mills (all)” and drag every LM song you have into there.

On the mobile device you use for teaching go to the ‘Settings’ app, select the ’Music’ item and turn off ‘Show Apple Music’ and ‘iCloud Music Library’.  This will remove pretty much all music from your device.

 

Now plug your device into iTunes.  Wait for it to finish doing it’s usual processing.  Switch over to the panel showing the device.  Select the ‘Music’ item in the left-hand column, turn on the ’Sync Music’ checkbox and then place a tick against the playlist you just put all your music into.  The hit the apply button.

 

This will now transfer the music from your iTunes to your mobile device without Apple adding DRM to it.  When it’s finished transferring open the app and your music and playlists should reappear.

When you get new quarterly releases simply drag them into the “les mills” playlist and they will automatically be copied across to your mobile device the next time you connect it.

 

Follow up

I have submitted this as a bug report to Apple because I consider this to be a fault of theirs caused by their actions.  Turns out I’m not alone as better developers than me have run into this problem – http://www.allenpike.com/2015/drm/

I’ve tried coming up with other solutions but most of them involve moving and storing your non-DRM files and that would get me threatening letters from those lovely record companies before you’ve even finished the warm-up track.  I’ll keep working on it but I suspect I’ll have to find a way around it rather than wait for the 800-pound gorilla of Apple software engineering to modify their internal code.