Monday, April 27, 2009

Album Art in Fremantle

Most of the media players need album art. So, ever application does the handling of the art by themselves (just like getting the metadata). For the metadata, we have tracker to get the metadata - one less headache. For the album art, we have tracker, hildon-thumbnailer and a standard to help alleviate another headache.

Standard? In Fremantle we have finally agreed on a standard on how to store album art and media art in general. This means that applications will be able to share the files, so album art is stored only once, retrieved from the internet if needed and a thumbnail suitable for list views is stored in common way.

We also worked together with Banshee team to make this a standard on linux desktop as well (Kudos to Philip van Hoof for it). You can read the spec in http://live.gnome.org/MediaArtStorageSpec

So, what does this mean in practice?
1. Tracker and hildon-thumbnailer do a lot of work for you in advance
- Tracker gets the embedded album art automatically
- Hildon-thumbnailer makes the thumbnails in advance in freedesktop.org standard
- Heuristic search is used in tracker and hildon-thumbnailer as specified in the media-art spec.
2. You can extend hildon-thumbnailer with content source plugins that download missing covers from the internet.
3. You can handle the album art all by yourself and just save some time if the condition 1. has hit its mark.

The option 2. is of course the preferred way of handling the album art, but heck: this ain't a perfect world, and I'm doing things a bit dirty, so I'll go for route 3. If the condition 2. would apply and an internet download plugin is available, I would change ukmp to depend on that package and thus, the following code would (mostly) not be needed at all.

However, quite a bit of it is useful. First of all: How is the album art filename (to the full version) calculated? I have nice copy paste functions here. Feel free to use them as is under any license.

You will need unicodedata and md5 as non-usual dependencies, so:

import md5
import unicodedata



Then the functions. First we create the album art filename as specified in the standard. The following function handles that conveniently for you. Now, it's up to you on how to use that. You can either just depend on whatever tracker and hildon thumbnailer have created for you (with or without the plugins), or, as ukmp does, which is, that if first checks whether the album art exists, if not, it downloads it from the internet.

coverlocation=homedir+"/.cache/media-art/"

def getCoverArtFileName( album ):
"""Returns the cover art's filename that is formed from the album name."""
albumString=dropInsideContent(album,"[","]" )
albumString=dropInsideContent(albumString,"{","}" )
albumString=dropInsideContent(albumString,"(",")" )
albumString=albumString.strip('()_{}[]!@#$^&*+=|\\/"\'?<>~`')
albumString=albumString.lstrip(' ')
albumString=albumString.rstrip(' ')
albumString=dropInsideContent(albumString,"{","}" )
albumString=albumString.lower()
albumString=string.replace(albumString,"\t"," ")
albumString=string.replace(albumString," "," ")

try:
albumString=unicodedata.normalize('NFKD',albumString).encode()
albumString=albumString.encode()
print albumString
except:
try:
albumString=albumString.encode('latin-1', 'ignore')
albumString=unicodedata.normalize('NFKD',albumString).encode("ascii")
albumString=str(albumString)
print albumString
except:
albumString="unknown"
if len(albumString)==0: albumString=" "

albumMD5=md5.new(albumString).hexdigest()
emptyMD5=md5.new(" ").hexdigest()
albumArt=coverlocation+"album-"+emptyMD5+"-"+albumMD5+".jpeg"
return albumArt


def dropInsideContent(s, startMarker, endMarker):
startPos=s.find(startMarker)
endPos=s.find(endMarker)
if startPos>0 and endPos>0 and endPos>startPos:
return s[0:startPos]+s[endPos+1:len(s)]
return s




Ok, great, now we have the full version. But, as ukmp needs mostly the thumbnail version, we need the filename to the thumbnail itself.


thumbnailLocation=homedir+"/.thumbnails/normal/"
def getCoverArtThumbFileName( album ):
artFile=getCoverArtFileName(album)
thumbFile=thumbnailLocation+md5.new(artFile).hexdigest()+".jpeg"
return thumbFile


If it happens that the thumbnail does not exist (e.g. wasn't created, has been removed or whatnot), you have a few options:
1. you can create the thumbnail yourself (I'll give an example soon for that)
2. you can request hildon-thumbnailer to create it for you

For the first option, you can just call hildon-thumbailer on the dbus:
https://stage.maemo.org/svn/maemo/projects/haf/trunk/hildon-thumbnail/daemon/thumbnailer.xml

I am not using the method myself at the moment, so here is a quick example. The method is not blocking, so proper use would need to also receive the finished signal from h-t with the thumbnailHandle property. Of course, you can also be polling to see when it has been generated. Usually in non congested situation, this is going to be some tenths of a second. If there is congestion, the content is handled lifo fashion in h-t.


import dbus, time
filename="file:///user/home/.images/01.jpg"
bus = dbus.SessionBus()
handle=time.time()
thumbnailproxy = bus.get_object('org.freedesktop.thumbnailer','/org/freedesktop/thumbnailer/Generic')
thumbnailHandle=thumbnailproxy.Queue([filename],["image/jpeg"],dbus.UInt32(handle))


I'm scaling inline in ukmp. I'm using PIL to scale down the image. It's slower than using pygame (or h-t), but looks better, as it has good anti-aliasing. Anyway, it's once in a lifetime happening, so it's ok to take a while. Here we are also using the above created functions (wehey). I'm using freedesktop org standard size: normal, which is 128x128. Be aware that the media player in Fremantle uses 124x124, so I might switch to that resolution as well. The coverlocation will then also switch from '~/.thumbnails/normal' to '~/.thumbnails/cropped'.

import PIL
thumbFile=getCoverArtThumbFileName(album)
fullCoverFileName=getCoverArtFileName(album)
if (os.path.exists(fullCoverFileName)):
thumbFile=getCoverArtThumbFileName(album)
fullCoverFileName=getCoverArtFileName(album)
image = Image.open(fullCoverFileName)
image = image.resize( THUMBNAIL_SIZE, Image.ANTIALIAS )
thumbFile=thumbFile
image.save( thumbFile, "JPEG" )

Sunday, April 26, 2009

A little tracking for the people waiting for Fremantle

I haven't been blogging much about Fremantle yet. But, yesterday there was a question on the maemo developers mailinglist about what files are indexed to the trackers metadata database, so I though to clear out that issue and also to tell a bit about how your app can use tracker.

So, to answer that question first: Tracker tracks the user home and any mounted media that is attached to the device. For the memory cards, it retains a stack of 3 cards in it's database, so, you can change card A to card B, and back to card A and tracker won't need to reindex the content of the cards. And yes, you can be then swithing to B, to A, to B and you won't lose any data. The amount of cards to support is a configuration option, but by default it's set to 3. So, internal card=1, external card1=2 and then you have the one more as exteral card2. For a third external card, the device will need to flush the oldest seen card away from it's indexes.

There was some concern also as to whether applications can put sound effects and pixmaps into the cards and to make sure they won't be indexed. Well, to this, we have two solutions:
1. put the files to a folder that is hidden (so, it has a "." in the beginning of the folder name) - tracker won't index any hidden folders by default
2. Add the folders to trackers blacklist file

I recommend the solution 1 for multiple reasons.
1. it's simple.
2. user doesn't have any reason to see application data anyway, so this way it'll be also hidden in the file manger. Just make sure your app will flush the data on uninstall of your app.

Ok, then a bit on how you can use Tracker.
As you probably have read, I've been on paternity leave, from which, I've taken a bit of time to integrate ukmp to the new Fremantle stack.

So, first thing I did was, I replaced my own indexing code with code to load all music metadata from tracker database. Loading of this data on startup takes almost no time and tracker also does sorting of the data really easily for me. Not that sorting would actually be any issue in python, nice anyway.

On startup, I hear you saying? Why not on demand? Sure, that would be an option, just happens that how ukmp was built, it's easier for me to get all the content on startup and not on demand. Both are fine. I could write a small comment on how to do stuff on demand as well, but let's start with this.

We'll need to use two interfaces: search and metadata

Corresponding dbus introspection files are: search and metadata

You can find the whole dbus introspection from here


Let's start with defining the needed proxy objects:

import dbus
bus = dbus.SessionBus()
searchproxy = bus.get_object('org.freedesktop.Tracker','/org/freedesktop/Tracker/Search')
metadataproxy=bus.get_object('org.freedesktop.Tracker','/org/freedesktop/Tracker/Metadata')

#Ok, let's then get all music files and for those, the artist, album, title and track# sorted by artist

metadata=searchproxy.Query(-1, "Music", ["Audio:Artist","Audio:Album","Audio:Title","Audio:TrackNo"],"", dbus.Array([], signature='s') ,"",False,["Audio:Artist"], False,0, 40000)

#Now that we have the data, we'll just add it to the internal structures

for songItem in metadata:
fileUrl=songItem[0]
artist=songItem[2]
album=songItem[3]
song=songItem[4]
track=songItem[5]
self.appendSong(track, album, artist, song, fileUrl)



Nice and simple. Now we have the data. This saved me about 300 lines of code, plus multiple library dependencies and tons of headache.

Of course, with my approach of loading everything on startup, I need to update the data when the data changes, but for this, tracker provides a really nice signal that looks like this:


signal sender=:1.15 -> dest=(null destination) serial=403 path=/org/freedesktop/Tracker; interface=org.freedesktop.Tracker; member=ServiceStatisticsUpdated
array [
array [
string "Files"
string "2320"
]
array [
string "Music"
string "543"
]
]

I won't show the implementation on how to keep the data update on this blog post, I'll save it for a future blog post. I'll instead now tell how to keep Tracker up-to-date on usage of the files. All media players on Fremantle should either use MAFW or do the following so that we would all be happy campers no matter which media player user uses.

When you are playing a music file, please notify tracker of the play event. I do so at the end of a track, but your heuristic may vary. Firts we get the current playcount, then we add 1 to it, then we set the new playcount and the curren playtime. We set the time in GMT in UTC format, which is rather easy to get in python.

import time
currentCount=metadataproxy.Get("Music",currentPlayFile, ["Audio:PlayCount"])
newcount=1
if len(currentCount[0])>0: newcount=int(currentCount[0])+1
currentTimeUTC=time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime())
metadataproxy.Set("Music",currentPlayFile, ["Audio:PlayCount","Audio:LastPlay"],[str(newcount),currentTimeUTC])


Ah, now we have the data in tracker, we are able to update the playcounts and playtimes so that all music players can benefit from the data. In my next blog post, I'll tell how the album art can be handled in common way across the platform. I'll tell you how you should do it and I'll tell you how I do it now (which might not be the case I will do when the device has been out for a while).

Then I'll make a blog about how to make dynamic lists, e.g. to list most popular tracks, most recently added and the most recently played tracks.

Then, to top this, I'll let you know how the signaling can be used to keep your internal data structures up-to-date, in case you are not using on-demand loading of the data.

edit: fixed typo as noticed by timeless

Monday, April 20, 2009

I'm a daddy!



Almost exactly a week ago I became a dad of a very sweet little girl. Her hello world message will be coming a bit later on when a name has been bestowed upon her.

I am the happiest man on the planet.