Monday, April 27, 2009

Album Art in Fremantle

Most of the media players need album art. So, ever application does the handling of the art by themselves (just like getting the metadata). For the metadata, we have tracker to get the metadata - one less headache. For the album art, we have tracker, hildon-thumbnailer and a standard to help alleviate another headache.

Standard? In Fremantle we have finally agreed on a standard on how to store album art and media art in general. This means that applications will be able to share the files, so album art is stored only once, retrieved from the internet if needed and a thumbnail suitable for list views is stored in common way.

We also worked together with Banshee team to make this a standard on linux desktop as well (Kudos to Philip van Hoof for it). You can read the spec in http://live.gnome.org/MediaArtStorageSpec

So, what does this mean in practice?
1. Tracker and hildon-thumbnailer do a lot of work for you in advance
- Tracker gets the embedded album art automatically
- Hildon-thumbnailer makes the thumbnails in advance in freedesktop.org standard
- Heuristic search is used in tracker and hildon-thumbnailer as specified in the media-art spec.
2. You can extend hildon-thumbnailer with content source plugins that download missing covers from the internet.
3. You can handle the album art all by yourself and just save some time if the condition 1. has hit its mark.

The option 2. is of course the preferred way of handling the album art, but heck: this ain't a perfect world, and I'm doing things a bit dirty, so I'll go for route 3. If the condition 2. would apply and an internet download plugin is available, I would change ukmp to depend on that package and thus, the following code would (mostly) not be needed at all.

However, quite a bit of it is useful. First of all: How is the album art filename (to the full version) calculated? I have nice copy paste functions here. Feel free to use them as is under any license.

You will need unicodedata and md5 as non-usual dependencies, so:

import md5
import unicodedata



Then the functions. First we create the album art filename as specified in the standard. The following function handles that conveniently for you. Now, it's up to you on how to use that. You can either just depend on whatever tracker and hildon thumbnailer have created for you (with or without the plugins), or, as ukmp does, which is, that if first checks whether the album art exists, if not, it downloads it from the internet.

coverlocation=homedir+"/.cache/media-art/"

def getCoverArtFileName( album ):
"""Returns the cover art's filename that is formed from the album name."""
albumString=dropInsideContent(album,"[","]" )
albumString=dropInsideContent(albumString,"{","}" )
albumString=dropInsideContent(albumString,"(",")" )
albumString=albumString.strip('()_{}[]!@#$^&*+=|\\/"\'?<>~`')
albumString=albumString.lstrip(' ')
albumString=albumString.rstrip(' ')
albumString=dropInsideContent(albumString,"{","}" )
albumString=albumString.lower()
albumString=string.replace(albumString,"\t"," ")
albumString=string.replace(albumString," "," ")

try:
albumString=unicodedata.normalize('NFKD',albumString).encode()
albumString=albumString.encode()
print albumString
except:
try:
albumString=albumString.encode('latin-1', 'ignore')
albumString=unicodedata.normalize('NFKD',albumString).encode("ascii")
albumString=str(albumString)
print albumString
except:
albumString="unknown"
if len(albumString)==0: albumString=" "

albumMD5=md5.new(albumString).hexdigest()
emptyMD5=md5.new(" ").hexdigest()
albumArt=coverlocation+"album-"+emptyMD5+"-"+albumMD5+".jpeg"
return albumArt


def dropInsideContent(s, startMarker, endMarker):
startPos=s.find(startMarker)
endPos=s.find(endMarker)
if startPos>0 and endPos>0 and endPos>startPos:
return s[0:startPos]+s[endPos+1:len(s)]
return s




Ok, great, now we have the full version. But, as ukmp needs mostly the thumbnail version, we need the filename to the thumbnail itself.


thumbnailLocation=homedir+"/.thumbnails/normal/"
def getCoverArtThumbFileName( album ):
artFile=getCoverArtFileName(album)
thumbFile=thumbnailLocation+md5.new(artFile).hexdigest()+".jpeg"
return thumbFile


If it happens that the thumbnail does not exist (e.g. wasn't created, has been removed or whatnot), you have a few options:
1. you can create the thumbnail yourself (I'll give an example soon for that)
2. you can request hildon-thumbnailer to create it for you

For the first option, you can just call hildon-thumbailer on the dbus:
https://stage.maemo.org/svn/maemo/projects/haf/trunk/hildon-thumbnail/daemon/thumbnailer.xml

I am not using the method myself at the moment, so here is a quick example. The method is not blocking, so proper use would need to also receive the finished signal from h-t with the thumbnailHandle property. Of course, you can also be polling to see when it has been generated. Usually in non congested situation, this is going to be some tenths of a second. If there is congestion, the content is handled lifo fashion in h-t.


import dbus, time
filename="file:///user/home/.images/01.jpg"
bus = dbus.SessionBus()
handle=time.time()
thumbnailproxy = bus.get_object('org.freedesktop.thumbnailer','/org/freedesktop/thumbnailer/Generic')
thumbnailHandle=thumbnailproxy.Queue([filename],["image/jpeg"],dbus.UInt32(handle))


I'm scaling inline in ukmp. I'm using PIL to scale down the image. It's slower than using pygame (or h-t), but looks better, as it has good anti-aliasing. Anyway, it's once in a lifetime happening, so it's ok to take a while. Here we are also using the above created functions (wehey). I'm using freedesktop org standard size: normal, which is 128x128. Be aware that the media player in Fremantle uses 124x124, so I might switch to that resolution as well. The coverlocation will then also switch from '~/.thumbnails/normal' to '~/.thumbnails/cropped'.

import PIL
thumbFile=getCoverArtThumbFileName(album)
fullCoverFileName=getCoverArtFileName(album)
if (os.path.exists(fullCoverFileName)):
thumbFile=getCoverArtThumbFileName(album)
fullCoverFileName=getCoverArtFileName(album)
image = Image.open(fullCoverFileName)
image = image.resize( THUMBNAIL_SIZE, Image.ANTIALIAS )
thumbFile=thumbFile
image.save( thumbFile, "JPEG" )

Sunday, April 26, 2009

A little tracking for the people waiting for Fremantle

I haven't been blogging much about Fremantle yet. But, yesterday there was a question on the maemo developers mailinglist about what files are indexed to the trackers metadata database, so I though to clear out that issue and also to tell a bit about how your app can use tracker.

So, to answer that question first: Tracker tracks the user home and any mounted media that is attached to the device. For the memory cards, it retains a stack of 3 cards in it's database, so, you can change card A to card B, and back to card A and tracker won't need to reindex the content of the cards. And yes, you can be then swithing to B, to A, to B and you won't lose any data. The amount of cards to support is a configuration option, but by default it's set to 3. So, internal card=1, external card1=2 and then you have the one more as exteral card2. For a third external card, the device will need to flush the oldest seen card away from it's indexes.

There was some concern also as to whether applications can put sound effects and pixmaps into the cards and to make sure they won't be indexed. Well, to this, we have two solutions:
1. put the files to a folder that is hidden (so, it has a "." in the beginning of the folder name) - tracker won't index any hidden folders by default
2. Add the folders to trackers blacklist file

I recommend the solution 1 for multiple reasons.
1. it's simple.
2. user doesn't have any reason to see application data anyway, so this way it'll be also hidden in the file manger. Just make sure your app will flush the data on uninstall of your app.

Ok, then a bit on how you can use Tracker.
As you probably have read, I've been on paternity leave, from which, I've taken a bit of time to integrate ukmp to the new Fremantle stack.

So, first thing I did was, I replaced my own indexing code with code to load all music metadata from tracker database. Loading of this data on startup takes almost no time and tracker also does sorting of the data really easily for me. Not that sorting would actually be any issue in python, nice anyway.

On startup, I hear you saying? Why not on demand? Sure, that would be an option, just happens that how ukmp was built, it's easier for me to get all the content on startup and not on demand. Both are fine. I could write a small comment on how to do stuff on demand as well, but let's start with this.

We'll need to use two interfaces: search and metadata

Corresponding dbus introspection files are: search and metadata

You can find the whole dbus introspection from here


Let's start with defining the needed proxy objects:

import dbus
bus = dbus.SessionBus()
searchproxy = bus.get_object('org.freedesktop.Tracker','/org/freedesktop/Tracker/Search')
metadataproxy=bus.get_object('org.freedesktop.Tracker','/org/freedesktop/Tracker/Metadata')

#Ok, let's then get all music files and for those, the artist, album, title and track# sorted by artist

metadata=searchproxy.Query(-1, "Music", ["Audio:Artist","Audio:Album","Audio:Title","Audio:TrackNo"],"", dbus.Array([], signature='s') ,"",False,["Audio:Artist"], False,0, 40000)

#Now that we have the data, we'll just add it to the internal structures

for songItem in metadata:
fileUrl=songItem[0]
artist=songItem[2]
album=songItem[3]
song=songItem[4]
track=songItem[5]
self.appendSong(track, album, artist, song, fileUrl)



Nice and simple. Now we have the data. This saved me about 300 lines of code, plus multiple library dependencies and tons of headache.

Of course, with my approach of loading everything on startup, I need to update the data when the data changes, but for this, tracker provides a really nice signal that looks like this:


signal sender=:1.15 -> dest=(null destination) serial=403 path=/org/freedesktop/Tracker; interface=org.freedesktop.Tracker; member=ServiceStatisticsUpdated
array [
array [
string "Files"
string "2320"
]
array [
string "Music"
string "543"
]
]

I won't show the implementation on how to keep the data update on this blog post, I'll save it for a future blog post. I'll instead now tell how to keep Tracker up-to-date on usage of the files. All media players on Fremantle should either use MAFW or do the following so that we would all be happy campers no matter which media player user uses.

When you are playing a music file, please notify tracker of the play event. I do so at the end of a track, but your heuristic may vary. Firts we get the current playcount, then we add 1 to it, then we set the new playcount and the curren playtime. We set the time in GMT in UTC format, which is rather easy to get in python.

import time
currentCount=metadataproxy.Get("Music",currentPlayFile, ["Audio:PlayCount"])
newcount=1
if len(currentCount[0])>0: newcount=int(currentCount[0])+1
currentTimeUTC=time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime())
metadataproxy.Set("Music",currentPlayFile, ["Audio:PlayCount","Audio:LastPlay"],[str(newcount),currentTimeUTC])


Ah, now we have the data in tracker, we are able to update the playcounts and playtimes so that all music players can benefit from the data. In my next blog post, I'll tell how the album art can be handled in common way across the platform. I'll tell you how you should do it and I'll tell you how I do it now (which might not be the case I will do when the device has been out for a while).

Then I'll make a blog about how to make dynamic lists, e.g. to list most popular tracks, most recently added and the most recently played tracks.

Then, to top this, I'll let you know how the signaling can be used to keep your internal data structures up-to-date, in case you are not using on-demand loading of the data.

edit: fixed typo as noticed by timeless

Monday, April 20, 2009

I'm a daddy!



Almost exactly a week ago I became a dad of a very sweet little girl. Her hello world message will be coming a bit later on when a name has been bestowed upon her.

I am the happiest man on the planet.

Thursday, November 27, 2008

UKMP 1.83 released to repositories

A week of tuneups and I feel confident putting the portrait ukmp to maemo extras. It's there now available as update to previous users of ukmp.Album art downloading was greatly improved on version 1.82, so 1.83 will remove your existing cover art upon install. 1.83 added the final touches like play/pause button to work properly and some layout tuneups. Shuffle logic is still horrible and I really need to revisit that for the next version.



Other known irritations are that only fisrt 35 songs are visible on each album, so if you have e.g. Rolling Stones top 500 songs as one album, you'll be very dissatisfied with ukmp. I'm trying to bridge the gap between what is now and having 500 songs on an album, but it's not going to work on the next release. Also, everyone should be plesed to notice the exit button on top right corner, with me being the most pleased, because perhaps this will stop emails to me asking how to close ukmp. 1.82 also came with proper support for collection albums. Now the song artist is displayed in the song list view for collection albums. It's simple, yet very rewarding feature.

No more words: Click here to install .

Oh, I'll be posting a new video for the next version. Check out my previous post for the 1.80 release video.

Saturday, November 22, 2008

UKMP 1.80 and Zen of music player

Long time in the making bears fruit. I'm publishing ukmp 1.8 now. It's beta, I have worked today six or seven hours on it, which is probably more I've done to it for a few months. I've been doing some testing, but as you can imagine, it needs a bit more tests before I feel confident putting it to maemo extras repositories and download pages.

However, what is there is something I'm pretty proud of. It's now the music player I always wanted it to be. Very easy to use, very smooth and now that it's in portrait mode, it's finally usable with one hand. Mind you, the optimal hand is left hand.

What is so very nice of this portrait mode is that it works without any support from the X. I made it with the xrandr support initially, but later coded the version that doesn't require the x changes, because I know only a handful of people who have it installed (even though I do love the portrait mode on most of the applications, especially the browser).


Ah, back to ukmp. Well, Here is a camera grab playing on n800. And of course, it's always nice to give you guys a short video as well. I'm really a bit tired, so I only made a one shot presentation. I hope you guys forgive the shaky cam.


If you want to try the latest ukmp, please install first ukmp 1.7 from: Maemo Downloads

and then update it with this link: ukmp 1.83


(or check if later releases already exist: ukmp releases )



Zen of UKMP

UKMP is trying to be a great music player with as little features as possible in the limited space a mobile device has. There is one way to access your music : by album. All music is sorted with one criteria: artist. This way all of albums of one artist are next to each other. Music collections are bundler as artist VA (various artists).

When you open an album, there is again just one sorting criteria: song number (that is, the original sort on the CD),

If you click on a song, it starts playing. Simple.

Sure, shuffle mode needs to be there. Check.
Party mode / queue needs to exists. Check.

What this model gains is that you are at most two clicks away from playing a file. This is quite different from overly generalized music players. I'm not saying there is anything bad about having to do 8 clicks before a song starts playing, It's just not my cup of tea.

There is work to be done for sure. Volume should be more obvious ( + and - buttons on your device), there should be an exit button. Nothing more really comes to my mind now. Please comment your ideas.

An extra one that might need explaining is delete on click. This mode is convenient for e.g. podcasts or for weeding your music out of filler songs. Don't bug me if you delete accidentally something. It is for a reason the only colored (red) widget in the otherwise black and white UI.

[edit] added the maemo.org dl link
[edit] modified link to point to 1.81 version. Will blog about 1.82 probably at some point. Will publish 1.82 to repositories. Please let me know of any bugs in 1.81.
[edit] modified link to 1.83. Please let know of ny 1.83 bugs. Perhaps 1.83 or 1.84 will go to repos.
[comment] ukmp code is pretty bloaded. 3500 loc.