December 09, 2005

The Aperture Database Structure

I'm working on decoding the Aperture database structure. Just playing around in the database has been very interesting.
These tables are incomplete. However, they do contain the columns that I used in my test queries. Update: I just found a cool tool called SQLite Database Browser that makes it much easier to look around the db than using the command line sqlite3 tool.
The Tables
Table NameDescription
ZRKARCHIVE
ZRKARCHIVERECORD
ZRKARCHIVEVOLUME
ZRKFILEInformation about the on-disk files
ZRKFOLDER
ZRKIMAGEADJUSTMENTImage adjustments made to versions
ZRKMASTERInformation about master images
ZRKPERSISTENTALBUM
ZRKPROPERTYIDENTIFIERProperty types (and each keyword!?)
ZRKSEARCHABLEPROPERTYLinks between properties and versions
ZRKVERSIONVersions of all masters
Searchable Property
Attribute NameDescription
Z_PKA unique ID for this property link
ZVERSIONLink to an image version in the ZRKVERSION table
ZPROPERTYIDENTIFIERLink to the a property identifier in ZRKPROPERTYIDENTIFIER
Property Identifier
Attribute NameDescription
Z_PKA unique ID for this property description
ZPROPERTYKEYA description of this property
ZPROPERTYTYPE2=Keyword, 3=EXIF, 5=Aperture, 7=Timezone
Version
Attribute NameDescription
Z_PKA unique ID for this version
ZFILELink to the original file in ZRKFILE
ZMASTERLink to the master image in ZRKMASTER
ZDATELASTSAVEDINDATABASETime the image was last modified (NSDate)
Folder (projects and albums are considered folders)
Attribute NameDescription
Z_PKA unique ID for this folder
ZLIBRARYRELATIVEPATHPath relative to the root of the library
Master
Attribute NameDescription
Z_PKA unique ID for this master
ZPROJECTA link to the project this master is part of in ZRKFOLDER
Here are some example queries: Print out all properties that can be assigned to an image.
sqlite3 Library.apdb "select ZPROPERTYKEY from ZRKPROPERTYIDENTIFIER"
Print out all images that are part of project 51.
sqlite3 Library.apdb "select * from ZRKMASTER where ZPROJECT=51"
Search all properties of specific type (keywords):
select * from ZRKPROPERTYIDENTIFIER where ZPROPERTYTYPE=2
I would like to reproduce the database query that goes along with slowdown in the filter hud. It involves selecting all property names for properties that are keywords and are assigned to versions of masters that are part of the selected folder. It's hard to say and even harder to read the query that does it. The brute force N^3 algorithm in psuedocode. On my machine a query of the searchable property table keyed on a specific property indentifier takes around 0.009s of user time. For my problem case that would be 600 versions * 118 keywords * 0.009 seconds = 10 minutes.
foreach version in folder
   foreach propertyidentifier of type 'keyword'
      foreach searchableproperty with propkey of propertyidentifier.key
         add keyword to hud
SQL provides a method for doing this in one query that will speed things up significantly. Using the "IN" selection option you can build up combinations of queries. Note that I'm not even using the speed optimized index data in the database and it's still really fast.
Sean-Houghtons-Computer:~/Desktop seanhoughton$ time sqlite3 Library.apdb "
   select ZPROPERTYKEY from ZRKPROPERTYIDENTIFIER where ZPROPERTYTYPE=2 and Z_PK in
      (select ZPROPERTYIDENTIFIER from ZRKSEARCHABLEPROPERTY where ZVERSION in
         (select Z_PK from ZRKVERSION where ZMASTER in
            (select Z_PK from ZRKMASTER where ZPROJECT=61)))"

Sean Houghton
Motorcycle
Crash
Trackday
Joel Pritchett
Stephane Etienne
Sean Houghton   People
Joel Pritchett  People
Stephane Etienne        People

real    0m0.269s
user    0m0.236s
sys     0m0.033s
Here's a selection to count the number of versions that had to be checked for keywords: 664. Opening the filter hud with this folder takes 3 seconds in Aperture, but the database query only takes around 1/4 second of that.
Sean-Houghtons-Computer:~/Desktop seanhoughton$ sqlite3 Library.apdb "
   select count(*) from ZRKVERSION where ZMASTER in
      (select Z_PK from ZRKMASTER where ZPROJECT=61)"

664

Posted by sean at December 9, 2005 10:15 PM

Comments

You wrote: "I would like to reproduce the database query that goes along with slowdown in the filter hud."

Actually, it seems you can do this; "With Mac OS X version 10.4.3 and later, you can use the user default com.apple.CoreData.SQLDebug to log to stderr the actual SQL sent to SQLite. (Note that user default names are case sensitive.)"
From http://developer.apple.com/documentation/Cocoa/Conceptual/CoreData/Articles/cdTroubleshooting.html#//apple_ref/doc/uid/TP40002320-DontLinkElementID_75

Looking forward to your findings!

Cheers,
Peter.

Posted by: Peter at January 18, 2006 02:39 PM

Did you notice the disclaimer:

Using this information for reverse engineering to facilitate direct access to the SQLite file is not supported. It is exclusively a debugging tool.

:-)

Posted by: Allan at February 7, 2006 02:44 PM

Starting Aperture with the following command line rules.

/Applications/Aperture.app/Contents/MacOS/Aperture -com.apple.CoreData.SQLDebug 1

Thanks for the tip Allen!

Posted by: Sean Houghton at April 5, 2006 11:14 PM

Post a comment




Remember Me?


© Sean Houghton