The Field Registry and Transformers

Creating a field registry and/or a custom Transformer object is an
easy yet flexible way to map data structures between desktopcouch and
existing applications.

>>> from desktopcouch.records.field_registry import (
...     SimpleFieldMapping, MergeableListFieldMapping, Transformer)
>>> from desktopcouch.records.record import Record

Say we have a very simple audiofile record type that defines 'artist'
and 'title' string fields. Now also say we have an application that
wants to interact with records of this type called 'My Awesome Music
Player' or MAMP. The developers of MAMP use a data structure that has
the same fields, but uses slightly different names for them:
'songtitle' and 'songartist'. We can now define a mapping between the
fields:

>>> my_registry = {
...     'songartist': SimpleFieldMapping('artist'),
...     'songtitle': SimpleFieldMapping('title')
...     }

and instantiate a Transformer object:

>>> my_transformer = Transformer('My Awesome Music Player', my_registry)

If MAMP has the following song object (a plain dictionary):

>>> my_song = {
...     'songartist': 'Thomas Tantrum',
...     'songtitle': 'Shake It Shake It'
...     }

We can have the transformer transform it into a desktopcouch record
object:

>>> AUDIO_FILE_RECORD_TYPE = 'http://example.org/record_types/audio_file'
>>> new_record = Record(record_type=AUDIO_FILE_RECORD_TYPE)
>>> my_transformer.from_app(my_song, new_record)

Now we can look at the underlying data:

>>> new_record._data #doctest: +NORMALIZE_WHITESPACE
{'record_type': 'http://example.org/record_types/audio_file',
 'title': 'Shake It Shake It',
 'artist': 'Thomas Tantrum'}

You might think that this doesn't really help all that much and that
the code you would have had to write to do this yourself would not
have been all that much bigger than using the Transformer and you'd be
right, but this is not all the transformers do. Let's say the song in
MAMP also has a field 'number_of_times_played_in_mamp':

>>> my_song = {
...     'songartist': 'Thomas Tantrum',
...     'songtitle': 'Shake It Shake It',
...     'number_of_times_played_in_mamp': 23
...     }

Obviously that is not a field defined by our record type, since it is
exceedingly unlikely that any other application would be interested in
this data. Let's see what happens if we run the transformation with
this field present, but undefined in the field registry:

>>> new_record = Record(record_type=AUDIO_FILE_RECORD_TYPE)
>>> my_transformer.from_app(my_song, new_record)

>>> new_record._data #doctest: +NORMALIZE_WHITESPACE
{'record_type': 'http://example.org/record_types/audio_file',
 'title': 'Shake It Shake It',
 'application_annotations': {'My Awesome Music Player': {'application_fields': {'number_of_times_played_in_mamp': 23}}},
 'artist': 'Thomas Tantrum'}

The transformer, when it encountered a field it had no knowledge of,
assumed it was specific to this application, and instead of ignoring
it, stuffed it in the proper place in application_annotations. That's
already quite useful.

Let's try something a little trickier and more contrived. Say MAMP
annotates each song in some other interesting ways: let's say it
allows three very specific tags on each song:

>>> my_song = {
...     'songartist': 'Thomas Tantrum',
...     'songtitle': 'Shake It Shake It',
...     'number_of_times_played_in_mamp': 23,
...     'tag_vocals': 'female vocals',
...     'tag_title': 'shaking',
...     'tag_subject': 'talking'
...     }

Our record type is a little more enlightened, and allows any number of
tags, in a field 'tags', where each tag has a field 'tag' and and a
field 'description'. It would be nice if we could keep a mapping
between the tags that MAMP cares about, and the ones in our
record. We'll have to do just a little more work, but we can. We'll
make a new field_registry, and instantiate a new transformer with it:

>>> my_registry = {
...     'songartist': SimpleFieldMapping('artist'),
...     'songtitle': SimpleFieldMapping('title'),
...     'tag_vocals': MergeableListFieldMapping(
...         'My Awesome Music Player', 'vocals_tag', 'tags', 'tag',
...         default_values={'description': 'vocals'}),
...     'tag_title': MergeableListFieldMapping(
...         'My Awesome Music Player', 'title_tag', 'tags', 'tag',
...         default_values={'description': 'title'}),
...     'tag_subject': MergeableListFieldMapping(
...         'My Awesome Music Player', 'subject_tag', 'tags', 'tag',
...         default_values={'description': 'subject'}),
...     }

>>> my_transformer = Transformer('My Awesome Music Player', my_registry)
>>> new_record = Record(record_type=AUDIO_FILE_RECORD_TYPE)
>>> my_transformer.from_app(my_song, new_record)

Since _data will now contain lots of uuids to keep references intact,
it's less readable, and a less clear example, so I'll show you what
using the higher level API results in:

>>> [tag['tag'] for tag in new_record['tags']]
['shaking', 'talking', 'female vocals']
>>> [tag['description'] for tag in new_record['tags']]
['title', 'subject', 'vocals']

Let's say we append a tag:

>>> new_record['tags'].append({'tag': 'yeah yeah no'})

and we do the same thing:

>>> [tag['tag'] for tag in new_record['tags']]
['shaking', 'talking', 'female vocals', 'yeah yeah no']
>>> [tag.get('description') for tag in new_record['tags']]
['title', 'subject', 'vocals', None]

and say we change the first tag:

>>> new_record['tags'][0]['tag'] = 'shaking it'

and now look at transforming in the other direction:

>>> new_song = {}
>>> my_transformer.to_app(new_record, new_song)
>>> new_song  #doctest: +NORMALIZE_WHITESPACE
{'tag_title': 'shaking it',
 'tag_subject': 'talking',
 'tag_vocals': 'female vocals',
 'songtitle': 'Shake It Shake It',
 'songartist': 'Thomas Tantrum',
 'number_of_times_played_in_mamp': 23}

We see that we got the data that was in the original song, except with
the tag_title value changed to 'shaking it', exactly as we'd expect'.

Many more things are possible by creating new Transformers and/or
FieldMapping types. I'll give one last example. Let us say that our
record_type defines a rating field that's a value between 0 and
100. Let's also say that MAMP stores a string with anywhere between
zero and five stars.

>>> class StarIntMapping(SimpleFieldMapping):
...     """Map a five star rating system to a score of 0 to 100 as
...        losslessly as possible.
...     """
...
...     def getValue(self, record):
...         """Get the value for the registered field."""
...         score = record.get(self._fieldname)
...         stars = score / 20
...         remainder = score % 20
...         if remainder >= 5:
...             stars += 1
...         return "*" * stars
...
...     def setValue(self, record, value):
...         """Set the value for the registered field."""
...         if value is None:
...             self.deleteValue(record)
...             return
...         star_score = len(value) * 20
...         score = record.get(self._fieldname)
...         if score is None or abs(star_score - score) > 5:
...             record[self._fieldname] = star_score
...         # else we keep the original value, since it was close
...         # enough and more precise

And we make a registry and a transformer:

>>> my_registry = {
...     'songartist': SimpleFieldMapping('artist'),
...     'songtitle': SimpleFieldMapping('title'),
...     'stars': StarIntMapping('score'),
...     }
>>> my_transformer = Transformer('My Awesome Music Player', my_registry)

Create a song with a rating:

>>> my_song = {
...     'songartist': 'Thomas Tantrum',
...     'songtitle': 'Shake It Shake It',
...     'stars': '*****',
...     'number_of_times_played_in_mamp': 23
...     }

>>> new_record = Record(record_type=AUDIO_FILE_RECORD_TYPE)
>>> my_transformer.from_app(my_song, new_record)
>>> new_record['score']
100

And, I don't know if you've ever heard the song in question, but that
is in fact correct! ;)
