Sunday 2 March 2014

my Trello backup code

Back in August I blogged about Trello, a great browser-based list manager.  I’ve been using it constantly ever since. I use Evernote to manage the past, and Trello to manage the future.

In that Trello blog post, I mentioned I written a bit of Python to backup my boards. Yesterday PK asked in a comment for that code.  I thought briefly of putting it up on something like GitHub, but the code is so short, I decided I’d give it here instead.

I also thought about refactoring the code a bit first.  I wrote this a while ago, and I know a bit more Python now.  But it works.  So I’ll leave any refactoring until I have to make some changes.  (Well, okay, maybe I made a few changes to remove the worst uglinesses…)

First, there are some libraries to import.
import json
import unicodedata
import urllib2
import datetime
I strip the unicode out of the Trello data, for simplicity.  (One day I will spend the time to really understand all the intricacies of unicode in HTML, but for now, out it goes.)  As usual, Stack Overflow was of invaluable help in finding a way to do this.
def stripUnicode(ustr):
    # translate unicode (accents) to non-accented chars
    ustr = unicodedata.normalize('NFKD', unicode(ustr))
    ustr = u"".join([c for c in ustr if not unicodedata.combining(c)])
    
    # translate whole string to ascii
    return ustr.encode('ascii', 'ignore')
The overall structure of the code is simple: loop over the boards; for each board, print an HTML header, print the lists (which involves printing their cards), then print an HTML footer.

The HTML header involves a title, importing the CSS style file, and a bit of stuff to control the width (a fudge, frankly, but this is for me, not general purpose code!).
def print_header(fhtm, board, lists):
    print >> fhtm, '<html>'
    print >> fhtm, '<head>'
    print >> fhtm, '  <title>' + board['name'] + '</title>'
    w = len([lst for lst in lists if not lst['closed']])*318
    print >> fhtm, '  <style type="text/css">'
    print >> fhtm, '    @import url(trello.css);'
    print >> fhtm, '    body {min-width: ' + str(w) + 'px;}'
    print >> fhtm, '  </style>'
    print >> fhtm, '</head>'
    print >> fhtm, '<body>'
Printing the lists involves looping over the lists, and for each open list printing a heading, and printing the cards in the list.
def print_lists(fhtm, lists, cards, checklists):
    for trello_list in [lst for lst in lists if not lst['closed']]:
        print >> fhtm, '<div class="list">'
        print >> fhtm, '<H2>' + trello_list['name'] + '</H2>'
        print_cards(fhtm, cards, checklists, trello_list['id']) 
        print >> fhtm, '</div>'
Printing the cards involves looping over the list of cards, and for each card, printing its due data, description, and checklists (including strike-through for completed items).  There is potentially more information in a card, but that’s all I currently use.  If you use more, you will have to update this bit, but the structure of what’s needed is fairly obvious.
def print_cards(fhtm, cards, checklists, id_list):
    for trello_card in [c for c in cards if c['idList'] == id_list]:
        print >> fhtm, '<div class="card" >'
        print >> fhtm, '<H3>' + stripUnicode(trello_card['name']) + '</H3>'
        print_card_due(fhtm, trello_card)    
        print_card_desc(fhtm, trello_card)    
        for id in trello_card['idChecklists']:
            print_checklist(fhtm, checklists, id)    
        print >> fhtm, '</div>'
    
def print_card_desc(fhtm, card):
    if card['desc'] != "":
        print >> fhtm, '<P>' + stripUnicode(card['desc'])
        
def print_card_due(fhtm, card):
    if card['due']:
        print >> fhtm, '<UL><LI type="square"><I>due</I> ' + card['due'][:10]
                     + '</UL>'
        
def print_checklist(fhtm, checklists, id):
    checklist = [c for c in checklists if c['id'] == id][0]
    print >> fhtm, '<H4>' + checklist['name'] + '</H4>'
    print >> fhtm, '<UL>'
    items = checklist['checkItems']
    items.sort(key=lambda x: x['pos'])   # pos = screen posn, for list order
    for item in items:
        print_checklist_item(fhtm, item)
    print >> fhtm, '</UL>'
    
def print_checklist_item(fhtm, item):
    if item['state'] == "complete":
        print >> fhtm, '<LI><I><DEL>' + stripUnicode(item['name'])
                     + '</DEL></I>'
    else:
        print >> fhtm, '<LI type="circle">' + stripUnicode(item['name'])
The HTML footer is trivial.
def print_footer(fhtm):
    print >> fhtm, '</body>'
    print >> fhtm, '</html>'
There are some secret keys and tokens needed, which you get when you register at Trello, to get access to your own boards.
key = 'a 32 hexdigit string'
token = 'a 64 hexdigit string'
I have hardwired in the various boards I want to backup. The 24 hexdigit string can be found right at the beginning of the board’s JSON file, so you have to download that manually, and copy the string from it. But you only have to do that once.
board_dict = {
    'board1': 'a 24 hexdigit string',
    'board2': 'a 24 hexdigit string',
    ...,
    'boardn': 'a 24 hexdigit string'
}
Then, finally, there’s the loop over the boards in this dictionary: reading in the board data, which requires reading the board, its lists, its cards, and its checklists separately, then printing out the relevant HTML, including the date of the backup:
for filename, board_id in board_dict.iteritems():
    print filename
    
    trello = 'https://api.trello.com/1/boards/' + board_id
    key_token = key + '&token=' + token

    get_url = urllib2.urlopen(trello + '?key=' + key_token)
    board = json.loads(get_url.read())
    get_url = urllib2.urlopen(trello + '/lists?key=' + key_token)
    lists = json.loads(get_url.read())
    get_url = urllib2.urlopen(trello + '/cards?key=' + key_token)
    cards = json.loads(get_url.read())
    get_url = urllib2.urlopen(trello + '/checklists?key=' + key_token)
    checklists = json.loads(get_url.read())

    fhtm = open(filename + '.htm', 'w')
    print_header(fhtm, board, lists)
    print >> fhtm, '<H1>' + board['name'] + ' (' + str(datetime.date.today())
                 + ')</H1>'
    print_lists(fhtm, lists, cards, checklists)
    print_footer(fhtm)
    fhtm.close()
Then to get this to format looking a bit like a Trello board, we just need a bit of simple CSS for the list and card classes used above:
.list {
  margin: 0 3px 1em 3px;
  padding: 0 4px 4px 4px;
  background-color: #eee;
  border: 1px #aaa solid;
  border-radius: 4px;
  float: left; 
  width: 300px;
}

.card {
  margin-bottom: 4px;
  padding: 0 0.6em 0 0.6em;
  border: 1px #aaa solid;
  border-radius: 4px;
}
I’ve also modified the various headings styles to make things look nice, but I’ll leave that as an exercise for the student!

Python makes writing this kind of stuff so straightforward.  It took me far longer to work out the JSON format and API of the Trello stuff than to write the code.

If I was writing this today, I might think about using a library like Beautiful Soup, or, given how straightforward it was, I might just do it this way again.

3 comments:

  1. Thank you for posting this! I've tucked it away just in case I ever need it.

    ReplyDelete
  2. Hi Susan, thanks for sharing the code!

    For your readers who're not as comfortable with running Python programs themselves, I'm making http://my.backuphamster.com/trello — an online backup & restore tool for Trello (no programming required). This tool allows you to restore your data to a specific point in time. It's still in "early access" mode, but if you need those features, feel free to give it a try.

    ReplyDelete