life on hold

AWS “down”

Means Trello is down.

Means I can’t do any work...
(or rather, I don’t know what work to do...)

Oh the joys of cloud-living....

---

For all my social networking posts, see my Google+ page

my Trello backup code

Back in August I blogged about Trello, a great browser-based list manager.  I’ve been using it constantly ever since. I use Evernote to manage the past, and Trello to manage the future.

In that Trello blog post, I mentioned I written a bit of Python to backup my boards. Yesterday PK asked in a comment for that code.  I thought briefly of putting it up on something like GitHub, but the code is so short, I decided I’d give it here instead.

I also thought about refactoring the code a bit first.  I wrote this a while ago, and I know a bit more Python now.  But it works.  So I’ll leave any refactoring until I have to make some changes.  (Well, okay, maybe I made a few changes to remove the worst uglinesses…)

First, there are some libraries to import.

import json
import unicodedata
import urllib2
import datetime

I strip the unicode out of the Trello data, for simplicity.  (One day I will spend the time to really understand all the intricacies of unicode in HTML, but for now, out it goes.)  As usual, Stack Overflow was of invaluable help in finding a way to do this.

def stripUnicode(ustr):
    # translate unicode (accents) to non-accented chars
    ustr = unicodedata.normalize('NFKD', unicode(ustr))
    ustr = u"".join([c for c in ustr if not unicodedata.combining(c)])
    
    # translate whole string to ascii
    return ustr.encode('ascii', 'ignore')

The overall structure of the code is simple: loop over the boards; for each board, print an HTML header, print the lists (which involves printing their cards), then print an HTML footer.

The HTML header involves a title, importing the CSS style file, and a bit of stuff to control the width (a fudge, frankly, but this is for me, not general purpose code!).

def print_header(fhtm, board, lists):
    print >> fhtm, '<html>'
    print >> fhtm, '<head>'
    print >> fhtm, '  <title>' + board['name'] + '</title>'
    w = len([lst for lst in lists if not lst['closed']])*318
    print >> fhtm, '  <style type="text/css">'
    print >> fhtm, '    @import url(trello.css);'
    print >> fhtm, '    body {min-width: ' + str(w) + 'px;}'
    print >> fhtm, '  </style>'
    print >> fhtm, '</head>'
    print >> fhtm, '<body>'

Printing the lists involves looping over the lists, and for each open list printing a heading, and printing the cards in the list.

def print_lists(fhtm, lists, cards, checklists):
    for trello_list in [lst for lst in lists if not lst['closed']]:
        print >> fhtm, '<div class="list">'
        print >> fhtm, '<H2>' + trello_list['name'] + '</H2>'
        print_cards(fhtm, cards, checklists, trello_list['id']) 
        print >> fhtm, '</div>'

Printing the cards involves looping over the list of cards, and for each card, printing its due data, description, and checklists (including strike-through for completed items).  There is potentially more information in a card, but that’s all I currently use.  If you use more, you will have to update this bit, but the structure of what’s needed is fairly obvious.

def print_cards(fhtm, cards, checklists, id_list):
    for trello_card in [c for c in cards if c['idList'] == id_list]:
        print >> fhtm, '<div class="card" >'
        print >> fhtm, '<H3>' + stripUnicode(trello_card['name']) + '</H3>'
        print_card_due(fhtm, trello_card)    
        print_card_desc(fhtm, trello_card)    
        for id in trello_card['idChecklists']:
            print_checklist(fhtm, checklists, id)    
        print >> fhtm, '</div>'
    
def print_card_desc(fhtm, card):
    if card['desc'] != "":
        print >> fhtm, '<P>' + stripUnicode(card['desc'])
        
def print_card_due(fhtm, card):
    if card['due']:
        print >> fhtm, '<UL><LI type="square"><I>due</I> ' + card['due'][:10]
                     + '</UL>'
        
def print_checklist(fhtm, checklists, id):
    checklist = [c for c in checklists if c['id'] == id][0]
    print >> fhtm, '<H4>' + checklist['name'] + '</H4>'
    print >> fhtm, '<UL>'
    items = checklist['checkItems']
    items.sort(key=lambda x: x['pos'])   # pos = screen posn, for list order
    for item in items:
        print_checklist_item(fhtm, item)
    print >> fhtm, '</UL>'
    
def print_checklist_item(fhtm, item):
    if item['state'] == "complete":
        print >> fhtm, '<LI><I><DEL>' + stripUnicode(item['name'])
                     + '</DEL></I>'
    else:
        print >> fhtm, '<LI type="circle">' + stripUnicode(item['name'])

The HTML footer is trivial.

def print_footer(fhtm):
    print >> fhtm, '</body>'
    print >> fhtm, '</html>'

There are some secret keys and tokens needed, which you get when you register at Trello, to get access to your own boards.

key = 'a 32 hexdigit string'
token = 'a 64 hexdigit string'

I have hardwired in the various boards I want to backup. The 24 hexdigit string can be found right at the beginning of the board’s JSON file, so you have to download that manually, and copy the string from it. But you only have to do that once.

board_dict = {
    'board1': 'a 24 hexdigit string',
    'board2': 'a 24 hexdigit string',
    ...,
    'boardn': 'a 24 hexdigit string'
}

Then, finally, there’s the loop over the boards in this dictionary: reading in the board data, which requires reading the board, its lists, its cards, and its checklists separately, then printing out the relevant HTML, including the date of the backup:

for filename, board_id in board_dict.iteritems():
    print filename
    
    trello = 'https://api.trello.com/1/boards/' + board_id
    key_token = key + '&token=' + token

    get_url = urllib2.urlopen(trello + '?key=' + key_token)
    board = json.loads(get_url.read())
    get_url = urllib2.urlopen(trello + '/lists?key=' + key_token)
    lists = json.loads(get_url.read())
    get_url = urllib2.urlopen(trello + '/cards?key=' + key_token)
    cards = json.loads(get_url.read())
    get_url = urllib2.urlopen(trello + '/checklists?key=' + key_token)
    checklists = json.loads(get_url.read())

    fhtm = open(filename + '.htm', 'w')
    print_header(fhtm, board, lists)
    print >> fhtm, '<H1>' + board['name'] + ' (' + str(datetime.date.today())
                 + ')</H1>'
    print_lists(fhtm, lists, cards, checklists)
    print_footer(fhtm)
    fhtm.close()

Then to get this to format looking a bit like a Trello board, we just need a bit of simple CSS for the list and card classes used above:

.list {
  margin: 0 3px 1em 3px;
  padding: 0 4px 4px 4px;
  background-color: #eee;
  border: 1px #aaa solid;
  border-radius: 4px;
  float: left; 
  width: 300px;
}

.card {
  margin-bottom: 4px;
  padding: 0 0.6em 0 0.6em;
  border: 1px #aaa solid;
  border-radius: 4px;
}

I’ve also modified the various headings styles to make things look nice, but I’ll leave that as an exercise for the student!

Python makes writing this kind of stuff so straightforward.  It took me far longer to work out the JSON format and API of the Trello stuff than to write the code.

If I was writing this today, I might think about using a library like Beautiful Soup, or, given how straightforward it was, I might just do it this way again.

Trello for tablets

I use Trello daily, and now my tablet version no longer looks like a really big phone version.  Yay!

---

For all my social networking posts, see my Google+ page

Trello

I’ve read and enjoyed some of Joel Spolsky’s Joel on Software posts, in dead tree format.  So the other day, I was link surfing, followed a link to his site, and ended up reading How Trello is Different, a mere 18 months after it was originally posted.

I was taken with the sentence: “The great horizontal killer applications are actually just fancy data structures.” Trello is a “team coordination system” that is a fancy data structure comprising a “list of lists”. (Actually, it is a list of lists (boards) of lists of lists (cards) of lists of lists (checklists); or tree, as we say in the trade.)

The point is that any killer app can be used for multiple different purposes, mostly not thought of by the original developer: Excel, for example, isn't just a calculator, it’s used for almost anything that needs tables.

So Trello is good for any task that needs lists of lists – including to do lists. I hadn’t consciously been looking for a to do list manager, as I was happily using Toodledo. Toodledo is great for to do lists, but I was finding that maybe I wanted something with a little more structure: not just a single to do list, but ... a list of lists! The pro version of Toodledo has more capabilities, but that requires a subscription. So, finding Trello felt like serendipity, and I decided to give it a try (it’s free, and web-based, like so many things nowadays; there is a pro version, but it looks well beyond my needs).

And it’s really great. I now have (a list of) seven boards, the major list category: think project, or similar large slice of life. I have one for teaching in general, one for a specific module I’m teaching, one for research, one for tracking my research students, one for travel and events, one for miscellaneous to do items, and one for template lists.

Each board has several lists: categories of tasks or items in that board. So my specific module teaching board has a list for lectures, for practicals, for seminars, for the web site, and for the assessment.

These lists are just called lists. They are lists of cards. Each card is a task or other specific unit of work. So my practicals list has a card for the preparation I need to do for each individual practical, and a further card for things I need to do for all the practicals. Lists, and cards within lists, can be dragged around to reorder things how you want.

It doesn’t stop there. There’s loads of data that can be associated with a card. Flip the card over, and there’s a free text field, a due date, coloured tags, a list of activities (things that have happened with the card), a list of checklists, and more. Each checklist is itself a list of check items. And that’s as far down as the lists go, I think.

For the group working part, multiple people can be members of a board, can be assigned different cards, and so on.

I’ve been using Trello for a few weeks now, and it fits in very smoothly to my workflow. There’s an “all cards” view that you can use to show cards with a due date, in date order, so I can easily see what’s urgent.

I liked it enough to want to use it permanently. But there was one issue: backup. The data is stored on the Trello server. If I lost my to do list, I would be in deep trouble. I went to the help page, searched on “backup”, and found:

To export your data, simply open a card or open the sidebar Menu, then click the ‘Share, Print, and Export’ button, and you’ll find export options. Currently, we allow export to JSON, which is a format that lends itself well to Trello’s specific data storage model.

Yes, but if Trello disppears, how will that help me? (I have no reason to believe Trello will disappear. But, I'm paranoid about backups.) I clicked the export button, and was delivered a .json file. I decided to see how easy it would be to extract my to do life from the file. I wrote a little Python and CSS, and converted the relevant bits of the file to HTML:

The CSS just formats to HTML to look like the original Trello board. It looks a bit different, because I added the info from the back of the card to the front in this backup display.

So that’s okay, then. I can recover the data, provided I remember to backup the boards. But that’s a bit boring, having to backup seven boards individually. Boring backups are no good: they tend to get skipped. So I went back to the site to see if there was a command to backup all the boards in one go. No, but there is an API; I’d seen that originally, but forgotten about it.

The API lets you get and set your data remotely. So I modified my Python script to read from URLs rather than JSON files, and voila, backup of the entire set of boards with a single click.

So now I’m happy, and will go over to Trello. And I’m not even scratching its surface yet: I haven’t made any use of the multi-person aspects so far. I may think about doing that on a new project that’s about to start up.