At work, we're building an app to edit objects in a database--a classic CRUD application. For now, we're trying out ExtJS as the client-side UI framework. One of the use cases is selecting and editing nested objects, represented in our relational database with foreign keys. Let's call the root object a Task, which consists of nested Goals, which have Steps. Each of those is defined by a model on the backend that is more or less mimicked by an Ext.data.Model on the client-side, and each model has a proxy to a RESTful endpoint on the backend for create/retrieve/update/delete operations. We want to use an Ext.tree.TreePanel for the UI, so we hold the data in an Ext.data.TreeStore. So far so good.

We coded up our prototype, but when a user selects a Task, Ext JS throws this error: Uncaught TypeError: Cannot read property 'internalId' of undefined. Hmm. Everything seems to be working. Our models are loading the correct data. No obvious bugs. A lot of inspecting and googling and reading documentation later, I discover this thread. The key quote:

Read More

My good friend Steve has hosted the lowercase, the monthly reading series associated with 826DC, for three years. Steve has a charming habit of introducing his readers with made-up bios, so in his honor, I asked some lowercase regulars to write fake bios of him and share them at the third anniversary reading on April 4. The results were highly entertaining; thanks to everyone who wrote one!

Here's mine:

Steve Souryal is a group of 15 small islets and rocks in the central equatorial Atlantic Ocean. He lies in the Intertropical Convergence Zone, a region of severe storms. Steve exposes serpentinized abyssal mantle peridotite and kaersutite-bearing ultramafic mylonite on the top of the second-largest megamullion in the world (after the Parece Vela megamullion under Okinotoshima in the Pacific). He is the only location in the Atlantic Ocean where the abyssal mantle is exposed above sea level! In 1986, Steve was designated an environmentally protected area, and since 1998, the Danish Navy has maintained a permanently manned research facility in him. His main economic activity is tuna fishing, and we are incredibly lucky to have him with us tonight.

Apologies to Wikipedia. But somehow, it just feels right.

Background

Some mathematically intense operations that use Numpy/Scipy can run faster with accelerated basic linear algebra subroutine (BLAS) libraries installed on your system (e.g., gensim's corpus processing).

To see what BLAS libraries you are using, do:

1

python -c 'import numpy; numpy.show_config()'

Read More

Today's the third annual Friendsgiving, a Thanksgiving-like pre-Thanksgiving event for a bunch of people who like each other; hence, Friendsgiving. Thanksgiving's always been my favorite holiday so I'm more than happy to celebrate it twice a year. The first two Friendsgivings took place at my house, but because in the spring I traded my room in a cavernous and amply chandeliered group rowhouse for cozier and warmer digs, the honor of hosting this year falls to two friends who're renting an entire lovely house for themselves up in Pleasant Plains. Sweet.

So much for traditions; recent novelties include starting a new job, about which more another time, but basically, I love it; and getting a lesson plan published in Don't Forget to Write, the second volume of lesson plans from 826. The lesson plan, "Busted," aims to make storytellers out of middle schoolers by having them write about a time they got caught doing something they shouldn't have been doing--a theme first cooked up by the folks who led the Get Used to the Seats book project. I wrote about leading the workshops that ultimately became the "Busted" lesson plan more than a year ago--right around the previous Friendsgiving. Hard to believe it's been that long!

An interlude from the recent trend of hardcore Django action:

When I spoke with a female intern this summer, she recounted how, in 2006, the GNOME Project, a free and open source software project, received almost 200 Google Summer of Code applicants. All of them were male. When GNOME advertised an identical program for women, emphasizing opportunities for learning and mentorship instead of tough competition, they received more than 100 highly qualified female applicants for the three spots they were able to fund. What amazed me even more was when she suggested that our own company slogan — “We Help the World’s Best Developers Make Better Software” — might alienate prospective female candidates. That had never occurred to me. But according to our intern, in the world of computer science, “when you hear the phrase ‘the world’s best developers,’ you see a guy.”

From When computer programming was ‘women’s work’.

Thanks to django-registration, I was able to build a working account registration/login system pretty easily. But I wanted to give users the ability to use their existing accounts through popular services such as Facebook, Twitter, etc., rather than have to create yet another account. Here's how I did it.

Sorting Through the Choices

There are a number of reusable Django apps out there to help with registration/login from social media sites. I found this Review of 4 Django Social Auth apps very helpful in sorting out the options. After reading it, I was left to choose between django-social-auth (I originally linked to the wrong app here, but this link is correct) and django-allauth. In the end, I went with django-social-auth (not to be confused with django-socialauth) because a friend had recommended it and because I'd already installed it before I read this article. However, the article's conclusion that django-allauth is best out of the box also seems valid.

Installation

The instructions in django-social-auth's docs are helpful in walking you through available settings and options.

I also found the included example app useful. To use this app, I cloned django-social-auth's git repo, created a virtualenv called django-social-auth, ran pip install -r requirements.txt inside this virtualenv to install all the required apps, ran manage.py syncdb, and finally ran manage.py runserver. Voila, example app is up and running at 127.0.0.1, showing a simple screen with options to login through about a dozen different different services.

API Keys

The first service I tested was Twitter. I use it more than any others, and I already had the API keys for it. I threw my API key and secret key into the example local_settings.py file provided with django-social-auth and tried to log in via the example app. Boom: 401 Unauthorized. I double-checked all my settings and installation and whatnot. Seemed fine.

I turned my attention to the API keys. The ones I had were generated for Readsr, i.e., I entered readsrs.com as the domain when I generated them at dev.twitter.com. But now I was running on localhost, 127.0.0.1, so I suspected the readsrs.com keys wouldn't be valid. I wasn't sure whether Twitter would hand over a new consumer key for 127.0.0.1, or baulk at the request. (It seemed like it should do so, but I hadn't seen any instructions anywhere that said to get a key for your development machine.) Turns out Twitter will happily give you a key for 127.0.0.1. Once I plugged the new keys in, I was able to log in with my Twitter credentials, and just as it should, django-social-auth automatically created an auth.user for this account.

Integrating with Readsr

I followed the instructions again to config my own app, Readsr. To add a login option using Twitter credentials, I put a link to the reversed view that begins the django-social-auth login process for twitter, i.e., `{% url socialauth_begin "twitter" %}`, to my login template. And it worked.

I still need to fix a few oddities. For example, Twitter returns my first and last names together in first_name (or else django-social-auth is concatenating them into that column), and doesn't supply any email address. But the basic functionality is there, and was relatively easy to achieve.

Postscript

The author of the article I linked above had an error using OpenID when using django-social-auth, which is why he preferred django-authall. He filed a bug for the error he got, and I notice that it was closed 15 hours ago (though if you read the comments, it seems it was actually fixed back in mid-July). Good timing.

Background

For Readsr, I need to track events that recur on a particular day of the week (e.g., first Sunday of the month, third Friday of the month). I created a DayOfWeek model to store any particular event's day of the week. It contains a method next_day_of_week() to return a datetime.date object set to the next occurrence of whatever weekday a given event instance is set to (this helps with figuring out when the next occurrence of an event is).

It's easier to show through an example. On Sunday 7/3/2011:

• For an object with DayOfWeek set to Sunday, next_day_of_week() would return 7/3/2011 (current day).
• For DayOfWeek set to Monday, it would return 7/4/2011 (first subsequent Monday).
• For DayOfWeek set to Saturday, it would return 7/9/2011 (first subsequent Saturday).

Sounds simple enough. It seemed like this would be a good place to do my first unit tests.

Unit Testing

To do unit testing, the typical method is to first write test cases and then write code. In this case, I'd already written my code, so I went back and wrote test cases, trying to forget how my code worked.

To write test cases, you have to detail requirements for each method you want to test: input and expected (correct) output. The list of examples for next_day_of_week() I wrote above works for this purpose. But there's a catch: next_day_of_week() calculates the next day of the week relative to the current date, by calling datetime.date.today(). So if I write expected output for 7/3/2011, it will no longer be the correct output on 7/4/2011 or any following day. I needed a way to make datetime.date.today() always spit out my input date when I run tests, yet still continue to function normally outside of testing. Enter mocking.

Mocking

The solution was to mock out the method--to replace the real datetime.date.today() with a fake one that produces the same output no matter what day it is. To accomplish this, I used the powerful Mock library. Specifically, I needed to use the patch decorator. This decorator makes it really easy to replace on particular object within the scope of a particular method.

Before I could patch the today() method, I needed to create my own fake method. It would look like this:

1
2
3

def faketoday()
    return date(2011, 7, 4)

There's a problem, though, when I try to patch (or mock out) the method:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

>>> import mock
>>> def faketoday():
...     return datetime.date(2011, 7, 4)
...
>>> @mock.patch("datetime.date.today", faketoday)
... def testfunc():
...     return datetime.date.today()
...
>>> testfunc()
Traceback (most recent call last):
  File "<console>", line 1, in <module>
  File "/Users/wbert/.virtualenvs/readsr_env/lib/python2.6/site-packages/mock.py", line 561, in patched
    arg = patching.__enter__()
  File "/Users/wbert/.virtualenvs/readsr_env/lib/python2.6/site-packages/mock.py", line 623, in __enter__
    setattr(self.target, self.attribute, new_attr)
TypeError: can't set attributes of built-in/extension type 'datetime.date'
>>>

datetime.date is considered a Python built-in and can't be modified.

Modifying a Class That Can't Be Modified

The trick is to write a child class that can be modified, and thus faked:

1
2
3
4

class FakeDate(date):
	"A fake replacement for date that can be mocked for testing."
	def __new__(cls, *args, **kwargs):
		return date.__new__(date, *args, **kwargs)

All this does is create a class whose constructor returns an instance of its parent's class, date. Usually, this would be pointless, but it's useful here because the new class isn't a built-in and thus can be mocked.

To use it, we simply decorate any test method that calls datetime.date.today() with a patch to replace datetime.date with FakeDate, and we also provide FakeDate a fake today() method that returns only and always the particular we are going to use for testing:

1
2
3
4
5
6
7
8
9
10

class TestDayOfWeek(TestCase):
	"""Test the day of the week functions."""

@mock.patch('series.models.date', FakeDate)
def test_valid_my_next_day_of_week_sameday(self):
	from datetime import date
	FakeDate.today = classmethod(lambda cls: date(2011, 7, 3)) # July 3, 2011 is a Sunday
	new_day_of_week = DayOfWeek.objects.create()
	new_day_of_week.day = "SU"
	self.assertEquals(new_day_of_week.my_next_day_of_week(), date(2011, 7, 3))

A couple things to note: the patch only applies within this particular method, so each method to use a patch must be decorated. Also, the real datetime.date is imported in the method so we can use it inside the fake today() method. We could put this fake today() method inside FakeClass, but making it a lambda (anonymous) method assigned inside the test case gives us the flexibility to set a particular date for each test case.

Namespacing

You may be wondering why the patch decorator takes "series.models.date" as the method to replace instead of "datetime.date". That was how I tried it at first, and I was confused when it didn't work. It seemed as if the patch hadn't taken effect.

Well, it hadn't. That's because within the module being tested (models.py in the series app, or series.models in Python dotted notation), date has been imported like so:

1

from datetime import date

This means that within series.models, date is now available as series.models.date, so that's the name that needs to be mocked out. For more on namespacing when mocking, checked out Mock's Where to patch documentation.

Now we can supply out unit tests with any date we want, ensuring that we know what the results should be and can test against them.

References

Learning how to do this stuff, I posted my first question at Stackoverflow (I ended up answering it myself). I also learned about using a fake class from this question. Finally, the Mock documentation was very helpful as well.

Background

I've been teaching myself Django by writing a web app that tracks reading series in cities: Readsr. A reading series is a kind of recurring event. It defines a time, location, and recurrence rule such as first Monday of the month. Writing a list view to display upcoming readings was easy, but I also wanted to create a calendar view, similar to what Google Calendar provides. That took a little more work.

Existing Solutions

First I searched for existing Django calendar solutions. I found several. Swingtime and django-agenda seemed very well thought out and comprehensive, but were also perhaps overkill for what I needed. django-gencal doesn't appear to be maintained and I had trouble understanding the documentation (though you may have better results, as I am slow).

A Way Forward

I found that Python's calendar module has a built template called HTMLCalendar, which sounded promising. Then I found a couple examples of people inheriting from HTMLCalendar to add data to the calendar display. This sounded right on, so I adapted this code for my reading events.

Problem: Presentation and Content Mixed

I noticed a problem in the code I was adapting. The view was producing HTML. That seemed to violate separation of content and presentation. Shouldn't the HTML be generated in the template? And since the templating language itself isn't powerful enough to generate an HTML table from a list of objects, that meant I needed to write my own template tag. Yikes.

Writing My First Template Tag

Django's documentation made it easy, though. A template tag consists of a few parts. Below is the code; it goes into a subdirectory of the django app called "templatetags."

The first part follows directly from the Django docs: get a register object.

Then I define a function to parse the template tag arguments and return the node (the HTML code from which the page is eventually build). The template syntax is defined here.

Then I define the node itself, which is made thread-safe by storing and retrieving the variables passed through the template tag from a context (again, this is straight from the Django docs).

Then I inherit from HTMLCalendar and redefine the format methods to add the particular reading event data. You could adapt this class to any kind of event that has an associated date/time by changing the groupby lambda function to use whatever field your event object uses to store its date and time (my reading object simply calls it "date_and_time").

Finally, I register this template tag so it is available to templates.

Here's the code.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93

    
from calendar import HTMLCalendar
from django import template
from datetime import date
from itertools import groupby

from django.utils.html import conditional_escape as esc

register = template.Library()

def do_reading_calendar(parser, token):
	"""
	The template tag's syntax is { % reading_calendar year month reading_list %}
	"""

	try:
		tag_name, year, month, reading_list = token.split_contents()
	except ValueError:
		raise template.TemplateSyntaxError, "%r tag requires three arguments" % token.contents.split()[0]
	return ReadingCalendarNode(year, month, reading_list)
	

class ReadingCalendarNode(template.Node):
	"""
	Process a particular node in the template. Fail silently.
	"""
	
	def __init__(self, year, month, reading_list):
		try:
			self.year = template.Variable(year)
			self.month = template.Variable(month)
			self.reading_list = template.Variable(reading_list)
		except ValueError:
			raise template.TemplateSyntaxError
		
	def render(self, context):
		try:
			# Get the variables from the context so the method is thread-safe.
			my_reading_list = self.reading_list.resolve(context)
			my_year = self.year.resolve(context)
			my_month = self.month.resolve(context)
			cal = ReadingCalendar(my_reading_list)
			return cal.formatmonth(int(my_year), int(my_month))
		except ValueError:
			return			
		except template.VariableDoesNotExist:
			return


class ReadingCalendar(HTMLCalendar):
	"""
	Overload Python's calendar.HTMLCalendar to add the appropriate reading events to
	each day's table cell.
	"""
	
	def __init__(self, readings):
		super(ReadingCalendar, self).__init__()
		self.readings = self.group_by_day(readings)

	def formatday(self, day, weekday):
		if day != 0:
			cssclass = self.cssclasses[weekday]
			if date.today() == date(self.year, self.month, day):
				cssclass += ' today'
			if day in self.readings:
				cssclass += ' filled'
				body = ['<ul>']
				for reading in self.readings[day]:
					body.append('<li>')
					body.append('<a href="%s">' % reading.get_absolute_url())
					body.append(esc(reading.series.primary_name))
					body.append('</a></li>')
				body.append('</ul>')
				return self.day_cell(cssclass, '<span class="dayNumber">%d</span> %s' % (day, ''.join(body)))
			return self.day_cell(cssclass, '<span class="dayNumberNoReadings">%d</span>' % (day))
		return self.day_cell('noday', '&nbsp;')

	def formatmonth(self, year, month):
		self.year, self.month = year, month
		return super(ReadingCalendar, self).formatmonth(year, month)

	def group_by_day(self, readings):
		field = lambda reading: reading.date_and_time.day
		return dict(
			[(day, list(items)) for day, items in groupby(readings, field)]
		)

	def day_cell(self, cssclass, body):
		return '<td class="%s">%s</td>' % (cssclass, body)

# Register the template tag so it is available to templates
register.tag("reading_calendar", do_reading_calendar)

Then, here's the view (and a couple helper functions) that gets called with the arguments from the URL, including the year, month, and series to display events for.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58

def named_month(month_number):

	"""
	Return the name of the month, given the number.
	"""
	return date(1900, month_number, 1).strftime("%B")
	
def this_month(request):
	"""
	Show calendar of readings this month.
	"""
	today = datetime.now()
	return calendar(request, today.year, today.month)
	
	
def calendar(request, year, month, series_id=None):
	"""
	Show calendar of readings for a given month of a given year.
	``series_id``
	The reading series to show. None shows all reading series.
	"""
	
	my_year = int(year)
	my_month = int(month)
	my_calendar_from_month = datetime(my_year, my_month, 1)
	my_calendar_to_month = datetime(my_year, my_month, monthrange(my_year, my_month)[1])

	my_reading_events = Reading.objects.filter(date_and_time__gte=my_calendar_from_month).filter(date_and_time__lte=my_calendar_to_month)
	if series_id:
		my_reading_events = my_reading_events.filter(series=series_id)

	# Calculate values for the calendar controls. 1-indexed (Jan = 1)
	my_previous_year = my_year
	my_previous_month = my_month - 1
	if my_previous_month == 0:
		my_previous_year = my_year - 1
		my_previous_month = 12
	my_next_year = my_year
	my_next_month = my_month + 1
	if my_next_month == 13:
		my_next_year = my_year + 1
		my_next_month = 1
	my_year_after_this = my_year + 1
	my_year_before_this = my_year - 1
	return render_to_response("cal_template.html", { 'readings_list': my_reading_events,
														'month': my_month,
														'month_name': named_month(my_month),
														'year': my_year,
														'previous_month': my_previous_month,
														'previous_month_name': named_month(my_previous_month),
														'previous_year': my_previous_year,
														'next_month': my_next_month,
														'next_month_name': named_month(my_next_month),
														'next_year': my_next_year,
														'year_before_this': my_year_before_this,
														'year_after_this': my_year_after_this,
	}, context_instance=RequestContext(request))

And finally, here's the template where we load the template tag and employ it, passing the year, month, and list from the view (you would also want to write some control elements that use previous_year, previous_month, etc. to allow the user to change what the calendar displays, but because I want to wrap this up I'll forgo writing that out):

1
2
3
4
5
6
7
8

{% load reading_tags %}

<div id="calendar">
	{% reading_calendar year month reading_list %}
</div>

Hopefully that makes sense. Enjoy!

You can also see this code (made generic for any kind of event) on django snippets.

Since March, I've been working with DC students from several high schools, including Duke Ellington, Ballou, Wilson and Bell, in evening poetry workshops at 826DC. A bunch of us have been helping out—Mike Scalise, Adam Pelligrini, and Sally Keith (the last two being actual poets, while Mike and I, both prose writers, were faking it the whole time). And of course program director Mariam Al-Shawaf and 826DC Executive Director Joe Callahan did the thankless behind-the-scenes labor to make it all happen.

The students came to us talented and hard-working, and we just helped them along with workshops on poetic forms, poetic sound, and metaphor (this is the one I'm awkwardly teaching in the photo above), and by giving plenty of exercises and time to generate new material. We even had a guest visit from the amazing Carolyn Forche.

All this culminates today in a workshop at the White House, hosted by the Obamas, where the students will work with the likes of Billy Collins, Common, Rita Dove, Aimee Mann, Jill Scott and several other amazing poets to hone their craft even further. Very, very cool. Congrats to 826DC and especially to the students!