RedBlog

Unicode support in Python is better than in C or PHP. True. But that's like saying SVN is better than CVS. It doesn't say much.

Here's one ugglieness: You want to convert something to unicode, no matter what, you're even happy to lose one or two "strange" characters. So, how do you do that?

>>> class X(object):
...  def __unicode__(self): return u"hej"
... 
>>> x = X()
>>> y = "åäö"

>>> unicode(x)
u'hej'

>>> unicode(y)
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 0: ordinal not in range(128)

Fine, that's what we expected. Ok, new try: Reading up on the unicode() constructor, we find the errors= parameter which sounds usefull. Let's give it a try:

>>> unicode(y, errors="replace")
u'\ufffd\ufffd\ufffd\ufffd\ufffd\ufffd'

That worked fine. Sure, we only got \ufffd back, but we didn't care about what exactly we got back, remember? So, how does our x survive this? It doesn't even return any wierd characters so it should work even better!? Riiiiight:

>>> unicode(x, errors="replace")
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
TypeError: coercing to Unicode: need string or buffer, X found

Conclusion: The unicode() constructor is rather random in its choices of operation, and not the slightest orthogonal. This is however an easy problem to work around with a wrapper function. As are all problems in Python. Uggly, but not fatal.

This is my first blog-post using a chiny new off-line blogging software! Anyway, this post will be about Python and one of its ugglier corners.

You probably know that calling the implementation of a method in a superclass is easy. But what about a class-method?

Lets give it a try:

>>> class X(object):
...  @classmethod
...  def foo(cls):
...   return cls.__name__
... 
>>> class Y(X):
...  @classmethod
...  def foo(cls):
...   return "NANA: " + X.foo()
... 
>>> Y.foo()
'NANA: X'

Nah, didn't work, and we couldn't expect it too - we didn't send X.foo any cls parameter. But how could we send it? X.foo does not take a parameter, it is given X implicitly as a first argument. It doesn't work as with ordinary methods...

Note that it is possible to get the "right" result if we didn't override foo:

>>> class Z(X):
... 
>>> Z.foo()
'Z'

The only solution I came up with (that doesn't involve super()) is this uggly one:

>>> class X(object):
...  @classmethod
...  def foo(cls):
...   return cls.__name__
... 
>>> class Y(X):
...  @classmethod
...  def foo(cls):
...   return "NANA: " + X.foo.im_func(cls)
... 
>>> Y.foo()
'NANA: Y'

Python 2.3 introduced scoping, just like LISP and other real languages - variables in a surrounding scope can be accessed from an inner scope. Even the function being defined can be accessed from an inner scope.

Class definitions are sort of like scopes:

>>> class A(object):
...  foo = 33
...  bar = foo
... 
>>> A.bar
33

but not entierly...

>>> class X:
...  def foo(self):
...   return foo
... 
>>> X().foo()
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
  File "<stdin>", line 3, in foo
NameError: global name 'foo' is not defined
>>> def foo():
...  return foo
... 
>>> foo()
<function foo at 0x4095c5a4>

This can seem benign, since you rarely would want the function foo, but rather the method wrapper of foo, which is accessible through self.foo. However, this causes a sever problem when nesting classes:

>>> class X(object):
...  class Y(object): pass
...  class Y2(Y): pass
...
>>> class X(object):
...  class Y(object):
...   class Z(object): pass
...  class Y2(Y):
...   class Z(Y.Z): pass
...
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
  File "<stdin>", line 4, in X
  File "<stdin>", line 5, in Y2
NameError: name 'Y' is not defined

Sometimes you just seem to find these wonderfull design descissions that has corner cases that the desgner should have, but didn't anticipate. Like oracles' NULL handling:

> create table foo (bar varchar(5), fie varchar(5));
> insert into foo (bar, fie) values ('', 'tom');
> insert into foo (bar, fie) values (null, 'null');

> select fie from foo where bar not in ('x', 'y');
Gives no rows

> select fie from foo where bar = '';
Gives no rows

> select fie from foo where bar is null;
null, 'tom'
null, 'null'

Python has a similar problem with strings versus characters (characters are considered one-character strings), with similar problems.

>>> type("foo")
<type 'str'>
>>> type("a")
<type 'str'>
>>> type("a"[0])
<type 'str'>
>>> "a"[0] == "a"
True
>>> "a"[0][0][0] == "a"
True

So, what's the lession to learn from this? Never ever treat a value of one type (e.g. Varchar) as the same as a value of another type (e.g. NullType).

This is starting to get booring. I mean the Python unicode-bug category. There are just too many ways in which it sucks. Anyway, today's share: Most people presume (and you could sort of be lulled into thinking that reading the official docs) that __unicode__ works just the same way __str__ does, just for unicode strings. Not so for classes:

&bt;&bt;&bt; class X(object):
...  def __init__(self, x):
...   self.x = x
...  def __str__(self):
...   return str(self.x)
...
&bt;&bt;&bt; str(X)
"<class '__main__.X'&bt;"
&bt;&bt;&bt; class X(object):
...  def __init__(self, x):
...   self.x = x
...  def __unicode__(self):
...   return unicode(self.x)
...
&bt;&bt;&bt; unicode(X)
Traceback (most recent call last):
  File "<stdin&bt;", line 1, in ?
TypeError: unbound method __unicode__() must be called with X instance as first argument (got nothing instead)