NumberPlanet Blog

AppEngine memcached key length limit

fileformat — Sat, 05 Jul 2008 16:44:28 +0000

I guess this isn’t news to people that are already using memcached, but it was news to me (and undocumented in the AppEngine docs) that the keys for memcached are limited to 250 characters.

I’m caching complete pages, and since number handles can be as big as a string, the keys get big too. I put in a hack to use the MD5 hash of the URL as the key if it would be a problem. In theory hash collisions could be a problem, but I don’t see that being an issue in reality.

The error message looks like:

Traceback (most recent call last):
  File "/path/to/google_appengine/google/appengine/ext/webapp/__init__.py", line 499, in __call__
    handler.get(*groups)
  File "/path/to/np/ffnp/number.py", line 161, in get
    cached = memcache.get(cachekey)
  File "/path/to/google_appengine/google/appengine/api/memcache/__init__.py", line 358, in get
    request.add_key(_key_string(key))
  File "/path/to/google_appengine/google/appengine/api/memcache/__init__.py", line 109, in _key_string
    'received %d bytes' % (MAX_KEY_SIZE, len(server_key)))
ValueError: Keys may not be more than 250 bytes in length, received 425 bytes

Default pages

fileformat — Sat, 05 Jul 2008 03:51:01 +0000

I like to end my URLs with a file name (just my personal preference). For SEO, I want to make sure that the default page (index.html in my case) for a directory is indexed only once, so I wrote a little RedirectIndex stub that I had mapped to the various possible directories.

Internally, I always use the page name, so I figured that was good enough. However, GoogleBot seems to guess that index.html is the default page, and I caught it trying to access a page with a query string without the file name. In other words, it tried to hit /tag/square/?page=2 instead of /tag/square/index.html?page=3.

Changing the mappings from / to /(?:[?].*)? solved it. It seems like something in the webapp needs the group to be non-capturing. Otherwise you get

Traceback (most recent call last):
  File "/path/to/google_appengine/google/appengine/ext/webapp/__init__.py", line 499, in __call__
    handler.get(*groups)
TypeError: get() takes exactly 1 argument (2 given)

Starting NumberPlanet

fileformat — Sat, 05 Jul 2008 03:02:33 +0000

I have started a new website NumberPlanet to experiment with new stuff outside of work.

I originally made it in Ruby on Rails, and while I like Ruby and the Rails framework, I found that the frequent RoR updates made maintaining the site a chore. I am only playing around in my spare time, and don’t want to spend it doing sysadmin grunt work.

So when Google came out with the new AppEngine, where they take care of all the hosting chores, I figured I’d try Python.

This blog is where I’m going to share my experience with writing and running the site.