Walled Gardens Response
Sander recently wrote about Facebook’s announcement that they would be exposing their new chat features via XMPP over at the Cocinella blog. I’m very happy to hear that another large company is planning to support open standards, and I completely agree with Sander that we should avoid walled gardens. However, I did have some comments on other parts of his post.
Regarding Digby and Adium adding proprietary Facebook chat support, Sander wrote
Both applications already support the open instant messaging standard XMPP since long. This means that these projects would have had support for Facebook Chat anyway if they just had a bit more patience.
I wonder if this is really true. If Facebook was so interested in XMPP, why didn’t they start with that? Surely it’s easier to start with XMPP than to write something else and then create a gateway. Perhaps they had no intention of an open garden, but later relented after they saw the cat was out of the bag with several clients getting Facebook chat support so quickly. Maybe we have these client authors to thank for this announcement?
I wonder how frustrated these developers currently are, now that they know that their brand fresh code will become obsolete even before it gets in a stable release.
This is a real concern. I hope they aren’t so frustrated that they don’t bother making the new XMPP based Facebook chat nicely integrated. It would be terrible if people continued to use the proprietary protocol when an open one is available. Maybe Facebook will shut off the wall garden once the XMPP support is in place. I definitely wouldn’t want to support two protocols.
Sander suggests using XMPP transports instead for access to proprietary networks. He’s written about it seperately as well.
Transports have several strategic advantages over native client support
While this is true, I think current transports lack in a lot of areas, and certainly I have not seen an XMPP client that has as nice an implementation of transports and transport set up as the multi protocol clients have for their proprietary networks. I think everyone agrees that the XMPP technology is better, but so far the developers (myself included) have not yet made this a win for users.
So let’s welcome Facebook into our community of open instant messaging standards, and let’s continue to improve the transports and the support for transports in the clients to lure those poor users living in the walled gardens.
In Do You Know What Your Database Is Doing, I mentioned that we had fixed several problems in Django that we discovered by using our query analyzer on Chesspark’s database logs. Here’s a list of the main tickets we’ve filed against Django. The relevant patches and discussion are attached to the tickets.
- Ticket #3460: psycopg2 backend uses wrong isolation level - This was the big one. The database should be used in autocommit mode for single queries, and transactions should be used where necessary. The python DB-API 2.0 mandates that the database wrapper automatically wrap transactions when not in autocommit mode, which leads to really inefficient database use when lots of single queries are used. This is pretty much how every web app works, so fixing this will give you quite a database speed boost.
- Ticket #3459: initializing queries called too many times - Django initialized the database by calling SET TIME ZONE and such for every query instead of only on connection setup. This patch is now in the official tree.
- Ticket #3575: iexact uses wrong sql - Django uses ILIKE instead of LOWER(col) = LOWER(’blah’). This works fine, but ILIKE queries are unindexable.
- Ticket #4102: saving only changed fields - Django saves every field on a call to modelobj.save() even if only one changed. This creates nasty race conditions where two unrelated field updates can clobber each other.
- Ticket #3461: passing kwargs to the database wrapper - Django doesn’t pass cursor keyword args through to the database wrapper. You need to do this if you wish to use dict cursors in the psycopg2 backend.
Some of these patches are specific to the postgresql_psycopg2 backend, but there may be similar problems with other backends since the efficiency issue is really a problem baked into DB-API 2.0 (for convenience).
There are so many different web frameworks these days that it is hard to keep count. Do you ever wonder exactly how they are accessing your data? Even if you write your own SQL, do you know how often it executes? In order to investigate performance issues at Chesspark, I wrote a query analyzer to answer these questions. The analyzer looks at all queries and their execution times. It generates a report of which queries are executed most often, which queries have the longest average duration, and which queries consume the most database time. The results were quite surprising for us. For example, we found that one of our simplest apps was generating too many queries by several orders of magnitude (a good candidate for caching!) and that Django and Twisted, two frameworks whose database layers we make heavy use of, had so much overhead that it was frightening.
Faster QA with Screenshots
In any given day, I find a lot of bugs. This is inevitable with software, especially software you are developing. I seem to have a knack for finding new bugs exactly when I’m in the middle of doing something else that is more important or requires concentration. The trick to capturing this information without getting side tracked is screenshots.
For web apps or desktop applications, even if a bug is not rendering related, a selective screenshot often has all the information you need to file a report at some later time. On most platforms you can find a way to take these screenshots with a minimal set of keystrokes. This has many benefits:
- screenshots are often the most efficient way to communicate the bug and you should probably have them anyway
- screenshots are easy to compare making bug fix verification especially easy in many cases
- you’ll be more likely to report bugs since capturing bug information is not interruptive
Read on for how I use OS X and Quicksilver to make this method especially painless.
Continue reading ‘Faster QA with Screenshots’
We do quite a bit of data visualization at Chesspark. Of course we have the standard things like graphs from MRTG, Google Adwords, and Google Analytics, but we also track a number of other things. For example, we track signups through each stage of our signup process. This let’s us see at a glance which parts of the process are losing people, and provides us a good guide on where to focus our attention. Sometimes a technical problem will be the root cause and sometimes it will just be that a page was confusing. Every now and then we get a curve ball.
