The site of the book of the sites, The Internet, now in handy book form, is good fun. Crackbook and Poormatch are particularly well observed. It reminded me of TV Go Home, but a little less bitter and scatological (only a little, mind you).

Quotable quotes of the week:

"... any time anyone's said anything comprehensible about the Trinity the Church has declared it a heresy." - gjm11 on a Rilstone post created specifically for him.

"The universe tends toward maximum irony. Don't push it." - jwz on taking reliable backups (which is much harder on a Mac than it ought to be).

"All those fine words about the rule of law safeguarding our liberties, the arbitrary exercise of power and Bunker Hill, Lexington and Normandy went right out the window on 9/11. That was when Henry and the rest of his stalwart defenders of the rule of law promptly wet their pants and then let their president use the constitution to clean up the puddle." - Digby, via a friend of a friend.

There's an option that I might have considered instead of apostasy. Unfortunately, in those conservative days, you couldn't really do that sort of thing. These days, if LiveJournal is anything to go by, it's all the rage. A woman tells us how she's in an open relationship with Jesus.

This is an article about the sort of thing I spend my days doing. Usually, I can't talk about that, for reasons of commercial confidentiality, however, this particular case is completely unrelated to anything my employer sells, so I should be OK. I've tried to explain things sufficiently well that someone non-technical can get it. Hopefully it's not too dull or incomprehensible.

First, we need some technical background...

The Naming of the Parts

A graph is a bunch of things connected by lines. The CDC Snog Graph is an example of what I mean. The things (representing people, in this case) are known as vertices or nodes, and the lines (representing joyous sharing experiences of some sort, in this case) are known as edges.

The lines on the Snog Graph don't have a direction, so we call it an undirected graph. If you add a direction to each edge, you get a directed graph, which we can represent by putting arrows on the lines. Friendship on Facebook can be represented as an undirected graph (where the nodes are people and the edges mean "is friends with"), because all friendships are mutual, but LiveJournal friendships need a directed graph to represent them, since I can be your friend without you being mine, and vice versa.

A graph is cyclic if there's a way to walk along edges starting at one node (following the arrows if the edges are directed) and get back to that node again without walking the same edge more than once. The Snog Graph is cyclic, as is the graph of friendships on a Facebook or LJ (trivially so on LJ, where it's possible to friend yourself. Regular dancers might consider how this applies to their graph, vis-a-vis people who consider avoiding other dancers an optional extra). Graphs where you cannot do this are called acyclic.

Usenet news: you tell kids today that, and they don't believe you

Usenet is an electronic discussion system which pre-dates all this newfangled World Wide Web nonsense. It distributes messages which are known as articles. It has some desirable features that web-based discussion forums often lack, like comment threading and remembering what you've already read. These days, people think Usenet is owned by Google, but in fact it's a distributed system, with no central server (another advantage over LJ). When you want to talk to Usenet without using Google, you run a client program on your own machine, which talks to your local server. Your local server forwards your article to servers it knows about, which then forward it to servers they know about, and so on (the path of an article through the servers forms a directed acyclic graph, in fact). When you want to read other people's articles, your client program fetches them from your local server.

Each article is posted to one or more groups, which are like communities on LJ. Note that, unlike LJ, the same article can be posted to more than one group without having to cut and paste it: the same article exists in each group it is cross-posted to.

On each server, articles have a number within each group. The first article to arrive in a group has number 1, the second article number 2, and so on. Cross-posted articles have more than one number, one for each group the article appears in.

Article numbers differ between servers, because the order of arrival depends on the path the article has taken to reach the server, but since your client program only talks to your local Usenet server, it usually refers to articles by their number (there's also a unique string of letters and numbers which identifies the article, which is how servers know which ones they've seen already, but that's not important right now). Remembering which articles you have read is then just a matter of storing some ranges of numbers for each group (so your client might remember that you have read articles 1-100,243-299 and 342-400, say).

The problem

We wanted to de-commission a Usenet server and move its articles to another server. The servers run different and incompatible software, so the most obvious way to get articles from one server to another is to post them like a client or another server would.

The new server is supposed to be a drop-in replacement for the old one, so we can't change the numbers or the existing client programs will get confused and think you've read articles you haven't, or vice versa. So you can't just grab all the articles from the old server any old how and post them to the new one, because they'll be jumbled up. Unfortunately, the old server has no way of directly telling us the precise order that articles arrived in, though it will tell us article numbers within each group.

"Aha!", we think. Since the order of arrival matters, we'll grab the articles in order from one group on the old server, post them in that order on the new server, and move on to the next group, where we'll do the same, and so on until we've done all the groups.

This idea is ruined by cross-posts, because they have more than one article number associated with them. If a cross-posted article is number 10 in one group and number 3 in another, you'd better post the first 9 articles to the first group, the first 2 to the second group, and then you can make the cross-post. But maybe there's a cross-post to a third group in those 9 articles, so you'll need to get that group up to date before you can post one of those. How do you work out what order to post the articles in?

Obscure Unix tools to the rescue

You might have guessed that the wibbling about graphs wasn't entirely tangential to all this. You can draw a graph of the problem. Each article is a node. For each group, connect each article to the one with the next number up by a directed edge (in this case, the direction of the arrow means "must be posted before"). You've drawn yourself a directed acyclic (since the article numbers only increase) graph. The cross-posted articles are then nodes with more than one edge coming into them.

One of the wizards at work realised this, and also pointed out that there's a standard Unix tool for converting such graphs into a list of nodes whose order preserves the order implied by the arrows, a procedure which is known as a topological sort. The tool's called tsort. From there, it's just a matter of representing each article in the way tsort understands. When you do that, tsort gives you an order in which you can post the articles from the old server to the new server so they'll be given the same numbers on the new server as they had on the old one.

In which I'm not as knowledgeable as the wizards

The way I thought of to do it was to write my own program. You pick a group, and start trying to post articles from the old to the new server in article number order. If they're not cross-posted, you just keep going. When you hit a cross-posted article, you switch to each group it is cross-posted to in turn, and try to post all the articles before the cross-post to each of those groups. If while you're doing that, you hit another cross-post, you remember where you got up to and do exactly the same thing with the groups that second cross-post is posted to, and when you've done that, you switch back to the groups for the first cross-post. I had a working version of this at about the same time as someone pointed out that tsort does the job :-)

(Aside for the techies: this can be done using recursion. It looks like this is pretty much equivalent to one of the ways of doing a topological sort, namely the depth-first search mentioned on the Wikipedia page).

So I learned about topological sorts by re-inventing one. Most of the problems I deal with aren't in fact cases of well-known problems (although I would say that if I was merely bad at recognising the fact, wouldn't I?), but when they are, recognising that can save a lot of time.

And that's what I do when I'm not on holiday.

I recently finished Andrew Hodges's Alan Turing: the Enigma. The book is a definitive account of Turing's life and work. In some places I found the level of detail overwhelming, but in others I admired the way Hodges uses his obviously extensive research to evoke the places and people in Turing's life. The book is well worth reading for the perspective it gives on Turing, something which is absent from other, purely technical, accounts of his work.

Hodges portrays Turing as a man ahead of time, conceiving of the Turing machine as a thought experiment before the invention of the general purpose electronic computer, and inventing the Turing test when computing was in its infancy. Turing's naivete was reflected in his refusal to accept what other people said could be done, but also in a lack of interest in the politics of his post-war work on computers and of his own homosexuality. A proto-geek, Turing was prickly, odd, and seemed to expect that the facts alone, when shown to people, would lead them to the same conclusions as he found.

Turing's suicide is placed in the context of a move from regarding homosexuality as criminal to regarding it as a medical problem, and an increasing suspicion of homosexuals in classified government work. Hodges seems to conclude that Turing felt he had nowhere else to go.

You can't help but wonder what else Turing might have accomplished had he not committed suicide. Greg Egan's short, Oracle, is an entertaining what-if story, which also features a character very obviously based on C.S. Lewis. What if Turing had received help from a friend? It's a pity that in reality there was no-one to lead him out of his cage.

Some of you might have played with emulators which let your PC pretend to be the classic computers of your youth (the BBC Micro, Acorn Electron, Commodore 64 and so on). The emulators work by simulating the processors in those machines on your PC, so you end up with Chuckie Egg running on a virtual BBC Micro running on a PC running on whatever the underlying physics of the universe happen to be.

robhu found a puzzle based on a similar idea. The International Conference on Functional Programming have an annual contest. Last year's was particularly fun:

In 1967, during excavation for the construction of a new shopping center in Monroeville, Pennsylvania, workers uncovered a vault containing a cache of ancient scrolls. Most were severely damaged, but those that could be recovered confirmed the existence of a secret society long suspected to have been active in the region around the year 200 BC... Among the documents found intact in the Monroeville collection was a lengthy codex, written in no known language and inscribed with superhuman precision.

You're given the codex, and the specification for a processor used by the Cult of the Bound Variable. Once you've written an emulator for that processor, you're off (unless, like me, you write the emulator in a few hours and spend half a day corrupting the files from the contest website with your editor and wondering why it isn't working). There's an ancient Unix system, complete with ancient spam, and various puzzles which I've not had a chance to look at yet.

According to reports on the contest, lots of people tried to write the emulator in high level languages where it ran like treacle. A C implementation like mine (which may count as a spoiler for people who want to do it themselves) without any namby-pamby array bounds checking nonsense works fast enough that you can't tell you're not talking to a real ancient Unix system. There's life in the C language yet.

I'm lost in admiration for the people who produced the codex. It required writing a compiler backend targeting the fictional processor, and implementing various other languages on top of that, as well as coming up with the puzzles. I'm not sure how far I'll get with the puzzles before my ability or patience expires, as I'm a hacker rather than a computer scientist, but still, the thing is amazingly cool. It's a time sink for geeks, so I pass it on to those of you who, like me, will waste many days on it. Have fun.

I am currently laid up. I've been trying out Firefox 2.0. It looks quite good. On the Mac, it's faster then 1.5 and doesn't get bogged down when you leave it running for a long time (I tend to put the Powerbook into sleep rather than shutting it down). I've not tried it on Windows yet, as I use Mozex for editing Wiki entries at work, and that's not been updated for 2.0. The essential extensions have been updated, though: AdBlock and Greasemonkey being the two I use the most. It's always a shock to use someone else's machine and find their intarweb has adverts. I mean, how quaint. And you need Greasemonkey for LJ New Comments, which the people on lj_nifty seemed to like, bless 'em.

I like the spelling checker for form entries, and the way that you can now have it save and restore sessions, move tabs around, and put a close tab button in the corner of each tab. The smart completion thingy for Google searches is quite nice, as is the way that sites can offer their own search plugins which Firefox picks up on and can then install automatically (you can tell the site offers a plugin when the little arrow in the search box glows blue: thanks to marnanel for pointing that one out). I like the way that it can be configured to add RSS feeds to Bloglines, too.

While I was enjoying all this web 2.0 excitement, I thought I'd try out del.ico.us. It's a site that lets you store your bookmarks externally, so you can find them on any computer, and also lets you tag them with keywords so you can find them again easily (which is my main reason for using it: my bookmarks were getting out of control). You can see my bookmarks here, and there's also an RSS feed of them if you'd like to stalk me.

Dear Lazyweb

I'd like some sort of house server thingy. scribb1e and I both have laptops, so we'd like somewhere to back-up important stuff. scribb1e has read the Pragmatic Programmer book and would like to keep her life under version control, which I think means Subversion here (not quite as good as Perforce, but free). I have the vague idea that we could connect something to the stereo and play my extensive collection of mp3z through it. We'd like the box to be more or less silent and quite small.

Possible candidates include the Mac Mini and the NSLU2, affectionately known as the Slug. With the Slug, I'd buy a big external drive onto which I would put a proper Linux distribution, as described on the web page. You don't seem to be able to buy small and quiet server boxes if you don't want to mess around with ordering the bits and building them yourself, which I don't.

The Mini's standard disk size is a bit small, but other than that it certainly does everything we want. However, it's somewhat pricey for a box we're not going to use interactively. The Slug is a lot cheaper but will require me to resurrect my Linux-fu (and if I want sound output, get into the sort of horrific kernel nargery that made jwz buy a Mac). I could buy the Slug and some networked other audio output thingy as I've heard you can get those these days, but I've no experience with them.

This must be a fairly common thing in other geek households. What have the rest of you done?

A couple of shiny new bits of software have come out in the last few weeks. Both of them are at version 7, for some reason.

Inform 7

Inform 7 is the latest version of Inform, the language for creating interactive fiction. The interesting thing about it is that Inform 7 programs are written in a subset of English:

The wood-slatted crate is in the Gazebo. The crate is a container. Instead of taking the crate, say "It's far too heavy to lift."

Inform is not capable of understanding arbitrary written English, but has a set of sentence forms it understands, and some inference rules built in (for example, if you tell it that "Mr Brown wears a hat", it will infer that Mr Brown is a person).

scribb1e pointed out that this makes the work of writing the story similar to playing it. That could turn out to be a bad thing: most programming languages are so stylised and full of random punctuation symbols that programmers realise they're not writing English and don't try writing arbitrary English text in the hope of being understood by the computer. Even for people who understand Inform isn't actually intelligent and that they have to write in Inform's dialect to be understood, writing in something close to English will make it harder to remember to restrict their vocabulary. At worst, it could become a game of guess the verb, which would be painful (as opposed to a game of Guess The Verb, which I thought was fun, especially the Old Man River bit in the help).

However, unlike playing a game, looking at the excellent and witty online help doesn't risk spoiling your fun. Since it's all English, it's easy to crib paragraphs of text from the examples and adapt them to your own works. Hopefully, writing the games in English will enable more people to create them without feeling that they have to be expert programmers. They'll still have to think like a programmer, but won't face the intimidating prospect of curly brackets.

Inform 7 itself isn't just the compiler, it's is a complete suite of tools for writing, testing and releasing interactive fiction, the IF equivalent of an Integrated Development Environment. It's rather nice (although not yet available for anything other than Windows and Mac OS, because of the difficulty of getting the graphical stuff going on a variety of platforms).

Vim 7

I use the Vim editor, which is the old Unix vi with all the features you want from a modern programmer's editor bolted on. New in Vim 7 there's a spelling checker, "Intellisense^TM" style context-sensitive completion of names in code, and tabbed windows (no software is complete without tabbed windows these days).

The completion stuff is particularly useful, as it now pops up a menu of possible completions which you can select from with the cursor keys, and appears to be trying harder to find completions from nearby files in the background as you're typing (I've not quite worked out what it's doing yet, it's reaching the stage where it's just magic). Completion isn't just for programmers, of course: when I'm typing an email, if I find myself using the same, long, word more than once, typing the initial letters and then letting Vim complete it is a boon.

The recent change to LJ's URL formats seems to be part of an attempt to defend against one or more attacks which allow the attacker to steal another LJ user's credentials, gaining the ability to impersonate that user. The theft occurs when the victim visits a page on LiveJournal which contains some malicious Javascript inserted by the attacker (more technical details below for those that care).

What's been happening?

Slashdot linked to an article with some more details on the attacks. This article includes details supplied by the Bantown group (who live at bantown.com, a site you probably want to visit using lynx). Bantown have use these attacks to pwn LiveJournal quite comprehensively: the comments on the news entry contained comments from tens of different users with the same demand from Bantown. It's likely that these users all had their credentials stolen by Bantown.

I found a comment quoting an explanation of the vulnerability in an entry on lj_dev, but that entry has now been deleted. The quoted explanation is about a vulnerability which only applies to browsers based on Mozilla (so, Mozilla, Firefox and Netscape). The Bantowners claim that this is not the vulnerability they were using, as they have a vulnerability which affects all browsers. LJ recently patched a vulnerability which would do the job for all browsers, but it's possible there are other, similar, vulnerabilities in LJ's code. Or it's possible that the Bantown people are lying.

Is it fixed?

LJ went down for a while on Friday afternoon, and seems to have invalidated all existing cookies. However, bradfitz is keeping quieter than I'd like about whether the risks still exist and about what workarounds users can use while LJ's crack programmers are working on a fix. bradfitz's use of "soon" suggests that the URL change was part of further changes. These further changes aren't in place as I write this, which I think means that it's still possible to use whatever attack the Bantowners have been using to steal credentials, although it's not possible for an attacker to use an old set of credentials from logins before this afternoon.

Edited: LJ has now fixed this, so it's safe to turn Javascript on again.

What can we do about it?

For now, I'm running with No Script turned on, and using that to disable Javascript for all but trusted sites, of which LJ obviously isn't one. LJ's lack of communication about the risks to user data, and about possible workarounds, displays a worrying incompetence, as I've said elsewhere.

The Science Part

LJ uses cookies, small pieces of data stored by your web browser, as your credentials. When you log in to LJ, you get a cookie. From then on, your browser presents the cookie whenever it requests a page from LJ. LJ trusts you because you have the cookie, and lets you do things that only you should be able to do. The cookie can persist just until you close your browser, or longer if you've ticked the "remember me" option when you log in.

The attacks on LJ are cross-site scripting or XSS attacks. A Javascript running on a particular page can access the cookies for that page. Currently, any Javascript running on an LJ page can see your cookie, because the same cookie applies to the entire site. If an attacker can cause their own Javascript to run on a page supplied by LJ, they can steal that cookie and send it to a remote server that they own.

How might the attacker get their script onto LJ's pages? Well, LJ lets you submit HTML as entries, comments, and as your own styles, and then displays it on its pages. LJ attempts to sanitise the HTML you supply it, but if it doesn't do this correctly, the attacker has a way in. They can put their Javascript on the page, and visiting that page would then send your cookie to their server. Also, browsers based on Mozilla (such as Netscape and Firefox) allow stylesheet authors to embed Javascript in a CSS stylesheet, so the way LJ lets users reference their own external stylesheet is another security hole (although as I said above, it's possibly not the one that the Bantown people are using).

There's some more discussion of how this works (in amongst a lot of sarcasm) in this thread on jameth's journal.

The LJ New Comments script now copes better with the bewildering variety of journal styles that are out there. I also stopped it from giving up in disgust if a style allows it to see the comments but doesn't provide a permanent link to each comment, as the "n" and "p" keys will still work in these styles (q.v. lauralaitaine's journal, for example).

I'm now using scrollIntoView to move each new comment to the top as you click or press keys, so you don't get a new history entry for each comment you visit (I was annoyed with having to hit the "Back" button multiple times to leave the entry). The docs for Greasemonkey allege that scrollIntoView doesn't work within Greasemonkey unless you do special stuff, but I seem to be getting away with it. Possibly I've broken the script for people not using Firefox 1.5, but such people need to feel the white heat of technology, anyway.

Ph34r my sk1llz!

ETA: Except that I broke it again trying to make it handle all the extra ways of denoting comments. v0.4, now on the userscripts.org site, seems to be working.

I've finally got around to writing the Greasemonkey script which I've long been threatening.

What it does

The script remembers which comments you've seen on LJ (or Dreamwidth) and helps you navigate to new comments. That's right, I'm finally dragging LiveJournal kicking and screaming into the 1980s.

If you're on an entry page, pressing "n" skips you to the next new comment, and "p" skips to the previous one. If the style has an "Expand" link, moving to an unexpanded comment with these keys will also expand the thread. If the style has a permanent link or a reply link for each comment in that comment's header or footer, the script inserts another link next to it, labelled "NEW". That link shows you that the comment is new at a glance. Clicking the "NEW" link selects the comment so that pressing "n" will go to the next comment from there. On some styles, the currently selected comment will be outlined with a dotted line.

On a journal or friends page, the script will also add the number of new comments to the link text, so that, say, "15 comments" becomes "15 comments (10 new)", and enable the "n" and "p" keys to move between entries which have new comments, and the "Enter" key to view the selected entry. This only works if you're looking at a journal which adds "nc=N" to entry links to say there are N comments on an entry (LJ can do this as a trick to confuse your browser's history function into thinking you've not visited that entry whenever there are new comments). If you want to turn this on for your journal then ensure you're logged in, visit this page, check the box which says "Add &nc=xx to comment URLs" and hit the "Save" button.

How it works

You don't need to understand this section to use the script. If you don't care about programming, skip to the next part.

( Gory details ) Using it

If you want to use it, you will need:

• Firefox, the web browser, version 1.5 or later.
• Greasemonkey, the extension which lets people write little bits of Javascript to run on certain pages.
• LJ New Comments, which is what I've imaginatively entitled my script. If the userscripts site is down again, you can find a copy on my site.
• Your flask of weak lemon drink.

After you've installed all of the above, visit an entry on LJ and marvel at the "NEW" links on all the new comments (which will be all of them at this point, as the script wasn't around previously to remember which ones you'd seen before). See above for operating instructions.

Privacy

Note that the script stores a Firefox preference key for each journal entry you visit, listing the IDs of the comments it finds there. The script doesn't let the database grow without limit: when the script has seen 500 entries, it starts to drop the history for the entries you've not visited recently.

Clearing the browser's history doesn't affect the script's list of visited entries. Thus your visits to polybdsmfurries will be recorded for posterity, even if you clear the browser's history. You can wipe the entire history by using the "Manage User Scripts" entry on the Tools menu to delete the script and its associated preferences (you can re-install it afterwards, but you must clear out the preferences for it to delete the history).

The script does not record the contents of any entry or comment. The script does not transmit any information to LJ or any other website, it merely acts on what it sees when you request journal entries.

Your questions

I've given this entry as the homepage for the script on Userscripts.org. That means this entry is intended to serve as a repository for questions about the script, so if you've got a question, comment here. I prefer this to commenting on my other entries or to emailing me, unless you already know me. Ta.

To keep up to date with new releases of my greasemonkey scripts, track the tag "greasemonkey" on my journal. This link should enable you to subscribe to that tag and get notified when I post a new entry about greasemonkey scripts.

Revision history

2006-01-02, version 0.1: First version.

2006-01-03, version 0.2: Added the "p" key. Used javascript to move between comments so doing so does not pollute the browser's history. Coped with the id=ljcmtNNNN way of marking comments. Made "n" and "p" keys work even in the absence of permalinks on each comment.

2006-01-04, version 0.3: Apparently you can have id=tNNNN, too.

2006-01-04, version 0.4: Broke 0.3, fixed it again. I hope.

2006-01-19, version 0.5: Updated to cope with LJ's new URL formats. Changed how comments are stored internally so that the database does not grow without limit: the script now remembers comments for the last 500 entries you visited, and forgets the entries you've visited least. Also added "New" marker based on reply link as well as thread link, for styles which don't have a thread link for every comment.

2006-01-19, version 0.6: Convert dashes I find in URLs to underscores internally, to preserve access to history from older versions of the script before LJ's URL change.

2006-02-09, version 0.7: Work around the fact that Firefox leaks memory like a sieve. Never display negative number of new comments. Change licence to MIT as GPL is overkill for this script.

2006-02-09, version 0.8: There was a bug in the workaround code I got off the Greasemonkey mailing list. Fixed that.

2006-06-04, version 0.9: Enabled the "n" and "p" keys on the friends/journal view. Added the box around the current comment.

2007-02-20, version 1.0, baby: Try harder to draw a box around the current new comment. Applied legolas's fix for pressing CTRL at same time as the N or P keys (see comments).

2008-03-31, version 1.1: Make it work faster on entries with lots of comments. Altered behaviour of "NEW" link so it now selects the comment you're clicking on, as that makes more sense.

2008-09-24, version 1.2: Support Russian keyboards thanks to mumi_0, make threads expand.

2009-01-27, version 1.3: Support for independentminds journals.

2009-05-04, version 1.4: Support for Dreamwidth.

Finally got Mozex going on Firefox with Mac OS X. This means I can edit my comments on LiveJournal with Vim rather than messing about with LJ's comment posting box and the less powerful editing facilities from my browser. I can also use Danny O'Brien's marvellous Google linkification script. Which is nice. It'd be even nicer if Firefox's process creation API worked properly on Mac OS X, though.

As a result on all this mucking about, I've not had time to respond to comments on the God Hates Hair entry. I'll get around to it sooner or later, though.

Greasemonkey is an extension for the Firefox browser which lets you write little programs to change how websites appear. For example, tymofiy has created a script which lets you expand collapsed LJ comment threads in place (that is, on the same page, rather than on a new one). It only seems to work with the standard comment layout at the moment, but I hope the author will fix that soon (if not, it doesn't look so hard that I couldn't do it myself).

I noticed that the later versions of Greasemonkey support a key/value database which persists when you shut down and restart your browser. This means that it's probably possible to write something which remembers how many comments there are for an entry and will highlight items (on your Friends list, say) which have new comments. It might even be possible to highlight the new comments themselves, although it's not clear how good the database is, so you'd want to avoid overloading it, I suppose. I was vaguely aware of Greasemonkey, but I don't think I'd realised just how much it can do. Greasemonkey may be the thing which makes me switch from Safari to Firefox (it's just a shame nobody has sorted out Mozex for the Mac, as that'd certainly clinch it for me, too: I miss being able to edit LJ comments in a proper text editor).

Think I'd better dance now.

Chiark is a Unix box on which a large number of Cambridge geeks have accounts (I'm not one of them, as it happens, but I know some of them by name and a few of them by sight). It runs some local newsgroups, which are only accessible to people with accounts. They've recently added a journals newsgroup, to which some people are publishing their LJs (it's a one way street at the moment, by the sounds of it: entries and comments go from LJ to the newsgroup, but not vice-versa). This has caused some excitement on my friends and friends-of-friends lists. Of particular note are atriec's posting on what LJ's are for, emperor's own views (I'm not sure why Chiark is "cabal" there, but it's the same thing being discussed), and mobbsy's comparison of LJ and newsgroups. There are a couple of coupled problems here: LJ's interface is not useful for having discussions (as opposed to simply pontificating) and some people don't actually want to have discussions anyway.

LJ's limitations do annoy me. As I said to livredor recently, I'm here for the people, not the interface. Compared to sites like Google or Flickr, LJ hasn't done very well at making its stuff accessible by computer programs which might do useful things with it, such as re-presenting it in a way which is easier to to read, remembering what I've already seen and alerting me to new stuff, and so on. OK, so there's RSS, but that's no good for comments. OpenID is a step in the right direction, but largely solves the opposite problem, namely letting non-LJers put their stuff here. The client protocol is, again, designed to let people put stuff on LJ, not to take it out. LJ explicitly says that they don't like screen scraping (that is, programs which extract information from the LJ pages which are designed to be read by humans) as lots of programs doing this will request lots of pages very rapidly and put more strain on their server more than they'd like.

LJ slowly getting better as a discussion forum, but the pace of change is slow. Tags are useful, OpenID is pretty cool, but on the whole LJ's developers also seem to spend a lot of time on making it look pretty (a worthy goal, since newsgroups are pretty ugly by comparison, but probably not worth all that much time from the developers, who could just provide the users with the tools to do it themselves). That's probably down to their target audience, I suppose: a few refreshes of the random journal link shows that LJ is largely populated by teenage girls (and by Russians, for some reason). See also the large number of people saying "actually, we want more user icons, not this OpenID thing" on the OpenID announcement.

There's also the question of what a LiveJournal is for. livredor's posting on manners on LJ made the point that nobody is very sure what the etiquette is for making comments on other people's postings. Having been brought up on newsgroups, I assume that anything I can see and which has comments enabled is fair game, although in deference to the fact that I'm entering someone's personal space, I'll usually introduce myself before diving in. But I suppose I could still end up horribly offending someone. It's possible that most LJ users don't want to have long discussions on their journals, in which case LJ would be wasting their time by making that easier, and I should just find somewhere better suited to that, which supports OpenID.

What would be the ideal, for me? The distribution system of Usenet (the network of servers which provides access to the public newsgroups) means that you can't really recall postings once you've made them, and also makes it hard to make the equivalent of friends-only postings (you could do it, but it'd be hard to conceal the fact that you'd at least made a posting that someone else couldn't see). So, I don't object to LiveJournal's centralisation in itself, because it helps me keep control (and now OpenID means I can entrust non-LJ people with my friends-only stuff, if I want). On the other hand, the interface sucks when you want to follow a discussion.

I'd like to see more machine readable stuff (especially comments) and a better API for clients to use to pull out comments and so on. I suppose I'd really like to see LJ run an NNTP (newsgroup) server which wouldn't distribute stuff, but which would allow the same restricted amount of HTML that LJ itself does. A journal would be a group, an article would start a new thread, and the comments would be followups. Stuff that you weren't meant to see just wouldn't show up in the group, because you'd need to log in to the server to see it. I like this idea, although I can't really see LJ implementing it. Maybe we should start a meme to campaign for it? We could call ourselves the Campaign for Real News.

If you're an English geek of a certain age, one Christmas you probably received a computer game which wasn't like any games which had gone before. It came in an A5 box, big enough to hold the manual, the novella, the poster with all the ships on it, the leaflet listing all the keys you needed to fly your spaceship, and, I dunno, some other stuff, probably. Elite pitted you and your trusty Cobra Mk III against an open-ended universe, where you could make a living by trade, piracy, bounty-hunting or, as your reputation grew, by carrying out missions for people. It depicted the universe in glorious wire-frame 3D. As Francis Spufford explains in Backroom Boys, it was groundbreaking and absorbing. That particular Christmas, I played it so much that, as I slept, visions of Pythons danced in my head.

Oolite is a free, open source Elite clone in Objective C for Mac OS X. It's faithful to the original, but there are some improvements (the targetting box which shows the distance, ship type and legal status in the picture above, for example), and some nice touches (notice the skull and crossbones on the Mamba?) You also feel more a part of the world than in the original, as you come across other ships engaged in combat, or a pirate caught by the long arm of the law. The nostalgia! Buy a Mac and play it.

Some while ago, Mark Pilgrim did a post on cool software tools he couldn't do without. In the absence of any pending rants about religion, here's mine. Probably only of interest to fellow geeks, so cut for length.
( Read more... )

ladysisyphus writes that Jesus has laser beams. As does Aslan, which makes sense if you think about it.

It turns out that Macs have speech synthesis built in. It's not bad, and it's easily accessible to programmers. So I've spent an entertaining evening making my MUD client talk. That way, if the window is hidden, I still find out when someone interesting logs in. I've ended up using MudWalker, a free, open source MUD client for Mac OS X. It's scriptable in Lua, and helpfully provides a speak function to Lua scripts. The thing prospective programmers will want to know is that your regular expression match groups (the things Perl would call $1 and so on) are arg[n] to the Lua scripts you can use to write triggers. For console use, I'd still recommend Crystal as a good MUD client, but it turns out to be a bugger trying to get that to talk (Crystal is supposedly scriptable in Lua, but my attempts killed it).

Also been looking at Twisted, Python's marvelous asynchronous mouse organ networked application framework thingy. It seems that as well as being very clever, it's actually reasonably well documented these days. The Perspective Broker and Avatar stuff seems to be a good fit for games where the players can write code which is not trusted by the system, since the choice of which methods allow remote access imposes some sort of capability based access control. If I ever wrote a MUD in Python, something I've been threatening for some years now, Twisted would be the way forward (indeed, it was originally created to provide multiplayer interactive fiction in the form of Twisted Reality, another addition to fine the fine Internet tradition of hugely ambitious, but largely unfinished, MUD servers).

It'd probably be easier just to do this in Java. Python's restricted execution stuff is not really there, so if you wanted to allow players to program (which I think is essential for holding people's interest once they've finished the game proper) you'd probably end up running untrusted code in another process and using PB to talk back to the server. Still, it's a nice dream. I saw that the author of MudWalker has got a prototype MUD written in E, the capability-based security language, which might well be worth a look too.

I much prefer old, depressive-to-rival-Leonard-Cohen Counting Crows to new, happy Counting Crows.

It's quieter now all those weddings and barbeques have subsided for a bit. Me and she had a lovely evening with Gareth and Emma the other night. Gareth is the Scourge of Uk.religion.christian, putting nutters to flight with his rapier-like logic. Or something.

I remember mentioning Leonard Richardson's Guess the Verb interactive fiction game, Munchkin, and opportunitygrrl (whose interests include geology, interplanetary video feeds, Mars, and Christina Aguilera).

Speaking of interactive fiction, I remember I promised terriem some links to IF works recommended by S. terriem also pointed me at Kingdom of Loathing, which I've not tried yet. I played through Slouching Towards Bedlam and enjoyed it, although I did have to resort to the help a couple of times. Still, the story's the thing in this one, not the puzzles so much.

S also recommended 9:05 (which is short and funny in a twisted sort of way), and Spider and Web (which I've played a little way, and which is apparently longer). Get them from Home of the Underdogs in their IF section. To play them, you'll need an interpreter which runs the files. On the Mac, I got Frotz from Fink for the .z5 files and MaxTADs for the TADS ones. This page lists IF interpreters for Windows. There's a selection of Beginner's Guides to help with the conventions of the medium.

There is a confusing multitude of spam filters out there. I once wrote an article listing all the ways of filtering spam I could think of. If you're confused by all this, here's what I do, along with ways of doing the same thing on both Unix and Windows systems.

( Read more... )

A couple of students in Another Place are in trouble for "hacking". The news papers aren't particularly specific about what they did, but it sounds like they installed a packet sniffer and listened in on traffic across their network.

Ethernet networks have everyone hanging off the same piece of wire. If you're on an Ethernet network, your network card has a unique address. As the traffic for everyone on that piece of wire flows by, your computer picks up traffic addressed to it. It doesn't listen to other people's traffic because you usually don't care about it. However, by running your network card in what is delightfully known as promiscuous mode, you can see other people's traffic. Programs which do this and present the results to you are called packet sniffers. Ethereal is a popular free packet sniffer. Packet sniffers have legitimate uses, like diagnosing network problems or writing and debugging software which uses the network (I installed Ethereal the last time I was having problems with DNS lookups, for example). The remedies for undesired sniffing are encryption and restructuring the network so everyone's packets don't share the same piece of wire.

The Oxford students seem to have been disciplined for drawing attention to what they did, but none of what they found is news. A college network probably has everyone hanging off the same wire. There are encrypted versions of telnet, HTTP, IMAP and POP3 but not many people use them. There are a lot of clever people with time on their hands. You work it out.

People who know this have done some sort of risk calculation and come up with a solution that they're happy with, which balances convenience against privacy. For example, I only permit encrypted logins to my machines and don't send my password itself when fetching email (although the mail itself comes across the wire as plain text). Now you know what's possible, you can do that calculation too.

Photos from lisekit and terriem's Karoake party are up. The close-ups of faces are courtesy of S, who does that composition thing (as well as being a talented pianist). Thanks for a good party, Terrie and Lise.

I promised Terrie a link to Milkbusters, so there it is.

My computer is unhappy. It freezes or resets itself about 20 minutes after it's powered on. After that runs for the rest of the day. I'm getting kernel oopses just before the bad stuff happens, which seem to be related to memory paging, so I suspect dodgy hardware.

I could just replace the memory modules and see whether that helps. However, Drogon people have been attempting to convince me that what I really want is one of these fine Apple computers. Apple's OS X has Unix underneath, and the Fink project makes a variety of open source stuff available to it using the Debian package tools. The PowerBooks look rather nice. They're also rather expensive. If anyone's used one, do let me know what you thought of it.

The ensuing discussion of laptops, 802.11, and hence using the computer on the loo made me think of Anthony Hopkins, for some reason.