I haven’t said a lot here about what I’ve been working on at Craigslist recently. But Craig mentioned me today in his blog and that made me remember that I should say something.

Much of my work has been behind the scenes infrastructure stuff, but some of that is translating into new features that craigslist users can see. And, as of this morning, a lot more users are seeing the fruits of that labor.

As I noted a few weeks back in Sphinx Search at Craigslist, I’ve been hacking a lot on search. Here’s a screen shot to show you what I’ve been calling “nearby search” (though “nearby results” is probably more appropriate).

If you run a search in a city and there aren’t many results, we’ll also run the search in nearby areas to see if we can find matches there too. The above example was a search for “2008 mazda” in my hometown of Toledo, Ohio. The “nearby” results are clearly separated from local matches and local matches are still given priority.

The feedback has been generally positive so far. Though, with any change, some folks aren’t happy. I can’t say it’s going to stay in this exact form. We may need to tweak the interface, the radius of the nearby search areas, and so on. But on the whole I think it’s a helpful improvement when you’re looking for something that’s a bit harder to find and you’re willing to drive an hour or two.

As of earlier today, it’s available in most smaller and medium sized US cities. It’ll probably come to the remainder of cities before long too. I’ve been testing it for about a week and a half, starting with about a dozen cities and then adding about twenty more late last week. This morning I mostly flipped the big switch.

Of course, this opened the flood gates for similar feature requests: custom radius searches, state wide searching, search ALL of craigslist, etc.

In related news, a couple months back I expanded the search help page to include advanced search syntax, including grouping, negation, OR queries, and more.

(comments)

A couple days ago, Andrew posted a news item titled Sphinx goes billions to the Sphinx web site.

Last but not least, Powered By section, now at 113 sites and counting, was updated and restyled. I had long wondered how much Sphinx search queries are performed per month if we sum all the sites using it, and whether we already hit 1B page views per month or not. Being open-source, there’s no easy way to tell. But now with the addition of craigslist to Powered By list I finally know that we do. Many thanks to Jeremy Zawodny who worked hard on making that happen, my itch is no more.

Well, I guess the cat’s out of the bag! My first project at Craigslist was replacing MySQL FULLTEXT indexing with Sphinx. It wasn’t the easiest road in the world, for a variety of reasons, but we got it all working and it’s been humming along very well ever since. And I learned a heck of a lot about both Sphinx and craigslist internals in the process too.

I’m not going to go into a lot of details on the implementation here, other than to say Sphinx is faster and far more resource efficient than MySQL was for this task. In the MySQL and Search and Craigslist talk I’m giving at the 2009 MySQL Users Conference, I’ll go into a lot more detail about the unique problems we had and how we solved them.

For what it’s worth, the implementation isn’t really done. I did update the search help page on the site to reflect some of the capabilities (hey, look! OR searches!) but there are features I have planned that I’d like to expose as time allows.

(comments)

In Tim O’Reilly’s Work on Stuff that Matters he elaborated on three criteria that constitute “stuff that matters” for his readers:

• Work on something that matters to you more than money.
• Create more value than you capture.
• Take the long view.

A number of folks where surprised when I announced that I was joining craigslist back in July but it’s an organization that I really admire. Having been there about 6 months now, I can definitely say that it’s a job that matters based on Tim’s thinking and my own.

Every time I meet someone and tell them where I work, their reaction is quite positive. They’ve had a good experience with craigslist, like the service, love the philosophy, and so on. Craigslist matters ordinary people–not just technology nuts.

Similarly, I know that we create more value than we capture. The majority of our service is free and usage seems to be growing all the time. People I talk to get such good responses with craigslist classifieds (compared to, say, newspapers) that I know we’re giving people more than their money’s worth.

As for taking the long view, I think being a non-public company helps that a lot. I’ve rarely thought about what “the next quarter” will bring. It’s quite a contrast from my years at Yahoo. When we’re discussing technology infrastructure, I’m always trying to think ahead a year or two (or more). But the day to day ups and downs just don’t feel as important the way we operate. I like that.

All in all, I’ve been very happy with the change and am glad that Tim posted something that helped me to explain what I like about it.

(comments)