I implemented Yahoo BOSS as the backstop search engine for Serendeputy a couple of months ago.

When you do a search, I first look to see if I have a match among the pre-compiled topics. If you do, I bring you directly there. If you don’t have a direct match, I do a call to Yahoo BOSS’s news search and show you those results.

But now, Microsoft is handling all the search for Yahoo. What does this mean for the future of BOSS? I hope they keep it up and running. Here is the current news.

UPDATE: It looks like Yahoo’s developers don’t know what’s happening with it either.

What specifically does it mean for BOSS? Honestly the team is still absorbing the implications and we just don’t know. We can tell you that BOSS will remain live for the time being. There are many aspects still to be considered. Over the next several days we’ll be working hard to get clarity and will update the community as soon as we can.

TechCrunch writes this morning about Fever, a personalized RSS reader from Shaun Inman. I’ve been a fan of Mr. Inman’s for a long time, having bought installations of Mint, his stat-tracking program, for several of my sites. From TechCrunch:

Besides offering a full-featured feed reader, the application attempts to create a personalized Techmeme by scanning a user’s feed list for popular (or hot) links. Fever then groups these links into stories and assigns each a “temperature.” This allows a user to quickly keep a pulse on what’s going on in his or her “slice of the web.”

I’m glad to see that other smart people are trying to solve the same problems that I am. Helping people make the most of their limit time and attention is one of the biggest challenges of the next couple of years. Fever is an interesting take on it.

This took me forever to figure out, so I hope I’ll be able to save someone a few hours of annoyance someday.

Serendeputy is always recalculating, and I needed to make sure that the browsers wouldn’t cache the page when someone clicked off and then hit the back button. This is how I was able to do it.


..in application_controller.rb..
  before_filter :set_cache_buster
  def set_cache_buster
    response.headers["Cache-Control"] = "no-cache, no-store, max-age=0, must-revalidate"
    response.headers["Pragma"] = "no-cache"
    response.headers["Expires"] = "Fri, 01 Jan 1990 00:00:00 GMT"
  end


I just tested this out, and it works on Safari and Firefox on the Mac, and IE7, Firefox and Chrome on the PC.

Hope this helps.

I’m not obsessing too much about SEO for the new site. I’m just doing all the basic blocking and tackling, and focusing on the customer experience for now.

That said, Aaron Wall’s new SEO toolbar is pretty damn useful. I really like the integrations with all the other services. It seems to do a really good job pulling all this disparate information together.

Plus, it doesn’t phone home. That would have been a deal-breaker.

I’m racing to alpha, so things have been a little quiet here on the blog. Sorry ’bout that. I have a few items in the can, but they’re waiting until people can actually see the site.

While working on the site, I came across a post from Steve Outing pointing to the winners from a contest on how to reinvent classifieds in the newspaper world. This was my world a couple of years ago, so it’s very interesting reading.

I’m intrigued by the consignment method. It’s a little like running eBay postings in the paper. I wonder if that would be an interesting upsell for eBay…

I am, as you might predict, @serendeputy on twitter. If you’re interested in this little project, please give it a follow.

As I’m going into alpha, I want to make sure everyone can get ahold of me. If it works best to hold conversations in public, I’m more than happy to.

This was the last project I worked on before I left Boston.com: Newton hyperlocal.

I’m looking forward to seeing the rest of the towns. I can’t wait until they get a version of this for Holliston.

Good move by the Times in opening up the Times Tags API. I can’t wait to play with it.

I’m glad they are being so clueful about this. Opening up API access to your content is the next form of Advanced SEO, and the Times is on top of it. I hope the Globe is able to follow suit.

Greg Linden talks about how Digg may be working on personalized news. This is entirely expected. Every infomediary, aggregator and publisher needs to be working on technology like this if they want to survive. By 2011, every web user will demand that all websites be intensely personalized. Like Serendeputy.

Hubris alert: I realize I haven’t launched anything yet. Working on it…

I have GMail and Yahoo Mail accounts for testing, but I’ve never relied on them as the primary email address. This is why.

Jeff Atwood and team have built a new product called Stack Overflow, a question and answer site for programmers. It’s very early, but I think it will be successful. It’s certainly been useful for me.

For Serendeputy, I’m building an index of articles relevant to me, and I need to be able to generate a unique id based on the url. I experimented with (what appeared to be) the obvious solution, but it wasn’t working. So, I asked the question: “What’s the best way to hash a url in Ruby?” Within three minutes, I got a useful answer. By morning, I received three answers, with the best answer sorted to the top.

This is incredibly useful. I have a stack of Ruby books at my desk, and I’m often flipping through them trying to find a specific answer. I generally know *what* I’m trying to accomplish; I’m just lost on the syntax and the proper Ruby idiom to use. If my experience is typical, then Stack Overflow is a very useful supplement to these references.

Stack Overflow has a ton of Google juice already. My question is number one for the relevant search query: hash a url ruby. Most of the time when I do a Google search for specific questions, I get links to the Ruby documentation (which I already have) and a decent amount of off-topic spam. If Stack Overflow gets a critical mass of specific questions with canonical answers, then Google will become much more useful for everyone.

I’m very impressed with how they’ve focused on the customer experience. I’m trying to keep the same ideals with my project. Unlike those folks at that annoying site, they have everything open and clean. They have advertising, but it’s inobtrusive. And, the value of the product was so high that I actually made a point of looking at the advertising, seeing that an advertiser who’s associating with this useful an application is probably worth checking out. It’s amazing what optimizing for the customer experience will do for you.

It reminds me of how I felt in 1999 when I switched from Alta Vista to Google.

One quibble: Requiring people to get an OpenId is a mistake. Everyone in their target market has a simple throwaway handle/password combination that they use on all these sites. Unless it’s a pragmatic choice — they don’t have to program the authentication module — I think it’s a mistake to go against the grain of what people expect for these sites. Requiring an OpenId dissuaded me from registering; I just posted as a guest.

Alan is continuing to do a great job with The Big Picture. I really like the way that he’s handling sensitive pictures in this gallery covering the Georgian crisis.

Joel is unhappy about the terms and conditions a community site he was about to join imposed on him.

So I own the content, but they can do anything they want with it, even in forms that haven’t been invented yet. About the only concession that they grant to my ownership rights is that the license is non-exclusive. It says later that you can delete your profiles and that removes your content from the site, but they could still use it anywhere else, perpetually.

This is a hard one. I had to write a similar document for Life Times Voice back in the day, and you have a couple of conflicting issues.

First, if you want to maintain safe harbor under the DMCA, you need to be used just for communication. If you are taking copyright on the content, then you need to police it, and you’re responsible for any copyright infringement. If you don’t claim the copyright, you’re in the clear.

You need to balance that with the need to be able to reproduce it. The copyright owner has to give you explicit right to reproduce it. The rest of the clause is to protect you if you want to also use it in some unforseen format (podcasts of the best posts of the day, for example.)

I hope to be able to find a smart way to balance each of these needs in the new project. I want to do right by the customers (without crippling the business.)

Once upon a time, I was talking to SmugMug about working with Boston.com on a photo project. It didn’t end up happening, but I was very impressed at how much they had their act together and how they were actually making money. In 2004, this seemed a foreign concept to the other companies I contacted.

I’ve been following them ever since, especially as they’ve become a poster child for Amazon Web Services (AWS), the cloud-computing system I’m using for Serendeputy. Don MacAskill, the CEO, just wrote an outstanding piece explaining how SmugMug is using AWS to handle all their document processing.

SkyNet [SmugMug's main controller] is completely autonomous – it operates with with zero human interaction, either watching or providing interactive guidance. No-one at SmugMug even pays attention to it anymore (and we haven’t for many months) since it operates so efficiently. (Yes, I realize that means it’s probably well on its way to world domination. Sorry in advance to everyone killed in the forthcoming man-machine war.)

Roughly once per minute, SkyNet makes an EC2 decision: launch instance(s), terminate instance(s), or sleep. It has a lot of inputs – it checks anywhere from 30-50 pieces of data to make an informed decision. One of the reasons for that is we have a variety of different jobs coming in, some of which (uploads) are semi-predictable. We know that lots of uploads come in every Sunday evening, for example, so we can begin our prediction model there. Other jobs, though, such as watermarking an entire gallery of 10,000 photos with a single click, aren’t predictable in a useful way, and we can only respond once the load hits the queue.

I’m architecting my systems in a similar way, trying to build everything out so that it’s as decoupled and asynchronous as possible. If I can fire up only the machines I want and only when I need them, then I can bootstrap the organization far longer than I could if I had to buy the equivalent physical machines. The experiments and prototypes I’m working on would be prohibitively expensive without AWS.

Although SmugMug isn’t using it, I’m using SQS for managing all the communications between these instances. Keeping it all in one system reduces my headaches. Now, I just need Amazon to incorporate CouchDB and I’ll be really able to roll. SimpleDB is a start, but it’s not really meeting what I need.

I look forward to hearing more from Mr. MacAskill and SmugMug as they continue to innovate with AWS — especially if I can pick up more architecture hints…