I just spent a ton of time figuring this out, so I figured I’d toss it up on the blog in case someone else needs it.

Problem: I was exporting and manipulating the data in a MySQL database using the ruby mysql gem. Unfortunately, the people writing the data in the database were using curly quotes and other typographic marks that were causing trouble.

Solution:

require ‘rubygems’
require ‘mysql’

@dbh = Mysql.init
@dbh.options(Mysql::SET_CHARSET_NAME, ‘utf8′)
@dbh.real_connect(“localhost”,@dbuser,@dbpass,@dbname)

The key is to add a line in the options call to set the character set to UTF-8. This seems to solve the problem.

Hope this helps!

This may be obvious to everyone already, but it was new to me.

Serendeputy hits a few Tokyo Cabinets when it assembles a page. I was having trouble with these sometimes getting hung up, especially when looking up some of the ancillary information. I keep all the tag metadata in the main librarian cabinet. The site looks up this data pretty often. This would manifest itself on the site with a 500 error, and the Passenger Rails process would hang. The site would return 500s until I restarted the process. Ugh.

So, I did a couple of things (which I’ll likely talk about later.) The easiest one was probably the simplest.

I ended up adjusting the librarian build to not only populate the appropriate cabinet, but to also generate a ruby source code file in which I declared a constant with the tag metadata hash defined. I then restart Passenger to pick up the new constant.

Now, Rails only loads this once, and it’s in memory for the rest of the time. This not only saves me the network latency, it makes the individual responses much faster.

It’s always fun to solve complex problems with simple brute-force actions.

After living with Serendeputy for the past year and a half, I’ve been able to reduce all the massive complexity of the application into just three core concepts: the gesture, the profile and the list. By making all the interactions on the site and through the API go through these three primitives, I’ve been able to solve the scalability issues and most of the performance issues inherent in mass personalization. Now, it’s like Legos. Shiny, geeky Legos.

Thank goodness Ruby and Tokyo Cabinet are so flexible. If I’d been doing this using the tools of five years ago, I would not have been able to pivot this smoothly.

Look for a couple of fun announcements in the next [real soon now] weeks.

(And yes, Tokyo Cabinet is probably one of the geekiest topics I have up on the site, right up there with Machine Learning.)

This took me forever to figure out, so I hope I’ll be able to save someone a few hours of annoyance someday.

Serendeputy is always recalculating, and I needed to make sure that the browsers wouldn’t cache the page when someone clicked off and then hit the back button. This is how I was able to do it.


..in application_controller.rb..
  before_filter :set_cache_buster
  def set_cache_buster
    response.headers["Cache-Control"] = "no-cache, no-store, max-age=0, must-revalidate"
    response.headers["Pragma"] = "no-cache"
    response.headers["Expires"] = "Fri, 01 Jan 1990 00:00:00 GMT"
  end


I just tested this out, and it works on Safari and Firefox on the Mac, and IE7, Firefox and Chrome on the PC.

Hope this helps.

I’m working interactively in irb, and I tweak the class to make changes. The changes won’t be reflected in the irb session unless you reload the class.


irb >> load 'document.rb'


Make sure to add the file extension. Unlike require, load needs the full filename.

Jeff Atwood and team have built a new product called Stack Overflow, a question and answer site for programmers. It’s very early, but I think it will be successful. It’s certainly been useful for me.

For Serendeputy, I’m building an index of articles relevant to me, and I need to be able to generate a unique id based on the url. I experimented with (what appeared to be) the obvious solution, but it wasn’t working. So, I asked the question: “What’s the best way to hash a url in Ruby?” Within three minutes, I got a useful answer. By morning, I received three answers, with the best answer sorted to the top.

This is incredibly useful. I have a stack of Ruby books at my desk, and I’m often flipping through them trying to find a specific answer. I generally know *what* I’m trying to accomplish; I’m just lost on the syntax and the proper Ruby idiom to use. If my experience is typical, then Stack Overflow is a very useful supplement to these references.

Stack Overflow has a ton of Google juice already. My question is number one for the relevant search query: hash a url ruby. Most of the time when I do a Google search for specific questions, I get links to the Ruby documentation (which I already have) and a decent amount of off-topic spam. If Stack Overflow gets a critical mass of specific questions with canonical answers, then Google will become much more useful for everyone.

I’m very impressed with how they’ve focused on the customer experience. I’m trying to keep the same ideals with my project. Unlike those folks at that annoying site, they have everything open and clean. They have advertising, but it’s inobtrusive. And, the value of the product was so high that I actually made a point of looking at the advertising, seeing that an advertiser who’s associating with this useful an application is probably worth checking out. It’s amazing what optimizing for the customer experience will do for you.

It reminds me of how I felt in 1999 when I switched from Alta Vista to Google.

One quibble: Requiring people to get an OpenId is a mistake. Everyone in their target market has a simple throwaway handle/password combination that they use on all these sites. Unless it’s a pragmatic choice — they don’t have to program the authentication module — I think it’s a mistake to go against the grain of what people expect for these sites. Requiring an OpenId dissuaded me from registering; I just posted as a guest.

There’s undoubtedly a smarter way to do this, but I couldn’t figure it out. I’m doing a functional (controller) test for one of my rails pages. I want to make sure that the picture coming back is exactly 224 pixels high.

I couldn’t find a way to do this using the assert_select syntax, but this workaround seems to work:


  def test_picture_height
    assert_select "div.secondary div.lead_picture img[height]" do |height|
      # pull the tag -- this gives you the whole img tag
      @h = height.to_s
      # extract the height
      @h.gsub!(/.*height="(.*?)".*$/, '\1')
      #run the test
      assert_equal @h, "224"
    end
  end


This can be tightened up, but I left it explicit so that I could understand what I was going. The assert select makes sure that I’m setting the height. The block makes sure that it’s equal to 224.

Are there easier ways to do this?