Grace and Beauty by digitalART2

• The big surprise news this week was Apple announcing OS X Mountain Lion being launched this summer. Nothing massively ground-breaking but I’m excited about getting an integrated Notification Centre (bye Growl?).
• Trying to remember all these common grammar mistakes is difficult, but it’s worth a read any way. I’ve probably made one or two of them in this post.
• Cloudberry Kingdom looks like a really interesting platformer. It algorithmically generates levels based on difficulty settings, and ‘insane’ really does look insane. They’ve been working on it for a ridiculous amount of time, I hope it gets in to the Live Arcade store.
• It’s a bit buggy, but Caffeine Zone 2 is a free iPhone app for tracking your caffeine intake and telling you how much caffeine is in your system. (via Lifehacker)

Finally, possibly my oldest childhood friend (we were born within 1 month of each other and my parents are his godparents) got married this weekend. Congratulations Matt and Rosie! Here’s a cinemagr.am of them being bombarded by confetti:

And finally, I got some Ikea DIODER lights shipped this week. I saw lots of cool examples of them being used as backlighting for TVs so thought I’d try it out. The results are pretty cool, as you can see in the photo below (it’s not so bright in real life!).

via @4colorrebellion

Maciej Cegłowski, creator of Pinboard, posted an article this week called The Five Stages of Hosting. In it he compares different hosting options (SaaS, cloud, your own datacentre) to different types of housing. It’s worth a read because Maciej is a firm believer in bare metal hardware, something which I can relate to. In particular, on Virtual Private Servers he says:

Sometimes you will REALLY notice the neighbors, and can’t do anything about it.

Atomix, the videogame magazine for iPad, have just launched version 2 of their iPad application with support for Newsstand. In addition, they have made their back-catalogue (and new issue) completely free. I’m not sure about this as a business move, but obviously it’s good for users! The first few issues are particularly good, as they have some great video and editorial content from Area5 (where are they now?!). They purposefully made sure the content wasn’t time-specific, so it’s definitely worth checking out the back issues if you’ve never read them before.

There’s a new version of Backbone – the Javascript MVC framework – out this week. They’ve made some interesting changes. If you use Backbone be sure to check out the Change Log.

And finally, The Times have launched a cycle safety campaign this week after one of their journalists, Mary Bowers, was hit by a lorry just yards from work. I heard the accident, it was on the road just outside my window (5 floors up), and I witnessed the aftermath. I don’t think I’ll ever forget what I heard and saw. Please pledge your support and cycle safely.

• Chrome Web Timer has found a place next to my address bar in Chrome. It’ll be interesting to see how the statistics work out after a few weeks of using it. I’m probably going to suggest it to Pete like every other ‘focus’ tool I find.
• I’ve been spending a lot of time with the R&D 3D printer this week, and it’s getting better and better and less temperamental (though isn’t working right now!).  Therefore I expect Tinkercad, a tool for designing simple 3D models in your browser, will be useful in the future.
• Twitter Bootstrap has become my default CSS toolkit, with the only disadvantage being everyone else uses it too! So I’m excited and intrigued by the news that Bootstrap 2 is ready for testing. Looks like they’ve added some really interesting stuff.

And finally, I bought myself a new coffee toy this weekend, a Zassenhaus coffee mill. It’s excellent and goes really well with my Aeropress.

In Rails 3.1+ with the asset pipeline enabled any files that are not referenced by a manifest file (e.g. a require line) will not be precompiled in production. They may work fine in development / compiling on the fly but they won’t be touched by the assets:precompile rake task.

The solution is to add them to config.assets.precompile in your application.rb file. However there is a little gotcha. I had a file I wanted precompiled called admin.sass, so I wrote the following line:

config.assets.precompile += %w(admin.sass)

However this didn’t work. Eventually I found out it’s because the asset pipeline expects you to specify the resulting filename after compilation, rather than the un-compiled SASS/CSS filename. So I changed the line:

config.assets.precompile += %w(admin.css)

Everything worked fine then.

In fact, if you add a few extra coins, you can have the right change from 1p all the way up to £6.10.

The magic combination is:

• One penny
• 2 two pence
• 1 five pence
• 1 ten pence
• 2 twenty pence
• 1 fifty pence

Then optionally you can add:

• 1 pound coin
• 2 two pound coins

To work this out, use a greedy strategy; keep picking the biggest coin less than or equal to the remaining amount. You can keep doing that up through notes as well if you want.

Obviously this only works the first time you need exact change, but it seems like a great way of getting rid of your change. I find I can take a handful of change with me and still have to go “sorry, I don’t have the right amount” and end up with even more change.

Note I didn’t work this out myself. I saw the idea on Lifehacker and others have discussed the problem.

Twitter power users seem to be unimpressed. They are unhappy with the loss of 1 tap access to Direct Messages (even though it’s just a swipe away) and feel the direction Twitter are taking is turning Twitter into less of a platform and more of a product, and a way to push advertising and branded information to them. Kinda like the rest of the web.

We all knew Twitter couldn’t be so nice to everyone forever and the signs were there when they started hanging on to private API calls and not giving access to key features to developers. Everyone has always asked the question “How will Twitter monetize?” and now the answer is starting to be evident.

My point is: When Facebook change their layout and millions set up groups to campaign for reversion, social media ‘experts’ look down their noses and laugh. But for some reason when Twitter redesigns it deserves a serious critique. In my opinion it doesn’t, stop whinging.

Tell the mysql driver that you’re getting BINARY back, then use .force_encoding(“UTF-8″)?

In a Rails application you can do this by setting encoding: binary in your database.yml file (I’m also using the mysql2 driver, not sure if that makes a difference). It seems to work (tested in Ruby 1.9.2) and it makes more sense, but I’ll update if I come across any issues!

Original post follows.

Recently I had to deal with a MySQL database where the character encoding was set to latin1 (the default) but the bytes sent to it were UTF-8. When this happens you get horribly messy strings coming out of your database if you try and access it with anything sane. For example “ümlaut” becomes “ümlaut”. What a mess!

In any normal situation you would fix the data storage itself, but I had no control over the database so I had to work out another way of fixing it on my side. And I managed to do it!

Some background

Latin1 is a character encoding used by MySQL. People mistakenly think it is equivalent to ISO-8259-1 but it isn’t, it’s actually CP-1252 (also known as Windows-1252). CP-1252 is a superset of ISO-8259-1 with some additional characters (recently including the € symbol).

We can simulate what happens in MySQL when you forcibly send it UTF-8 with the following Ruby snippet:

"ümlaut".force_encoding("cp1252").encode("UTF-8")
# => "ümlaut"

Here, we start with UTF-8, then we tell Ruby that it’s actually CP-1252. It isn’t, so that just results in garbage. We then ask Ruby to give us that encoded back in to UTF-8.

When I realised this (thanks to Yehuda Katz for some pointers in his blog post about Ruby 1.9 Encodings) I tried running the reverse process against the text I was trying to fix:

"ümlaut".encode("cp1252").force_encoding("UTF-8")
# => "ümlaut"

Success! However when I put that code into my application and ran it across a much larger set of data, I encountered a new problem:

"(“What do you think?â€\u009D)".encode("cp1252")
                                 .force_encoding("UTF-8")
# => Encoding::UndefinedConversionError: U+009D to WINDOWS-1252
     in conversion from UTF-8 to WINDOWS-1252

Clearly strings coming out of MySQL aren’t strictly CP-1252.

It turns out that Windows API methods (and thus, perhaps, others) will accept certain decimal code values for characters that aren’t strictly accepted in the CP-1252 standard. Just 5 of them: 81, 8D, 8F, 90, and 9D. Looking at the error above, 9D is referred to so that must be what is going on!

I’m not 100% sure, but what I suspect is happening is MySQL is happily storing the 9D byte it receives, which when read by Ruby as UTF-8 comes out as the Unicode byte character \u009D. But Ruby is strict about encoding CP-1252 and won’t encode that byte because it considers it invalid. However if you drop those invalid bytes the resulting text won’t come out properly, because that byte is actually part of a multibyte Unicode character, so needs to be passed across to the encoded string.

To do this, you can use the badly documented :fallback option in the encode method. :fallback takes a hash where the key is the invalid character and the value is the character to replace it with. We want to replace all instances of \u009D etc. with \x9D (an escaped single byte character). One final caveat is that for Ruby to accept that as a replacement character, you must use force_encoding("cp1252") (essentially overriding Ruby’s lack of conversion). The resulting code looks like this:

def fix_cp1252_utf8(text)
  text.encode('cp1252',
              :fallback => {
                "\u0081" => "\x81".force_encoding("cp1252"),
                "\u008D" => "\x8D".force_encoding("cp1252"),
                "\u008F" => "\x8F".force_encoding("cp1252"),
                "\u0090" => "\x90".force_encoding("cp1252"),
                "\u009D" => "\x9D".force_encoding("cp1252")
              })
      .force_encoding("utf-8")
end

A bit messy, but it gets the job done. I believe Ruby 1.9.2 will not accept a dynamic hash in the :fallback option for some weird reason, but Ruby 1.9.3 will. Just bear that in mind.

In action:

fix_cp1252_utf8("(“What do you think?â€\u009D)")
# => "(“What do you think?”)"

Hurrah!

One really important caveat: If you pass that method nice tidy UTF-8 it will screw up the text. I haven’t figured out a way of stopping that; so be careful!

All you need to do, is add Compass to your Gemfile and you should be all set. You can even group it under assets (so long as you precompile assets before deploying).

But you might find it still doesn’t work, that’s because the stable version of Compass (0.11) doesn’t work with Rails 3.1. So you need to explicitly request version 0.12:

group :assets do
  gem 'sass-rails', "  ~> 3.1.0"
  gem 'coffee-rails', "~> 3.1.0"
  gem 'uglifier'
  gem 'compass', '~> 0.12.alpha'
end

Now you can carry on using the great Rails 3.1 asset pipeline with all the Compass and Blueprint mix-ins you want.

I’ve then received confirmation that my account will be closed within 48 hours, and then I had a further email from David Sherwin, who is the Web Traffic and Communication Manager at Prospects:

Hi David

Thanks for your email.

We are investigating the issues you have raised.

We have taken steps to deal with your immediate concerns.

Regards

David Sherwin

The email was CC’d to 6 other people within Prospects, which suggests they are taking my issues seriously, which I am pleased about.

Original post follows:

Today Prospects - ’the UKs official graduate careers website’ – decided to remind me by email that I still have an account with them. Just in case I had forgotten, they included my password in plaintext, which means either they are storing it in plaintext or they are encrypting it in a reversible manner (just as bad, I guess).

I have contacted them about this, and also emailed them asking them to delete my account as there doesn’t seem to be a way of doing it online.

The horrors don’t stop there though. If you visit their ‘Change my password’ page not only is communication not over SSL but for some ridiculous reason they embed your password as a value in the ‘Your existing password’ field even though it’s hidden. Someone actually made a conscious decision to do this. I would love to know why.

I don’t think I’ve ever seen something quite so stupid on the Internet.