DigitalHobbit

Earlier this year, I started playing with electronic music creation, mostly using Propellerhead Reason. There's a lot left to learn, and I don't have any masterpieces to show off yet, but it's been a fun, creative outlet so far.

I recently kicked off a music blog, where I'm planning to document my experience. Check it out and subscribe in case you're curious.

I recently bought a new iMac to replace my aging (3.5+ years old) MacBook Pro, and I love it! Since I've been getting some questions about it (and I haven't blogged in a while anyway), I figured I'd take this opportunity to write a quick blog post.

It took me a while to figure out what kind of computer to get:

• I considered getting another MacBook Pro, but I already have one at work, so I didn't really need another laptop.
• The Mac Mini is potentially interesting, but none of the configurations are perfect for me. The two consumer models only come with dual-core CPUs. The server model does come with a quad-core, but it only supports the basic, onboard Intel HD Graphics 3000 as opposed to a higher performing ATI or Nvidia video card. All Mac Minis also max out at 8GB. Sufficient for now, but does't leave much headroom for the future.
• I also toyed with the idea of building a Hackintosh. Until recently, I wasn't aware that this had become feasible, but it sounds like it very much has, and in fact one of my coworkers is very happy with his setup. It's certainly tempting to build a nice custom machine (something I haven't done since my Windows / Linux days), and tools like Kakewalk seem to make it pretty easy to install OSX on it. For less than $1000, significantly less than a comparable Apple computer, you can get a very powerful machine. But in the end, I was worried about the amount of time I would have to invest every time there is an OSX update, resolving driver incompatibilities and other issues. Time is my most valuable asset, and I'd rather invest more money now and be confident that I won't have to deal with any hassles.
• Last not least, my 24" Dell monitor was starting to show symptoms of an impending failure, which is what finally swayed me from a MacBook Pro or Mac Mini to a 27" iMac.

This is hands-down the best computer I've owned yet. Some specs:

• 3.4GHz Core i7: I chose this over the 3.1Ghz i5 as I'm starting to work with some CPU heavy music applications, so the extra horsepower (particularly the i7's Hyper-Threading) should come in handy.
• 27" 2560x1440 display: This is the largest display I've used yet, and it's gorgeous. The high resolution allows me to comfortably run multiple apps side by side, such as a browser window on one side and a text editor, email app, or even a music app on the other. To be perfectly honest I'm not a big fan of Apple's glossy displays (on MacBooks I always opt for the matte screen), but since I only use this iMac indoors, this is not as much of an issue. And pictures and videos do look great on the glossy screen.
• AMD Radeon HD 6970M 1GB: I don't do any hardcore gaming on my Mac and didn't see the need to upgrade this to the 2GB version (for an extra $100).
• 16GB RAM: Aftermarket! Apple charges a ridiculous $600 for this upgrade (the default is a measly 4GB). Amazon sells several 2x4GB kits from various brands for below $40, so you can get the same 16GB RAM for about $75. I went with two of these Crucial kits, which work great. This gives me more than enough RAM to run all the apps I need, including virtual machines and memory-hungry music apps.
• 256GB SSD + 1TB HDD: This $600 upgrade admittedly hurts, but I'm incredibly glad I opted for this. OSX and apps start amazingly quickly, and files open instantly. I'm writing more about my storage setup below.

A few additional notes:

The iMac has two Thunderbolt ports. While I don't have any existing Thunderbolt peripherals, this will be interesting in the future. One immediate possibility is to occasionally use the iMac as an external monitor for my MacBook Pro (2011 model with Thunderbolt). Unfortunately you need a pricey $50 Apple Thunderbolt Cable to hook this up, but it works flawlessly. Simply press Command+F2 on the iMac to put it into Target Display Mode, and repeat the same command to leave it later. Note that this does not work with older Macs with Mini DisplayPort instead of Thunderbolt. The two ports look identical and fit the same cables, but they are quite different.

The iMac supports a maximum of 16GB memory. Each of the 4 RAM slots supports up to 4GB, and the basic configuration comes with 2x2GB and the other two slots empty. The cheapest non-wasteful memory upgrade is therefore to add 2x4GB, yielding a total of 12GB. But in my opinion, given the current low RAM price of less than $40 for 2xGB, upgrading to 16GB is probably worth it.

There are many different ways to use a mixed SSD + HDD configuration. I ended up going with Matt Gemmell's solution, which is working out great. Read his blog post for more details, but the basic idea is to keep OSX and your home directory on the fast (but small) SSD, and selectively offload certain folders to the slow(er) (but large) HDD, by moving these and then symlinking them back to the home directory on the SSD. This approach saves precious SSD space by storing movies, music, and other bulky files on the HDD, while still being able to conveniently access everything from the home directory on the SSD.

Last not least, note that the hard drive is not user upgradable. That generally hasn't stopped me from upgrading similar devices in the past (such as my TiVo, original Xbox, or 2008 pre-unibody MacBook Pro), but aside from the mechanical challenge, it looks like Apple has introduced some additional issues with this HDD upgrade. Therefore, you'll want to make sure that you're going to be happy for the next couple years with whatever storage configuration you go with. Of course, purely for data storage purposes, it's easy to hook up additional hard drives via USB or Thunderbolt. But you probably want your boot drive to be internal, so if you think you might want to boot from a fast SSD, you should purchase this as part of your iMac configuration.

Last weekend, I migrated my blog from Wordpress to Jekyll.

Wordpress is a solid blog platform and has served me well for the past 7 years, but at this point I simply don't need anything that sophisticated. The frequent security exploits have also become a bit of a liability, and in fact my blog was already hacked in the past. To be fair, the Wordpress development team tends to react quickly with security updates, but I simply don't have the diligence to keep my Wordpress installation up to date all the time. I've also never been a fan of the Wordpress web interface for writing posts, and in fact I have switched to Markdown and MarsEdit several years ago.

Jekyll simply takes this another step further (or backwards, depending on how you want to look at it), by eschewing server-side functionality altogether. Instead, content lives in simple markdown text files and is rendered into static HTML files that can then be served up by a regular webserver like Apache or Nginx, or even hosted on Amazon S3. It's fairly basic, but allows me to serve posts, feeds, and other pages. Comments are elegantly handled by using third party services such as Disqus (as in my case) or the Facebook Comments plugin. Jekyll is extensible via its plugin architecture. Last not least, it is written in Ruby, which gives me the option to fork it and completely adapt it to my needs if necessary.

I'll leave it at that. There are already plenty of posts out there about using Jekyll in general, or migrating to Jekyll from Wordpress or other blog platforms.

With any luck, this change will have the nice side effect of motivating me to post more frequently on my blog again, which has been dormant for the past year or so.

Like many iPhone apps, the app I'm currently working on uses several table views. Most of these display actual lists of data, and some are used as a convenient layout mechanism for input fields, navigation, and other UI elements (similar to iPhone preference screens).

UITableView and its associated classes like UITableViewCell, UITableViewDataSource, and UITableViewDelegate are very powerful, but they also require a fair amount of boilerplate code split across several methods. The specific matter that I am tackling in this post is the creation of cells, which happens inside the [UITableViewDataSource tableView:cellForRowAtIndexPath:] method. When dealing with only a single type of cell, it typically looks like this:

- (UITableViewCell *)tableView:(UITableView *)tableView cellForRowAtIndexPath:(NSIndexPath *)indexPath {
    // See if there's an existing cell we can reuse
    UITableViewCell *cell = [tableView dequeueReusableCellWithIdentifier:@"Foobar"];
    if (cell == nil) {
        // No cell to reuse => create a new one
        cell = [[[UITableViewCell alloc] initWithStyle:UITableViewCellStyleDefault reuseIdentifier:@"Foobar"] autorelease];

        // Initialize cell
        cell.selectionStyle = UITableViewCellSelectionStyleBlue;
        cell.textLabel.textColor = [UIColor blueColor];
        // TODO: Any other initialization that applies to all cells of this type.
        //       (Possibly create and add subviews, assign tags, etc.)
    }

    // Customize cell
    cell.textLabel.text = [NSString stringWithFormat:@"Row: %d", indexPath.row];
    // TODO: Any other customization
    //       (Possibly look up subviews by tag and populate based on indexPath.)

    return cell;
}

As you can see, there's a lot of boilerplate code here. This works well enough with one type of cell, but if you're dealing with multiple types of cells (particularly in a grouped table view), this approach quickly gets out of hand. You end up with a long method with large, ugly switch statements. But if you look at this method closely, you'll notice that there are only a few cell-specific areas:

1. The cell identifier (MyCell in my example). This is used to look up and reuse existing cells (e.g. when scrolling through a large table of items) and avoids the costly creation of new cells every time. It's a standard cell creation pattern for the iPhone and makes a lot of sense, but it also means that the cell specific code is spread across several places.

2. The initialization code. This is where a cell of a given type is initialized for the first time. If you can get away with the standard cell styles (which cover a few different layouts of labels and images), you usually don't need to do much here, besides setting your colors, fonts, and perhaps selection style. Otherwise, this is where you want to create and add your subviews, and assign a tag to them so you can populate them later.

3. The customization code. Given a cell with the correct properties and subviews (which may have been reused or created during this call), this is where you populate it with the correct data. This typically involves looking up some sort of data based on the indexPath, and setting it either on the cell itself (using the textLabel, detailTextLabel, or imageView properties) or on its subview. The latter requires looking up the subviews using the tags you've assigned earlier.

With this in mind, I decided to factor out all the cell specific code, resulting in a generic method in my base view that can be used to create all the cells of my app. Here's what that method looks like:

- (UITableViewCell *)createCellForIdentifier:(NSString *)identifier
                                   tableView:(UITableView *)tableView
                                   indexPath:(NSIndexPath *)indexPath
                                       style:(UITableViewCellStyle)style
                                  selectable:(BOOL)selectable {
    UITableViewCell *cell = [tableView dequeueReusableCellWithIdentifier:identifier];
    if (cell == nil) {
        cell = [[[UITableViewCell alloc] initWithStyle:style reuseIdentifier:identifier] autorelease];
        cell.selectionStyle = selectable ? UITableViewCellSelectionStyleBlue : UITableViewCellSelectionStyleNone;
        
        SEL initCellSelector = NSSelectorFromString([NSString stringWithFormat:@"initCellFor%@:indexPath:", identifier]);
        if ([self respondsToSelector:initCellSelector]) {
            [self performSelector:initCellSelector withObject:cell withObject:indexPath];
        }
    }
    
    SEL customizeCellSelector = NSSelectorFromString([NSString stringWithFormat:@"customizeCellFor%@:indexPath:", identifier]);
    if ([self respondsToSelector:customizeCellSelector]) {
        [self performSelector:customizeCellSelector withObject:cell withObject:indexPath];
    }
    return cell;
}

Structurally, this method is very similar to the previous one. It first tries to reuse an existing cell, creating and initializing a new one if it doesn't find one. It then customizes the cell. You'll notice that I'm using some performSelector calls here, coupled with a naming convention. For example, given an identifier of "Foobar", I will look for initCellForFoobar:indexPath and customizeCellForFoobar:indexPath to initialize and customize this cell respectively. A simple example might look like this:

- (void)initCellForFoobar:(UITableViewCell *)cell indexPath:(NSIndexPath *)indexPath {
    cell.textLabel.textAlignment = UITextAlignmentCenter;
    cell.textLabel.textColor = [UIColor blueColor];
    cell.textLabel.font = [UIFont boldSystemFontOfSize:16.0];
}

- (void)customizeCellForFoobar:(UITableViewCell *)cell indexPath:(NSIndexPath *)indexPath {
    cell.textLabel.text = [NSString stringWithFormat:@"Row: %d", indexPath.row];
}

Note that both methods are optional. In some cases (particularly in table views that are used for preferences or similar types of UI elements), there's only a single cell of any given type, so I perform the complete initialization in initCellForFoobar and omit the customizeCellForFoobar method. In other cases, I may not require any special initialization and only have a customizeCellForFoobar method.

Obviously, the example above is trivial, and both methods can get significantly longer when dealing with custom subviews. In that case I am using the same tag based approach I mentioned above: Assign a tag to each subview inside initCellForFoobar, then look up the subview using the tag in customizeCellForFoobar. But the important thing is that the code is well factored, and the cell specific code is not mixed with boilerplate code.

Last not least, an example of the actual UITableViewDataSource method to create a new cell:

- (UITableViewCell *)tableView:(UITableView *)tableView cellForRowAtIndexPath:(NSIndexPath *)indexPath {
    NSString *identifier;
    BOOL selectable = YES;
    UITableViewCellStyle style = UITableViewCellStyleDefault;

    switch (indexPath.section) {
        case 0:
            identifier = @"Foo";
            break;
        case 1:
            identifier = @"Bar";
            selectable = NO;
            break;
    }
    
    return [self createCellForIdentifier:identifier
                               tableView:tableView
                               indexPath:indexPath
                                   style:style
                              selectable:selectable];
}

The example above is what I would typically use for a grouped table view, where each section contains a specific type of cell. But obviously this approach supports any type of table view, grouped or non-grouped. You just need to plug in your own logic to determine the type of cell, and leave everything else to the createCellForIdentifier:tableView:indexPath:style:selectable method we created earlier.

There are probably other approaches for simplifying working with table views, but this approach has worked very well for me. It really cuts down significantly on the amount of boilerplate code and allows me to focus on the actual application specific code.

Any questions, suggestions for further improving this, or perhaps alternative solutions that you've used? Leave a comment!

It's been way too long since my last programming focused blog post, so let's try to rectify this situation:

Background

A couple months ago, Twitter made available their Streaming API. This provides developers with a very efficient way to tap into the public Twitter stream. All you need to do is open and maintain a single HTTP connection, passing in a few filter parameters. Twitter then keeps streaming matching tweets to you. You have the option of either sampling the entire public stream, or passing in a list of keywords and / or user ids to track. In this post I will focus on the latter, but the basic usage remains the same.

My interest was piqued when I came across the excellent TweetStream library, which makes it trivially easy to write a Ruby client application for the Twitter Streaming API. I decided to take this opportunity to play with some other technologies and write a simple web app that displays a subset of tweets, along the lines of cursebird or twistori.

The app I came up with is Twatcher, so go check it out to get an idea of what I'm talking about. It (admittedly very crudely) identifies funny tweets by looking for tweets that contain the word "lol". It then renders matching tweets using a simple UI not unlike that of twitter.com itself, and visually highlights the word "lol" in each tweet for emphasis. Perhaps most importantly, the app uses AJAX to periodically (currently every 10 seconds) pull in new tweets.

Architecture

In the remainder of this post, I will describe the architecture of Twatcher, along with the rationale behind it. I will also share some code snippets that should allow you to follow along and build your own Twitter filter app.

Given that the connection to the Twitter Streaming API has to happen in its own application (let's call it our filter app), outside of the actual web app, we need a way for it to pass tweets to the web app. There are a couple of options here. Obviously we could store the tweets in a database like MySQL, and have the web app read them from there. But given the small schema (only tweets; we don't care about users or any other relational data) and the ephemeral nature of the Twatcher app (at any given time we really only care about the N most recent tweets), this seems like overkill and leads to an unnecessarily write-heavy app. Instead, one of the various in-memory key/value stores seems like a much better fit. I first thought of memcached, but while it would be entirely possible to build this type of app on top of memcached, it's not ideal. When a new tweet comes in from the Streaming API, we need to append it to our in-memory list of tweets. Memcached is a very low-level data store and only supports string values, so we would have to implement lists by serializing them as YAML, JSON, or binary Ruby objects. Either way would mean that instead of writing just the latest tweet, we would always have to re-write the entire list of tweets. Similarly, on the web app side, we would always have to read the entire list, even though we may only be interested in the 5 most recent tweets (say in our AJAX action). Combined, this would lead to a fair amount of overhead on both the networking as well as Ruby processing side.

Luckily, there's another data store that is perfect for this type of app: Redis. On the most basic level it can be thought of as a key/value store like memcached, so it can act pretty much like a drop-in replacement for this. But it also has first class support for basic data structures, such as sets and lists. This means that instead of reading and writing entire lists of tweets, we can append a single tweet at a time, and we can efficiently retrieve the exact number of tweets that we need on the web app side (i.e. 20 for a full page view, and 5 for an AJAX request). Redis is stable, highly performant, and has a solid, extremely easy to use Ruby library. It also supports basic persistence, although we won't need this for our app.

With the filter app and data store out of the way, that leaves the actual web app. Our requirements are very humble: We will only have two actions (one for the full page of tweets and one for AJAX updates), and perhaps a few more trivial actions in the future for things like help pages. Since we're not using a relational database, we don't need any sort of ORM layer. While we could use Ruby on Rails, this would mean shooting sparrows with cannons. For our purposes the Sinatra micro-framework seems like a much better fit.

I'm a big fan of HAML, so we'll use this for our views. Of course there's nothing HAML specific about our app, so you're welcome to use ERB or your template language of choice instead.

We'll use jQuery as our Javascript library, mainly for AJAX requests and basic visual effects (so we can smoothly slide new tweets into the existing page). Once more, our needs are simple, and I'm sure any of the popular Javascript frameworks would be more than up to the challenge. But my personal preference is jQuery.

I won't go much into the deployment side of things, but twatcher.com relies on the usual suspects: Nginx (Apache would work fine as well), Passenger (you're welcome to use Mongrel, Thin, etc.), Capistrano, and God (to start and monitor Redis and our filter app, though I may end up giving Bluepill a try). All of this runs very smoothly on a 256MB VPS slice on Webbynode (and I'm sure just as well on Slicehost or Linode). If necessary, we could easily scale up this app by bringing up additional Sinatra slices and adding HAProxy to the mix (or perhaps even just relying on DNS round robin).

Code

Now that the architectural overview is done, let's take a look at some of the code. This isn't the complete code base that I'm using on the site, but it's a fully functional subset and hopefully enough to demonstrate the overall approach and get you started. Alternatively, you can grab the code from the twatcher-lite Github repository. I will eventually make the complete project (which includes configuration options, RSpec specs, etc.) available on Github as well.

But first a couple of prerequisites: We need to install a bunch of gems. For a production app, I would typically unpack these into the vendor directory, but for now let's just install them system-wide:

sudo gem install tweetstream yajl-ruby ezmobius-redis json haml rack sinatra shotgun

You also need to install and start Redis. It's easy enough, but beyond the scope of this blog post. Simply follow these instructions (I highly recommend the entire Redis article series by the way), but make sure to use the latest Redis release from the official website (currently 1.02) rather than the 1.0 version mentioned in the article.

Filter App

twitter_filter.rb:

This is the standalone filter app that mainly relies on the TweetStream library to retrieve tweets and then pushes them to Redis. In our final app we would want to use the Daemons library to run this app as a proper daemon, but for now you should be able to simply run it directly from the command line. Note that it relies on two additional files below. Simply place all of these into the same folder.

Make sure to set USERNAME and PASSWORD to your actual Twitter credentials. A word of caution: Apparently Twitter only allows a single Streaming API connection for standard accounts, and they will disconnect or blacklist you if you attempt to start multiple connections. I'm using a dedicated Twitter account for production, and my regular Twitter account during development. The actual version of this file that I'm using reads the (environment specific) credentials from a YAML file, but I didn't want to distract from the core functionality for the purpose of this tutorial.

require 'tweetstream'
require File.join(File.dirname(__FILE__), 'tweet_store')

USERNAME = "my_username"  # Replace with your Twitter user
PASSWORD = "my_password"  # and your Twitter password
STORE = TweetStore.new

TweetStream::Client.new(USERNAME, PASSWORD).track('lol') do |status|
  # Ignore replies. Probably not relevant in your own filter app, but we want
  # to filter out funny tweets that stand on their own, not responses.
  if status.text !~ /^@\w+/
    # Yes, we could just store the Status object as-is, since it's actually just a
    # subclass of Hash. But Twitter results include lots of fields that we don't
    # care about, so let's keep it simple and efficient for the web app.
    STORE.push(
      'id' => status[:id],
      'text' => status.text,
      'username' => status.user.screen_name,
      'userid' => status.user[:id],
      'name' => status.user.name,
      'profile_image_url' => status.user.profile_image_url,
      'received_at' => Time.new.to_i
    )
  end
end

tweet_store.rb:

This is a thin abstraction layer on top of Redis that encapsulates both pushing and retrieving tweets. This allows us to keep Redis specific persistence code out of the filter and web apps and also comes in handy for testing (which I'm not getting into in this post), as we can easily swap it out for a mock implementation.

Note how we're using the push_head operation to push a single tweet to Redis, and list_range to retrieve the N most recent tweets.

require 'json'
require 'redis'
require File.join(File.dirname(__FILE__), 'tweet')

class TweetStore
  
  REDIS_KEY = 'tweets'
  NUM_TWEETS = 20
  TRIM_THRESHOLD = 100
  
  def initialize
    @db = Redis.new
    @trim_count = 0
  end
  
  # Retrieves the specified number of tweets, but only if they are more recent
  # than the specified timestamp.
  def tweets(limit=15, since=0)
    @db.list_range(REDIS_KEY, 0, limit - 1).collect {|t|
      Tweet.new(JSON.parse(t))
    }.reject {|t| t.received_at <= since}  # In 1.8.7, should use drop_while instead
  end
  
  def push(data)
    @db.push_head(REDIS_KEY, data.to_json)
  
    @trim_count += 1
    if (@trim_count > 100)
      # Periodically trim the list so it doesn't grow too large.
      @db.list_trim(REDIS_KEY, 0, NUM_TWEETS)
      @trim_count = 0
    end
  end
  
end

tweet.rb:

The Tweet class wraps an individual tweet's data hash and allows us to access the data using method call syntax (tweet.username) rather than hash element references (tweet['username']). It also contains some tweet related functionality, such as generating Twitter user links, highlighting the word "lol", and making URLs clickable.

class Tweet
  
  def initialize(data)
    @data = data
  end

  def user_link
    "http://twitter.com/#{username}"
  end

  # Makes links clickable, highlights LOL, etc.
  def filtered_text
    filter_lol(filter_urls(text))
  end

  private
  
  # So we can call tweet.text instead of tweet['text']
  def method_missing(name)
    @data[name.to_s]
  end
  
  def filter_lol(text)
    # Note that we're using a list of characters rather than just \b to avoid
    # replacing LOL inside a URL.
    text.gsub(/^(.*[\s\.\,\;])?(lol)(\b)/i, '\1<span class="lol">\2</span>\3')
  end
  
  def filter_urls(text)
    # The regex could probably still be improved, but this seems to do the
    # trick for most cases.
    text.gsub(/(https?:\/\/\w+(\.\w+)+(\/[\w\+\-\,\%]+)*(\?[\w\[\]]+(=\w*)?(&\w+(=\w*)?)*)?(#\w+)?)/i, '<a href="\1">\1</a>')
  end
  
end

Web App

twatcher.rb:

This is the actual Sinatra web app. This is the entire app (minus the views), so perhaps now you can see why we're using Sinatra instead of a full-blown Rails app. The views follow below.

Note that our two actions both return tweets. The main difference is that the /latest action (which is used by AJAX requests) only returns up to 5 tweets, and only if they're newer than the specified date. It also omits the layout and specifies a special CSS class named latest. This allows us to initially hide the new tweets and then make them visible using a nice Javascript slide effect.

require 'sinatra'
require 'haml'
require File.join(File.dirname(__FILE__), 'tweet_store')

STORE = TweetStore.new

get '/' do
  @tweets = STORE.tweets
  haml :index
end

get '/latest' do
  # We're using a Javascript variable to keep track of the time the latest
  # tweet was received, so we can request only newer tweets here. Might want
  # to consider using Last-Modified HTTP header as a slightly cleaner
  # solution (but requires more jQuery code).
  @tweets = STORE.tweets(5, (params[:since] || 0).to_i)
  @tweet_class = 'latest'  # So we can hide and animate
  haml :tweets, :layout => false
end

views/layout.haml:

A pretty simple layout.

!!! Strict
%html{:xmlns=> "http://www.w3.org/1999/xhtml", 'xml:lang' => "en", :lang => "en"}
  %head
    %meta{'http-equiv' => "Content-Type", 'content' => "text/html; charset=utf-8"}
    %title twatcher
    %link{:rel => 'stylesheet', :href => '/stylesheets/style.css', :type => 'text/css'}
    %script{:type => 'text/javascript', :src => 'http://ajax.googleapis.com/ajax/libs/jquery/1.3.2/jquery.min.js'}
  %body
    #container
      #content
        = yield

views/index.haml:

The actual HTML content is pretty minimal: A heading and a list of tweets, which is included from a separate file (below) so we can reuse it for the AJAX action. We're also inlining some jQuery code to refresh the tweets every 10 seconds. We insert the new tweets at the beginning, but remember that we're using a CSS class to initially hide them. We then call slideDown to make them visible using a nice slide effect. We also trim the list of tweets at 50 to prevent the page from getting too long.

:javascript
  function refreshTweets() {
    $.get('/latest', {since: window.latestTweet}, function(data) {
      $('.tweets').prepend(data);
      $('.latest').slideDown('slow');
      $('.tweets li:gt(50)').remove();
      
      setTimeout(refreshTweets, 10000);
    });
  }
  $(function() {
    setTimeout(refreshTweets, 10000);
  });
  
%h1 Recent LOL Tweets
%ul.tweets
  = haml :tweets, :layout => false

views/tweets.haml:

Simply renders a list item for each tweet, with some basic CSS for styling purposes. I wouldn't normally hardcode height and width for an img tag (and instead let CSS handle this), but for the purpose of this tutorial I wanted the page to render decently without a style sheet, and the Twitter profile pictures can be pretty large, making it look weird.

We also emit some simple Javascript to record the timestamp of the most recent tweet. We pass this into our AJAX request in index.haml above. An alternative solution (perhaps slightly cleaner from an HTTP perspective) would be to use the Last-Modified HTTP header instead of a Javascript variable, but this would mean messing with date parsing (never fun...) and also result in slightly more complex jQuery code, so I opted for the simpler solution.

- @tweets.each do |tweet|
  %li.tweet{:class => @tweet_class}
    %span.avatar
      %a{:href => tweet.user_link}
        %img{:src => tweet.profile_image_url, :alt => tweet.username, :height => 48, :width => 48}
    %span.main
      %span.text= tweet.filtered_text
      %span.meta== &#8212; #{tweet.name} (<a href="#{tweet.user_link}">@#{tweet.username}</a>)

- if !@tweets.empty?
  :javascript
    window.latestTweet = #{@tweets[0].received_at};

public/stylesheets/style.css:

The stylesheet is pretty basic, but since this blog post is already way too long, I'm not going to reproduce it here. Simply grab the live one instead.

Putting it all together

You should now have a bunch of Ruby files in the same folder, and three HAML files in a views subdirectory. Make sure you have started Redis according to the instructions above. Then open two shells:

In your first shell, start the filter app:

ruby twitter_filter.rb

The app should start and continue running until you hit CTRL+C.

In your second shell, start the web app. You could simply start it using:

ruby twatcher.rb

However, assuming you've installed the Shotgun gem according to the instructions above, I recommend using the following command instead:

shotgun twatcher.rb

This will cause Sinatra to automatically reload modified files during development, similar to the default behavior in Rails.

If everything started successfully, you should now be able to bring up the site in your browser. If you've started the web app by itself, hit port 4567. With Shotgun, hit port 9393 instead.

Conclusion

I hope you can appreciate how little code it took us to implement a complete Twitter filter web application, complete with AJAX updates. I count around 150 lines of code, and this includes plenty of comments and whitespace (granted, including the stylesheet it would be closer to 300 lines).

I also hope I've managed to pique your curiosity about Redis, Sinatra, and the TweetStream library. Many of us (myself included) tend to stick with the tools we're familiar with, such as Rails and MySQL. But often, surprisingly elegant solutions emerge when using better-suited (and often simpler) tools.

Personally, I am excited about adding Redis and Sinatra to my standard toolset. I am also curious about what other types of applications might be able to get away with simple, ephemeral solutions like this. Definitely something worth exploring...

Yesterday, Amazon finally delivered the Magic Mouse that I had pre-ordered a couple weeks ago. (Since I was in the market for a Bluetooth mouse anyway, this seemed like a perfect excuse to pick one up...)

The final verdict is still out, but I figured I'd share my initial impressions. Obviously, the biggest feature the Magic Mouse adds is Multi-Touch, very much like the iPhone or iPod Touch. And I can confirm that this indeed works flawlessly. Flicking the finger up or down to scroll through windows is an amazing improvement over using a mouse wheel, let along a scroll bar. Momentum scrolling works just like on the iPhone, feels very intuitive, and adds a tangible touch to most applications.

I haven't gotten used to the two-finger sideways swipe yet, for example to navigate back and forward in the browser. It works well enough, but I'm used to using the keyboard to navigate back and forward or switch tabs, so I don't typically reach out for the mouse for this task. But if you usually do, it should come in handy.

I've never owned a Bluetooth mouse before and wasn't 100% sure what to expect, but I have no complaints at all about this. Pairing it with my Macbook was very easy, and it works just as well as a USB mouse, without any annoying cables. This is especially handy for me because I tend to frequently switch the mouse between my left and right hands to suppress RSI symptoms, and not having to deal with a cable definitely makes this easier.

Now for the things that are not that great:

Most importantly, the tracking is way too slow, almost to the point of not being usable at all. The acceleration is high enough, so when moving the mouse quickly it is possible to move it from one edge of the screen to the other without having to lift it. But this doesn't help for finer movements, such as selecting an item from a menu or a toolbar. Many people are complaining about this issue, but thankfully this article describes a workaround in form of a terminal command to change the scaling factor. You can also use the MouseZoom tool to accomplish the same thing using a convenient preference pane. It's old but works just fine in Snow Leopard.

I'm now running on the highest possible scaling setting, which is a major improvement over the standard configuration, though I might end up tuning things down a bit. Overall I've never been very happy with the mouse support in OSX. The acceleration curve just feels off to me -- it starts off too slow, then accelerates too quickly. This article describes this issue in detail, and I agree with it. It also describes some solutions, such as USB Overdrive and SteerMouse. I have bought and used SteerMouse in the past, and it worked pretty well for me then. It doesn't support the Magic Mouse yet, and according to the website they are currently evaluating whether to add this functionality. I'm not sure if any of these options are compatible with the Magic Mouse, which I'm sure requires its own driver to support the Multi-Touch functionality. I'll have to experiment with this...

The mouse button (a single button that is sensitive to where it's being touched and can therefore emulate left and right mouse buttons) works ok, but clicking requires a bit too much effort for my taste (my previous Logitech mouse was significantly more sensitive, requiring barely any pressure). In fact I find operations that require moving the mouse with the button pressed (such as when selecting several paragraphs of text in a document) somewhat difficult.

Last not least, I miss having a middle mouse button, mainly to open links in new browser tabs or copy & paste text in the terminal. I suppose I will have to get used to holding down the Command key instead. I wonder if the mouse surface is sensitive enough that it could differentiate between left, middle, and right clicks.

Moving the mouse over my wooden desk is fairly noisy. Unlike my Logitech mouse, which was quiet and smooth, the Magic Mouse almost feels like it's scraping the surface. I may end up getting a mouse pad for this reason, but this seems to defeat the purpose of the new and improved laser technology that works on any surface.

In terms of ergonomics it's not the best mouse in the world, but it's not horrible either. It's still too early to tell how well I adjust to it. We'll see in a few weeks...

In conclusion, the Magic Mouse is definitely a fascinating piece of technology, and I am excited about future iterations of this or similar products. I absolutely love the touch based interface of my iPhone, and the Magic Mouse does a good job at bringing some of this to the desktop. But it certainly has its share of flaws. Hopefully a driver update will resolve the slow tracking issue soon, but in the mean time the workaround described above will need to suffice. As for the other issues: If you can, I recommend you try out the mouse in the Apple Store first, to see how it feels for you.

If you own a MacBook Pro and would like to upgrade to more than 4GB of RAM but think that your model does not support this, you may want to read the rest of this article.

I bought a MacBook Pro 17" in April 2008, as my primary development machine. I knew that the standard config with 2GB of RAM wouldn't be enough for my purposes, but I also wasn't about to spend a ridiculous amount of money on an official memory upgrade from Apple, so I picked up two cheap G.SKILL 2GB DIMMs from Newegg.

This worked great for me so far, but even with 4GB of RAM, I occasionally ran into memory limits. For example I sometimes work on iPhone and complementary Rails apps at the same time, and having both Xcode (plus Interface Builder and the iPhone Simulator) and a Rails app, IDE, etc. running at the same time definitely uses a fair amount of memory. Especially if I use RubyMine (which is pretty nice, by the way, but a major memory hog). That's one of the reasons why I often still work with a regular text editor such as TextMate. The situation gets even worse when I need to run a virtual machine, such as for IE browser testing. And of course there are all the other memory hungry apps that tend to be running all the time (Firefox and / or Safari, iTunes, etc.).

The last time I researched potential memory upgrades, I quickly discovered that my model (apparently) only supports a maximum of 4GB, so I gave up.

But this time I complained on Twitter, and a reply prompted me to research this issue more closely. Well, it turns out that many MacBook Pro models do indeed unofficially support 6GB of RAM, in form of 4GB + 2GB DIMMs. This MacRumors Guide has all the info you need. My model appears to be the Rev. E (as identified by the date of purchase, as well as the CPU frequency, video card, and video memory). And sure enough, the Rev. E and F models can handle up to 6GB of RAM.

Since I was already using G.SKILL memory, I opted for a 4GB G.SKILL DIMM (currently $129 at NewEgg). I would not recommend mixing DIMMs from different manufacturers, and in fact I have read some reports of people having trouble getting these configs to work.

The actual memory upgrade process is quick and easy (at least on the pre-unibody models), and Apple provides a convenient guide.

I should point out that due to the mixed (4GB + 2GB) memory configuration, you lose the Dual Channel capability. But based on what I read, this only affects certain types of apps and makes little difference in practice. I definitely didn't notice any lower performance after the upgrade.

The increased memory means that my system rarely (if ever) has to swap. Now I can run my whole development stack as well as two virtual machines (Windows and Linux) and the machine is still very responsive.

Now I just have to find some new memory intensive applications to bring my system down to its knees... ;)

Update: This upgrade was the single biggest bang for the buck and has made a tremendous difference on my system. Having 6GB instead of 4GB was exactly the additional RAM I needed to be able to run all my various development tools at the same time. I am now running several Rails apps and have one RubyMine as well as two Xcode projects open without any issues, along with the usual productivity software, iTunes, etc. Definitely highly recommended!

It's been a while since I completed my first iPhone development project, and I figured I'd finally write up my initial experience with this platform. A bit late, but better than never...

To put this into perspective, here's a brief summary of my previous professional programming background:

I worked with C++ back in University but quickly adopted Java in 1998 and never looked back (good riddance to pointers and manual memory management!). Java served me well for the better part of the past 10 years, but I've increasingly become a fan of dynamic languages, and Ruby in particular (although I've dabbled a bit with Python as well). These days I mainly work with Ruby (mostly Ruby on Rails, but also standalone apps such as daemons, command line apps, etc.). Among many other reasons, I just love its expressiveness over Java's verbosity.

With this in mind, I was admittedly a bit hesitant at first about iPhone development using Objective-C, even though I was definitely curious about the CocoaTouch platform (and Cocoa in general).

Objective-C

At first glance, Objective-C syntax looks quite odd. Instead of the dot-notation for method calls that Java and Ruby use (e.g. "foobar".length), Objective-C uses a square bracket based message passing syntax (e.g. [@"foobar" length]). This is harmless enough for simple method calls, but can become confusing when using nested calls (e.g. [[[MyClass alloc] init] autorelease]). Apparently, Apple decided this was the case as well, so when they introduced their simplified support for properties in Objective-C 2.0, they chose to go with dot-notation. This definitely cuts down on some of the clutter (especially when setting properties), but it also leads to a slightly awkward, inhomogeneous mix of notations in the code, as dot-notation cannot be used for regular method calls (and in fact some long-time Objective-C developers are boycotting the new properties syntax for that very reason). But in the end, this is a relatively minor stylistic difference, and I eventually got used to it.

The other Objective-C oddity is the Smalltalk-inspired way that parameters are passed to methods. Most current programming languages pass parameters as comma-separated lists. Objective-C essentially breaks method names up into multiple segments, each of which has its own parameter. This means that you end up with method signatures like this:

- (void)insertSubview:(UIView *)view atIndex:(NSInteger)index;

Invoking this method would look like this:

[myView insertSubview:mySubview atIndex:3];

It felt weird at first, but once I got my head wrapped around this syntax, I actually kind of started to like it, as it makes method calls read like regular English sentences. I have however found that it can be difficult to come up with natural sounding method and parameter names for my own classes, so my preference would be a C-based method call syntax with named params like Python has. But in the end, I got used to this syntax fairly quickly.

Probably the thing that bothers me most about Objective-C / Cocoa is the overall verbosity, especially compared to a very succinct language like Ruby. This is partially because variables need to be declared, partially because Cocoa's method names tend to be long, and of course because as a C-derived language, Objective-C simply isn't as expressive as a language like Ruby, with its blocks, enumerations, literal syntax for creating arrays and hashes, etc.

Then there's the usual duplication also found in C/C++ with its header and implementation files. For example, in Ruby I can define a property simply by using attr_accessor :foo. In Objectice-C, I need to explicitly declare the actual member variable as well as the property in the header file, and add a synthesize statement to the implementation file. And this is already the shorthand syntax for properties...

Perhaps the thing I was most worried about: Memory Management. Objective-C 2.0 actually introduced garbage collection, but unfortunately this isn't supported on the iPhone. Luckily, as it turns out, Objective-C's retain count based memory management combined with autorelease pools is actually quite elegant and a major improvement over C/C++. I won't go into details here (there are many resources on this topic), but the bottom line is that as long as you adhere to the accepted conventions (particularly regarding which methods return autoreleased vs. non-autoreleased objects), you should be fine. With Instruments, Apple also provides a powerful tool to help track down actual memory leaks. Would I prefer garbage collection? Absolutely! But the situation isn't as bad as I had feared.

Objective-C also provides many nice features, such as Protocols and Categories. Protocols are very similar to Java Interfaces and are heavily used for delegation (see below). Categories allow functionality to be added to existing classes. For example, you could use this mechanism to allow NSDictionary, NSArray, or other NSObject subclasses to generate a JSON representation. They remind me of a less powerful (but still convenient) version of Ruby Mixins (less powerful because Mixins allow modules to be mixed into arbitrary classes, whereas Categories are implemented for a specific class).

CocoaTouch

This is really the best part of iPhone development. Having mostly worked on server side code, I've never been much of a GUI developer. But with a few positive exceptions (such as Qt), most of the GUI frameworks I have messed with were quite painful (MFC anyone?), with horrendous amounts of unmaintainable, auto-generated code.

CocoaTouch (like its big brother Cocoa on the Mac) is very well thought out and makes good use of design patterns. In particular, it heavily relies on delegation rather than inheritance, which leads to significantly cleaner and more modular code. For example, instead of subclassing a UI control in order to add the desired custom behavior, you typically implement the behavior in your controller class and specify this as the delegate for the UI control.

Aside from a few lines of boilerplate code that are part of the project templates, there is absolutely no auto-generated code. This is mostly possible because Interface Builder stores actual, serialized object instances inside xib files and all custom behavior is injected using the delegate mechanism.

CocoaTouch also provides very simple, high-level APIs for many powerful features, such as the iPhone's camera / image picker.

The only thing that bugs me a lot is the disconnect between the Objective-C based high-level Cocoa API and the C-based lower-level APIs. Often times, the high-level APIs are limited to a few common cases, but when you need to stray from these, you need to talk straight to the underlying low-level APIs. This leads to an awkward mix of nice, object-oriented Objective-C code and ugly, procedural C code.

Development Tools

The various development tools are an integral part of the iPhone SDK. In particular, Xcode is a very decent IDE. It may not be the best IDE I've ever used, but it gets the job done and even sports some modern IDE features (such as refactoring). I wish it had Git support, though.

Interface Builder is an extremely convenient tool, so I try to use it whenever it makes sense. However, I do have a few complaints: Many properties are not exposed in IB, so they have to be set manually in the code. More importantly, IB is not extensible at all. For example, I would like to be able to implement my own subclass of UIView and have its relevant properties show up in IB. If my UIView subclass defines a text property (perhaps with some sort of annotation that specifies additional, Interface Builder specific metadata), I would like to see a text field in IB. If I define a color property, I would like to see a color picker, etc. Hopefully this will be possible in a future version.

Instruments is a powerful tool for finding memory leaks and performance bottlenecks. I have only used this a little bit, but it's good to know that it's there when I need it.

My major complaint in this area is about the horribly complicated and error prone provisioning process. But I'll save this rant for another blog post.

Conclusion

Overall, iPhone development is a lot of fun, and a very refreshing change from web development. The iPhone is an amazingly powerful device, and after the initial learning curve, it is surprisingly easy to leverage many of its unique abilities, such as the accelerometer, multi-touch, Internet connection, and more.

I would love to be able to use CocoaTouch in conjunction with Ruby (such as RubyCocoa or MacRuby. But given that this is not an option on the iPhone, Objective-C is a decent alternative.

Now that I got this post out of the way, I plan on posting more about specific iPhone development topics. Stay tuned!

If you need to perform any time consuming work in your Rails actions, you'll probably want to offload this into a background job. There are many different frameworks to help with this, and the one we use is Workling. The nice thing about Workling is that it provides an abstraction layer that allows you to decouple your actual background job implementation from the background execution strategy. For example, in our development environment we are using the Spawn runner (which simply forks the Rails process for each background job), but we need a proper, queue based runner in production. Up until recently we were using the Starling runner, which worked pretty well for a small set of machines.

However, after migrating our infrastructure to Amazon EC2 and rapidly scaling up the number of app servers, we figured it would be great to take advantage of Amazon SQS (Simple Queue Service), rather than maintaining our own queue servers. Fortunately, Workling's plugin architecture makes it very easy to implement your own clients, so writing an SQS Workling client turned out to be fairly straightforward.

If you are interested in using this in your own project, simply use my Workling fork on Github. I haven't decided yet whether to extract this into a separate plugin that you could install alongside Workling, so let me know if you have a strong preference. I'll also get in touch with the Workling developers to see if they might be interested in pulling this feature into the main code base. But for now, you can simply install it by following the regular Workling plugin installation instructions, except using my Workling fork:

script/plugin install git://github.com/digitalhobbit/workling.git

The README includes detailed instructions on configuring the client, but it's actually very easy:

Install the RightAws gem:

sudo gem install right_aws

Configure Workling to use the SqsClient. Add this to your environment:

Workling::Remote.dispatcher = Workling::Remote::Runners::ClientRunner.new
Workling::Remote.dispatcher.client = Workling::Clients::SqsClient.new

Add your AWS key id and secret key to workling.yml:

production:
  sqs_options:
    aws_access_key_id: <your AWS access key id>
    aws_secret_access_key: <your AWS secret access key>

You can optionally override the following settings, although the defaults will likely be sufficient:

    # Queue names consist of an optional prefix, followed by the environment
    # and the name of the key.
    prefix: foo_

    # The number of SQS messages to retrieve at once. The maximum and default
    # value is 10.
    messages_per_req: 10

    # The SQS visibility timeout for retrieved messages. Defaults to 30 seconds.
    visibility_timeout: 30

    # The number of seconds to reserve for deleting a message from the qeueue.
    # If buffered messages are getting too close to the visibility timeout,
    # we drop them so they will get picked up the next time a worker retrieves
    # messages, in order to avoid duplicate processing.
    visibility_reserve: 10
        
    # Below are various retry and timeout settings for the underlying right_aws
    # and right_http_connection libraries. You may want to tweak these based on
    # your workling usage. I recommend fairly low values, as large values can
    # cause your Rails actions to hang in case of SQS issues.
        
    # Maximum number of seconds to retry high level SQS errors. right_aws
    # automatically retries using exponential back-off.
    aws_reiteration_time: 2
        
    # Low-level HTTP retry / timeout settings.
    http_retry_count: 2
    http_retry_delay: 1
    http_open_timeout: 2
    http_read_timeout: 10

Now start the Workling Client:

script/workling_client start

That's it!

There are still some caveats, such as the fact that messages are currently deleted from the queue at the beginning of processing rather than at the end (unfortunately Workling currently doesn't provide the necessary hooks). This is good enough for us (we're not relying on our background jobs for anything highly critical), but you probably don't want to build your financial transaction processing on top of this... If there's demand for this, I may try to extend Workling at some point to fix this issue.

Please leave a comment if you find this useful or have any other feedback. Also let me know if you encounter any bugs, or better yet, update my test case to reproduce the issue or send me a Github pull request with your fix. :)

We're using Facebooker for our Rails based Facebook apps. However, we ran into a problem after migrating our session store to the MemCacheStore. Every request was producing the following stacktrace:

/!\ FAILSAFE /!\  Sat Feb 28 10:24:09 -0800 2009
  Status: 500 Internal Server Error
  session_id '2.wYavYw2U9jBTnFOcX9rjMw__.86400.1235934000-1023424742' is invalid
    /Library/Ruby/Gems/1.8/gems/actionpack-2.2.2/lib/action_controller/session/mem_cache_store.rb:54:in `initialize'
    /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/cgi/session.rb:273:in `new'
    /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/cgi/session.rb:273:in `initialize_without_cgi_reader'
    /Library/Ruby/Gems/1.8/gems/actionpack-2.2.2/lib/action_controller/cgi_ext/session.rb:19:in `initialize_aliased_by_facebooker'
    /Users/mirko/Work/questionx/vendor/plugins/facebooker/lib/facebooker/rails/facebook_session_handling.rb:35:in `initialize'
    /Library/Ruby/Gems/1.8/gems/actionpack-2.2.2/lib/action_controller/cgi_process.rb:94:in `new'

It turns out that Facebook session ids include dots and underscores, which the MemCacheStore chokes on. Luckily I came across this forum post. The solution below is based on the approach outlined in the post, with a few modifications to more cleanly hook the patch into the method chain rather than replacing the original functionality completely. Simply drop the code below into an initializer (e.g. config/initializers/facebooker_memcache_session_patch.rb):

class CGI
  class Session
     class MemCacheStore
       def check_id_with_strip_fb_chars(id)
         check_id_without_strip_fb_chars(id.gsub(/[-\._]/, ''))
       end
       alias_method_chain :check_id, :strip_fb_chars
     end
  end
end