Indexing rich documents with Rails, Sunspot, Solr, Sunspot Cell and Carrierwave (cookbook-style)

Solr / Sunspot installation and configuration is easy when you just need to index and search your model data. I won’t go into details about configuring basic Sunspot / Solr for Rails here. For a great primer on Sunspot and basic installation instructions, I recommend Ryan Bates’ Railscast.

But configuring Solr to index rich documents (e.g. PDFs, Word documents) via Sunspot Cell is really quite badly documented, and it doesn’t need to be. Once you configure things correctly, it really does work. Learn from my pain…

Read the rest of this story…

Beautiful Logging for Ruby on Rails 3.2

In my previous post I showed you a simple way to get beautiful, easy-to-read logs in your Rails application. That trick stopped working in Rails 3.2. So for Rails 3.1 or earlier, refer to my earlier post; but for 3.2 on, read on…

It’s really easy. Just make a new file in your ‘config/initializers’ directory called something like ‘log_formatting.rb’ and paste into it the following code. Restart your app, and voila: pretty logs again!

class ActiveSupport::BufferedLogger
  def formatter=(formatter)
    @log.formatter = formatter
  end
end

class Formatter
  SEVERITY_TO_TAG_MAP     = {'DEBUG'=>'meh', 'INFO'=>'fyi', 'WARN'=>'hmm', 'ERROR'=>'wtf', 'FATAL'=>'omg', 'UNKNOWN'=>'???'}
  SEVERITY_TO_COLOR_MAP   = {'DEBUG'=>'0;37', 'INFO'=>'32', 'WARN'=>'33', 'ERROR'=>'31', 'FATAL'=>'31', 'UNKNOWN'=>'37'}
  USE_HUMOROUS_SEVERITIES = true

  def call(severity, time, progname, msg)
    if USE_HUMOROUS_SEVERITIES
      formatted_severity = sprintf("%-3s","#{SEVERITY_TO_TAG_MAP[severity]}")
    else
      formatted_severity = sprintf("%-5s","#{severity}")
    end

    formatted_time = time.strftime("%Y-%m-%d %H:%M:%S.") << time.usec.to_s[0..2].rjust(3)
    color = SEVERITY_TO_COLOR_MAP[severity]

    "\033[0;37m#{formatted_time}\033[0m [\033[#{color}m#{formatted_severity}\033[0m] #{msg.strip} (pid:#{$$})\n"
  end

end

Rails.logger.formatter = Formatter.new

Credit for figuring out how to get log formatting working with Rails 3.2 goes to JRochkind. If you’d like a complete log formatting solution in the form of a gem, see his FormattedRailsLogger gem at Github.

Humorously informative Rails logging

I’ll just leave this here:

module ActiveSupport
  # Format the buffered logger with timestamp/severity info.
  class BufferedLogger
    NUMBER_TO_NAME_MAP  = {0=>'meh', 1=>'fyi', 2=>'hmm', 3=>'wtf', 4=>'omg', 5=>'???'}
    NUMBER_TO_COLOR_MAP = {0=>'1;30',  1=>'0;36', 2=>'0;33', 3=>'1;33',  4=>'1;31',  5=>'0;37'}

    def add(severity, message = nil, progname = nil, &block)
      return if @level > severity
      sevstring = NUMBER_TO_NAME_MAP[severity]
      color     = NUMBER_TO_COLOR_MAP[severity]

      message = (message || (block && block.call) || progname).to_s
      message = "\033[0;37m#{Time.now.to_s(:db)}\033[0m [\033[#{color}m" + sprintf("%-3s","#{sevstring}") + "\033[0m] #{message.strip} (pid:#{$$})\n" unless message[-1] == ?\n
      buffer << message
      auto_flush
      message
    end
  end
end

(To use, put this code in an initializer in config/initializers and restart. Provides highly-readable, timestamped, colorized logging for your rails app.)

Twitter, OAuth, and Ruby on Rails integrated cookbook-style in the console (updated for Twitter 1.0)

Update March 17, 2011: This is a rewrite of my October post of the same name, modified for the changes that were made in the Twitter gem from 1.0 onward. John Nunemaker removed native oauth capability from the gem at 1.0, necessitating a rewrite of my instructions below.

As of August 31, 2010, Twitter officially deprecated HTTP Auth from the Twitter API in favor of OAuth. OAuth has a number of benefits as far as user authorization is concerned, but it is a tad complicated for the developer, especially if you simply want your application to interact with Twitter. So even if you aren’t concerned at all with authorizing your application’s users — for example, if you want your application to post tweet updates to its own account — OAuth is your only choice at this point. Here is a quick and dirty, cookbook-style recipe to get your Ruby on Rails application to interact with Twitter.

Read the rest of this story…

An init script for managing delayed_job

If you run delayed_job as a daemon on a server, you’d probably like a way to start it when the system boots. Furthermore, you’re probably using RVM at this stage, which requires its own finicky management details within a script. This little init script will handle all those details for you and make starting and stopping the delayed_job daemon a straightforward process.

You’ll want to modify paths appropriate for your system. Also, although this init script is tuned for Gentoo, it can easily be adapted for Red Hat and other Linux variants.

Read the rest of this story…

Beautiful logging for your Ruby on Rails application

Update April 5, 2012: this method only works with Rails 3.1 and older. For a way that works with Rails 3.2, see my new post.

I like pretty logs. I like logs with timestamped entries. The default Ruby on Rails BufferedLogger is so-so on the former and neglects the latter. This very simple patch fixes that.

This monkey-patch is Rails 2 and 3 compatible. Like all monkey-patches, it simply overrides predefined behavior with your own variant. In this case we add timestamp info as well as color-coded severity info; the latter makes it very easy to spot urgent items such as errors and fatalities. Here are some examples:

Read the rest of this story…

Beautiful standardized RDoc comments for your Ruby / Rails methods

When I started working in Ruby on Rails five years ago, one thing that struck me was that there seemed to be no standard style for method documentation. Javadoc documentation, by contrast, is fairly uniform in the Java world.

I share with you today the template that I employ for documenting Ruby methods. I like this template because it, like Javadoc, creates a somewhat standard, easily readable comment structure.

  #
  #
  # * *Args*    :
  #   - ++ ->
  # * *Returns* :
  #   -
  # * *Raises* :
  #   - ++ ->
  #
  def method_name

  end

Read the rest of this story…

Saving attachments with Ruby 1.9.2, Rails 3 and the Mail gem

Using the TMail gem in Rails 2 / Ruby 1.8.7, this was probably how you saved an attachment:

# tmail is a TMail object
tmail.attachments.each do |tattch|
  fn = tattch.original_filename
  File.open(fn, File::CREAT|File::TRUNC|File::WRONLY,0644) { |f| f.write( tattch.string ) }
end

…or perhaps something like this if you’re cautious:

# tmail is a TMail object
tmail.attachments.each do |tattch|
  fn = tattch.original_filename
  begin
    File.open(fn, File::CREAT|File::TRUNC|File::WRONLY,0644) { |f| f.write( tattch.string ) }
  rescue Exception => e
    logger.error "Unable to save data for #{fn} because #{e.message}"
  end
end

But if you try that in Rails 3 / Ruby 1.9.2 — which uses the new Mail gem — you’ll get a curious little error like this one (where x80 could be many different hex codes, such as xC5 or xC3):

Encoding::UndefinedConversionError Exception: "\x80" from ASCII-8BIT to UTF-8

Your first instinct might be to suspect the Mail gem itself. The problem is, in fact, not Mail’s fault; it is rooted in Ruby 1.9.2′s new string encoding mechanism. Fortunately the fix is dead simple: open the output file in binary mode.

Here is the above code snippet corrected for Rails 3 / Ruby 1.9.2 with the new Mail gem:

# tmail is now a Mail object
tmail.attachments.each do |tattch|
  fn = tattch.filename
  begin
    File.open( fn, "w+b", 0644 ) { |f| f.write tattch.body.decoded }
  rescue Exception => e
    logger.error "Unable to save data for #{fn} because #{e.message}"
  end
end

For more information about how things changed in Ruby 1.9, consider reading James Gray’s article on Ruby 1.9 string encodings.

Twitter, OAuth, and Ruby on Rails integrated cookbook-style in the console (for Twitter 0.9)

Update March 17, 2011: This post has been superseded by a newer one by the same name. The instructions in this post are for the Twitter gem 0.9.  John Nunemaker removed native oauth capability from the gem at 1.0, so if you are using 1.0 or later, visit my newer post for corrected instructions.

As of August 31, 2010, Twitter officially deprecated HTTP Auth from the Twitter API in favor of OAuth. OAuth has a number of benefits as far as user authorization is concerned, but it is a tad complicated for the developer, especially if you simply want your application to interact with Twitter. So even if you aren’t concerned at all with authorizing your application’s users — for example, if you want your application to post tweet updates to its own account — OAuth is your only choice at this point. Here is a quick and dirty, cookbook-style recipe to get your Ruby on Rails application to interact with Twitter.

If you haven’t already, create the Twitter account that your Rails app will use. Verify the account, et cetera, and sign in to the Twitter web interface.

Navigate to http://dev.twitter.com/ and click “Register an app.” Complete the form, choosing “client” as the application type, and “Read & Write” as the access type. After saving your application, view the app’s settings screen and note the “consumer key” (also known as the consumer token) and “consumer secret” data fields — you’ll use those two items in a moment.
Click here to read the rest of this article…