The Skinny on Scopes (Formerly named_scope)

Rails 3.0

The source for the examples contained in this article are located at: http://github.com/rwdaigle/edgerails-support/tree/master/the-skinny-on-scopes-formerly-named-scope/

I remember my heart fluttering with a boyish crush the first time I saw Nick Kallen’s has_finder functionality make it into Rails in the form of named_scope. named_scope quickly made its way into my toolset as a great way to encapsulate reusable, chainable bits of query logic. While it had its downsides (namely its lack of first-class chain support for the likes of :joins and :include) it redefined how I thought about structuring my model logic. Once you taste the chainable goodness of named_scope you never go back.

So here we are with Rails 3 completely refactoring the internals of ActiveRecord - what’s up with our beloved named_scope? Well, the simple answer is that it’s been renamed to scope and you can use it just as you’re used to … but that’s taking the easy way out. Let’s see what else we can do with scope in Rails 3.

Basic Usage

Let’s assume a standard Post model with published_at datetime field along with title and body (to follow along in code, see the accompanying project in github).

In Rails 2.x here’s how we’d have to define the self-explanatory published and recent named scopes:

class Post < ActiveRecord::Base

  named_scope :published, lambda { 
    { :conditions =>
      ["posts.published_at IS NOT NULL AND posts.published_at <= ?", Time.zone.now]
    }
  }  
  named_scope :recent, :order => "posts.published_at DESC"
end

The reason we need to use a lambda here is that it delays the evaluation of the Time.zone.now argument to when the scope is actually invoked. Without the lambda the time that would be used in the query logic would be the time that the class was first evaluated, not the scope itself.

With Rails 3 the bulk of ActiveRecord is now based on the Relation class. Think of relation as named_scope on steroids, weaving chainable query logic into the very fabric of ActiveRecord.

You can see how to use the individual where, order etc… commands of Relation on Pratik’s great writeup of the new query interface as well in this Railscast. Understanding these are important as the new scope is built upon them.

Let’s see how - here’s how the two named scopes from our previous Post example will look in Rails 3:

class Post < ActiveRecord::Base

  scope :published, lambda { 
    where("posts.published_at IS NOT NULL AND posts.published_at <= ?", Time.zone.now)
  }  
  scope :recent, order("posts.published_at DESC")
end

While the bulk of the logic is the same - the SQL string portions - you start to see how scopes use the new query interface directly to create reusable query logic versus constructing an options hash as was done in Rails 2. This really is our first glimpse of how much more flexible the new query interface allows our scopes to be. No longer are they a slightly different construct than your normal query methods. They are now built upon the very same query methods that you would use were you to execute a query directly. This consistency is now prevalent all throughout ActiveRecord.

And there’s more…

Scope Reusability

Suppose we want to update our recent scope to only include published posts. We’ve already defined what published means and shouldn’t have to redefine it to create a new scope. Well, you can also chain scopes within scope definitions themselves as we’ll do here with the new recent and published_since scopes.

class Post < ActiveRecord::Base
  
  scope :published, lambda { 
    where("posts.published_at IS NOT NULL AND posts.published_at <= ?", Time.zone.now)
  }
  scope :published_since, lambda { |ago|
    published.where("posts.published_at >= ?", ago)
  }
  
  scope :recent, published.order("posts.published_at DESC")
end

Ok, now we’re getting warmed up.

Dynamic Scope Construction

I’ve been in love with the scoped the anonymous named_scope constructor in Rails 2.3 for sometime now, using it to create dynamic and chainable scopes on an as-needed basis. One use-case you see a lot for this type of functionality is for creating a search method that you can still append other query manipulations onto.

For instance, to search our posts we can create this method which will return a scope for your caller to further filter (notice the use of scoped to start the chain off with an innocuous scope upon which others can be appended):

class Post < ActiveRecord::Base
  
  class << self
    
    # Search the title and body fields for the given string.
    # Start with an empty scope and build on it for each attr.
    # (Allows for easy extraction of searchable fields definition
    # in the future)
    def search(q)
      [:title, :body].inject(scoped) do |combined_scope, attr|
        combined_scope.where("posts.#{attr} LIKE ?", "%#{q}%")
      end
    end
  end
end

The use of inject here somewhat obfuscates the intent of the method if you’re not used to looking at such iterations - here’s an easier to follow version with the searchable fields more hard coded (which actually doesn’t even use an anonymous scope to get bootstrapped):

class Post < ActiveRecord::Base
  
  class << self
    
    # The less-slick but, perhaps, more obvious version
    def search(q)
      query = "%#{q}%"
      where("posts.title LIKE ?", query).where("posts.body LIKE ?", query)
    end
  end
end

Since we’re building upon the chainable goodness of then new query interface (think of scopes now as named bundles of the new ActiveRelation construct), we can do the following with the search method:

# What's in the db, titles ~= publish date
Post.all.collect(&:title) #=> ["1 week from now", "Now", "1 week ago", "2 weeks ago"]
Post.published.collect(&:title) #=> ["Now", "1 week ago", "2 weeks ago"]

# Search combinations
Post.search('1').collect(&:title) #=> ["1 week from now", "1 week ago"]
Post.search('1').published.collect(&:title) #=> ["1 week ago"]
Post.search('w').published_since(10.days.ago).collect(&:title) #=> ["Now", "1 week ago"]
Post.search('w').order('created_at DESC').limit(2).collect(&:title) #=> ["2 weeks ago", "1 week ago"]

You can imagine a scenario where more complex query-string support could be built, all using anonymous scopes.

Feels great, huh? It also feels a lot like the utility_scopes gem I released awhile ago which was my attempt to package up the chainable goodness of named_scope for common query operations. Rest-assured, there’s a much smoother implementation under the covers here in Rails 3 than just some hacks on top of named_scope

Cross-Model Scopes

Scopes are great for operating solely on the columns of a singular class’s table, but they can also be used to package cross-model queries (i.e. any SQL that would require a join). Let’s add in users (who can author and comment on posts) to the mix and write some scopes on User that will fetch only users that have authored published posts as well as users that have commented on a post:

class User < ActiveRecord::Base

  has_many :posts, :foreign_key => :author_id
  has_many :comments
  
  # Get all users that have published a post
  scope :published, lambda {
    joins("join posts on posts.author_id = users.id").
    where("posts.published_at IS NOT NULL AND posts.published_at <= ?", Time.zone.now).
    group("users.id")
  }
  
  # Get all users that have commented on a post
  scope :commented, joins("join comments on comments.user_id = users.id").group("users.id")
end

Also, as Steffen pointed out in the comments, ActiveRelation is smart enough to know how to do a join based on an association definition, allowing us to collapse the joins relations from SQL strings to an association reference:

class User < ActiveRecord::Base
  
  # Get all users that have published a post
  scope :published, lambda {
    joins(:posts).              # No need to write your own SQL
    where("posts.published_at IS NOT NULL AND posts.published_at <= ?", Time.zone.now).
    group("users.id")
  }
  
  # Get all users that have commented on a post
  scope :commented, joins(:comments).group("users.id") # Just reference :comments
end

Since we’ve got the full arsenal of ActiveRelation operators at our disposal in scopes, we can do joins and group bys within scopes that will be safely chained in complex queries - something where the old named_scope crapped the bed:

# Get all users that have a post published
User.published.collect(&:username) #=> ["tim", "dave"]
User.published.to_sql
  #=> SELECT "users".* FROM "users" join posts on posts.author_id = users.id
  #   WHERE (posts.published_at IS NOT NULL AND posts.published_at <= '2010-02-22 21:33:00.892308')
  #   GROUP BY users.id
  
# Get all users that have commented on a post
User.commented.collect(&:username) #=> ["ryan", "john", "tim", "dave"]
User.commented.to_sql
  #=> SELECT "users".* FROM "users" join comments on comments.user_id = users.id
  #   GROUP BY users.id
  
# Combine them to get all authors that have also commented
User.published.commented.collect(&:username) #=> ["tim", "dave"]
User.published.commented.to_sql
  #=> SELECT "users".* FROM "users"
  #   join posts on posts.author_id = users.id
  #   join comments on comments.user_id = users.id
  #   WHERE (posts.published_at IS NOT NULL AND posts.published_at <= '2010-02-22 21:33:00.892308')
  #   GROUP BY users.id

As I’ve done here, use scope#to_sql to peek at what SQL the scope will execute. Very useful for debugging purposes.

Scope-based Model CRUD

Since ActiveRelation lets you invoke all the builder/update/destroy methods on a relation that you’re used to using directly against your models, that power is also available at the end of a scope/scope-chain. Let’s play around with our post scopes and use them to do more than just query:

# Increment the views_count for all published posts
Post.published.collect(&:views_count) #=> [59, 71, 42]
Post.published.update_all("views_count = views_count + 1")
Post.published.collect(&:views_count) #=> [60, 72, 43]

# Nobody cares about unpublished posts
Post.unpublished.size #=> 1
Post.unpublished.destroy_all
Post.unpublished.size #=> 0

You can also create a new model from existing scopes - suppose we have a (very contrived) scoped that gets only posts of certain title:

class Post < ActiveRecord::Base
  
  # Ludacris
  scope :titled_luda, where(:title => 'Luda')
end

We can use this scope to directly build instances (as well as create, new, create! etc…):

Post.titled_luda.size #=> 0
Post.titled_luda.build
  #=> #<Post id: nil, title: "Luda", ...>

Scopes really can be thought of now as named packages of both query and construction logic. Very powerful.

Crazy Town

One thing that’s always bugged me is how the logic for what makes a Post published is split between scopes in both the Post class and the User class. To refresh our collective memories:

class Post < ActiveRecord::Base
  
  scope :published, lambda { 
    where("posts.published_at IS NOT NULL AND posts.published_at <= ?", Time.zone.now)
  }
end

And:

class User < ActiveRecord::Base
  
  scope :published, lambda {
    joins(:posts).
    where("posts.published_at IS NOT NULL AND posts.published_at <= ?", Time.zone.now).
    group("users.id")
  }
end

Most good developers will immediately cringe at the duplication of the where("posts.published_at IS NOT NULL AND posts.published_at <= ?", Time.zone.now) relation.

Thanks to a tip by Railscast’s Ryan Bates, there’s a pretty slick way to refer to, and combine, scope logic: the merge method, aliased as ‘&’. Let’s look at how we can use scope#& to refer to the query logic of the Post.published scope from within our User.published scope:

class User < ActiveRecord::Base
  
  scope :published, lambda {
    joins(:posts).group("users.id") & Post.published
  }
end

Just so we’re clear what happens when you merge relations/scopes with the & operator, let’s look at the resulting SQL:

User.published.to_sql
  #=> SELECT users.* FROM "users"
  #   INNER JOIN "posts" ON "posts"."author_id" = "users"."id"
  #   WHERE (posts.published_at IS NOT NULL AND posts.published_at <= '2010-02-27 02:55:45.063181')
  #   GROUP BY users.id

Notice how the conditions defined within Post.published are merged into the joins and group relations of the User.published scope? Very nice. And merging works with all the mergeable relations, not just where conditions we used here.

Summary

This post somewhat glosses over the new query interface for ActiveRecord in Rails 3 to get to the meat of using scopes. However, none of the scoped yumminess could have happened without the slick new underpinnings of ActiveRecord. So, if you’re still a little confused about all this, definitely read some more about ActiveRecord before jumping into scopes. Once you do have that foundation, however, you will use scopes on a very regular basis.