Posts tagged full text search

Hi Guys, it has been a long time since I last posted. I had worked on several things since then, and have couple of posts pending/draft. One of those posts is related to Ultrasphinx, when it is used with STI models.

For those who are new to Ultrasphinx: Ultrasphinx is a rails plugin and client to the Sphinx(full text search engine) written by Evan Weaver. More about Ultrasphinx here.

Lets get into the situation. Consider a STI case where we are using ultrasphinx to index several fields:

class Post < ActiveRecord::Base

end

class Article < Post
  is_indexed :fields => [:title, :body]
end

class Story < Post
  is_indexed :fields => [:title, :body]
end

Now assume we have following data in posts table:

and now in application root:

rake ultrasphinx:configure
rake ultrasphinx:index
rake ultrasphinx:daemon:start

By this point we have created sphinx configuration file, indexed all records from ultrasphinx models and started sphinx search daemon.

Now open script/console and create and fire a query to find some articles:

>> search = Ultrasphinx::Search.new(:query => "second article", :class_names => 'Article')
=> #<ultrasphinx::Search:0x2105bec @subtotals={}, @response={}, @facets={}, @options={"indexes"=>"main", "class_names"=>["Article"], "sort_by"=>nil, "parsed_query"=>"second article", "sort_mode"=>"relevance", "weights"=>{}, "filters"=>{}, "per_page"=>20, "query"=>"second article", "page"=>1, "facets"=>[], "location"=>{"units"=>"radians", "long_attribute_name"=>"lng", "lat_attribute_name"=>"lat"}}, @results=[]>
>> search.run
ActiveRecord::RecordNotFound: Couldn't find Article with ID=5
	from /Users/akhilbansal/work/sti/vendor/plugins/ultrasphinx/lib/ultrasphinx/search/internals.rb:308:in `reify_results'
	from /Users/akhilbansal/work/sti/vendor/plugins/ultrasphinx/lib/ultrasphinx/search/internals.rb:286:in `each'
	from /Users/akhilbansal/work/sti/vendor/plugins/ultrasphinx/lib/ultrasphinx/search/internals.rb:286:in `reify_results'
	from /Users/akhilbansal/work/sti/vendor/plugins/ultrasphinx/lib/ultrasphinx/search.rb:357:in `run'
	from /Users/akhilbansal/work/sti/vendor/plugins/ultrasphinx/lib/ultrasphinx/search/internals.rb:352:in `perform_action_with_retries'
	from /Users/akhilbansal/work/sti/vendor/plugins/ultrasphinx/lib/ultrasphinx/search.rb:337:in `run'
	from (irb):3

Here when you run the query you get an exception, because it is trying to find an article with id 5, which actually a story type not article. So why it is trying to find such article?

Now have a look into config/ultrasphinx/development.conf file. Under section “source articles_main” you’ll get “SELECT (posts.id * 3 + 0) AS id, ” AS article_title, posts.body AS body, ‘Article’ AS class, 0 AS class_id, posts.title AS title FROM posts WHERE posts.id >= $start AND posts.id <= $end GROUP BY posts.id" Which is a SQL to get record to index.

If you check it carefully you'll findout that it should select all articles record to index but unfortunately it is selecting all records from table and considering them as articles. To fix it you need to modify this query and add "posts.type = 'Article'" in where condition. So the query should be "SELECT (posts.id * 3 + 0) AS id, '' AS article_title, posts.body AS body, 'Article' AS class, 0 AS class_id, posts.title AS title FROM posts WHERE (posts.type = 'Article') and posts.id >= $start AND posts.id <= $end GROUP BY posts.id"

But still there is a problem. If you do this manually you have to do it again and again whenever you issue "rake ultrasphinx:configure" because this configuration file will be overwritten.

Better option is to add conditions in models as is_indexed options like:

class Article < Post
  is_indexed :fields => [:title, :body], :conditions => "posts.type = ‘Article’"
end

and it worked fine.

Manik gave me an idea to write a patch for ultrasphinx to add such conditions automatically in case of STI. So may be another post related to ultrasphinx will be soon

Update: Here is the patch Git patch: Fix STI Issue. After applying this patch you need not to add conditions explicitly for such case. It will automatically check and add conditions for STI.