Troubleshooting Sitecore Lucene search and indexing

11 June 2013
Marek Musielak
Frink_Cognifide_2016_HeaderImages_0117

sitecore lucene search and indexing

While looking at Sitecore questions on stackoverflow.com I was quite surprised by the number of issues related to introducing Lucene index and search to Sitecore applications. Everyone knows about Sitecore Search and Indexing pdf on the Sitecore SDN site and about Sitecore Advanced Database Crawler, but it clearly doesn't contain the answers for some of the common questions. That's why I've written this blog post which I hope to be helpful for any developer having difficulties with Sitecore and Lucene working together.

Case 1: New index configured but it doesn't work.

First lets check if the index was created. In order to do this, explore to the location of your Sitecore dataFolder and open the indexes folder. There should be a directory with a name same as the value of the <param desc="folder"> property in your index configuration and with the index files inside of it. If the directory is not there then the problem is with the configuration of the index. In this case just check the http://localhost/sitecore/admin/showconfig.aspx if the configuration of you index is there and if looks like described in the doc from the Sitecore SDN.

If the index files are created, the next step of determining why the index doesn't work is to ensure that it has been rebuilt. You can do this:
  • from the Sitecore Desktop using Control Panel application and then Database and Rebuild the Search Index options,
  • Or from the code by executing the method:
    Sitecore.Search.SearchManager.GetIndex("your_index_name").Rebuild()
Now we are sure that the index is created and rebuilt so we can check what's indexed. We have 3 easy options of getting the information from the Sitecore Lucene indexes:
  • You can use Sitecore Lucene index viewer module but this require installing Sitecore package to your application and you not necessarily want to do this,
  • Use the Luke - Lucene Index Toolbox which is a standalone java jar application for browsing any Lucene index (just remember to download the version of Luke which supports the same version of Lucene as your Sitecore app),
  • Or you can use
    SearchManager.GetIndex("your_index_name").CreateSearchContext()
        .Search(new MatchAllDocsQuery(), int.MaxValue)
        .FetchResults(0, int.MaxValue).Select(r => r.GetObject<Item>())
    
    and iterate over the returned values to check all the indexed items.

Case 2: Specify the templates which should be indexed.

This is one of the most common problems - we want to index only the Sitecore items which are defined by chosen set of templates. Developers often create the first row of the included templates configuration, copy it and change the id of the templates getting:
<include hint="list:IncludeTemplate">
    <template>{71D42CF2-CE89-4030-9EB1-0065B35B78C4}</template>
    <template>{ED9F466B-D436-4A3F-B22F-EA6E8097085D}</template>
    <template>{78166FE4-EDFB-4B0D-A3ED-860AEB44CD40}</template>
</include>
which is incorrect!. You need to have different names for each of the template nodes. In other case only the last of them will be included. Proper configuration for this looks like this:
<include hint="list:IncludeTemplate">
    <template1>{71D42CF2-CE89-4030-9EB1-0065B35B78C4}</template1>
    <template2>{ED9F466B-D436-4A3F-B22F-EA6E8097085D}</template2>
    <any-other-name>{78166FE4-EDFB-4B0D-A3ED-860AEB44CD40}</any-other-name>
</include>
And one more important thing - you have to specify the exact template not the base one - items using inherited template will not be indexed.

Case 3: Searching Sitecore items by date range.

Sitecore Date and Datetime fields are stored as yyyyMMddTHHmmss format string, e.g. 20130418T140122. Assuming that we want to search all the items created on particular day, we can use either WildcardQuery with asterisk like this: 20130611* or a TermRangeQuery starting from 20130611T000000 including this value and ending on 20130612T000000 excluding this value:
new WildcardQuery(new Term("date-field-name-lowercase", "20130611*"));

new TermRangeQuery("date-field-name-lowercase", "20130611T000000",
    "20130612T000000", true, false);

Case 4: My fields are indexed but my search doesn't work.

The most common reason why search fails despite the fact that everything seems to be indexed well are uppercase characters in the Lucene field names in queries. By default Sitecore.Search.Crawlers.DatabaseCrawler uses lowercase names for document fields names so remember to use __created instead of __Created, etc. If you still have any doubts, use Luke to see what exactly is indexed.

Sitecore
and Lucene works together really well, just sometimes they need a little bit of patience at the beginning. I hope this post helped you with setting them right. If you found this post interesting, do checkout other Sitecore blog posts on www.cognifide.com.

This information is based on Sitecore 6.6 update 4 and may not work properly with other Sitecore versions.