Solr Replication With Sitecore

In this blog post i will explain what is the solr replication and how to configure Solr Master-Slave replication with Sitecore with minimum number of servers.

First of all  I want to explain when we should use Solr instead of lucene.

When Should We Use Solr Instead Of Lucene ?

As we know Sitecore support both Lucene and Solr engine, and the main LINQ query will remain the same, the differences only in the configurations, In most situations, we have a choice between using Solr or Lucene, based on Sitecore recommendation,  there are some reasons to use Solr instead of Lucene :

  1. When you need to index large numbers of items (50,000 and up), Solr performs better.
  2. Solr is more robust. If your site depends on search as the primary interface, consider using Solr.
  3. If you use multiple content delivery servers (or plan to do so later), use Solr. Solr works automatically in such an environment. You could use Lucene, but you have to make sure that indexes are synchronized across servers yourself.

So we should therefore use Solr if you plan to scale our site (have a distributed setup with multiple servers).

More details on Sitecore documentation, checkout this link.

What is Solr Replication :

Solr Replication distributes complete copies of a master index to one or more slave servers. The master server continues to manage updates to the index. All querying is handled by the slaves. This division of labor enables Solr to scale to provide adequate responsiveness to queries against large search volumes.

In simple words, when the master Solr index is updated by the CM server, the slave Solr indexes will be updated to reflect those changes

Advantages :

  1. Splits read and write load and operations.
  2. Load distribution for search queries.
  3. High availability for searching.
  4. Any number of slave instances can be created to scale query performance.
  5. Usually less frequent index updates on the slaves and better use of the cache

Solr Master-Slave Architecture 

Solr Consumes CPU resources specially if you have alot of items or the Wesbsite is purely driven by search, so it is better to have dedicated server for SOLR. but sometimes client has limited budget, he doesn’t want to add dedicated server for solr, so let’s assume that we have only one CM and two CDs as below :

daigram1

in this situation we will have solr installed on CM server, we will configure it as solr-master and solr installed on each CD server we will configure it as solr-slave, so our architecture will be like this :

diagram2

Important Note: If you are using Bitnami to install solr, make sure SOLR is
running on JAVA 64-bit runtime, By default, Bitnami SOLR will install 32-bit
java version. for more details about how to configure it to work on java 64-bit
checkout this blog post, Thanks Ahmed for the great post.

Solr Master-Slave Configurations :

No changes on solr-master on cm server, you just need to configure solr-slave on each cd like this :

  1. Open solrconfig.xml where your solr core installed (ex:C:\Bitnami\solr-4.10.3-0\apache-solr\solr\yourcore\conf\solrconfig.xml)
  2. Search for <requestHandler name=”/replication” class=”solr.ReplicationHandler”>
  3. inside this section uncomment the following section and provide the master url from cm server :

<lst name=”slave”>
<str name=”masterUrl”>http://your-master-hostname:8983/solr/yourcore/replication</str&gt;
<str name=”pollInterval”>00:00:60</str>
</lst>

The slave continuously keeps polling the master (depending on the pollInterval parameter) to check the current index version of the master. by default it is every 60 seconds and you can change it.

after that when you open solr dashboard from master and slave it should look like this :

Master :

master

Slave:

slave

from Sitecore open Sitecore.ContentSearch.Solr.DefaultIndexConfiguration.config and make sure that ContentSearch.Solr.ServiceBaseAddress  value is the local solr instance on each  server.

Conclusion

I was talking in this post about solr replication, also there is another approach called solrcloud, and how to use zookeeper to make sure replication is done right, this blog post explains how you can configure Sitecore with solrcloud.

I hope you find this post helpful, If you have any additional knowledge on this subject, comments or questions, please let me know in the comments section below.

Advertisements

12 thoughts on “Solr Replication With Sitecore

  1. Nice post!

    Just would like to add that when you have multiple analytics_index you would also require SOLR as a centralized search index.

    So not only multiple CD servers, so it could be 1CM, 1Processing, and 1CD server – would require SOLR

    Liked by 1 person

  2. I have 2 CD Servers but there i am unable to find slave(searching) instead in master(searching) and master(replicable) getting same Version,Gen,Size .Please do you suggest me what i am doing wrong

    Liked by 1 person

    1. In this case you need to check if solr master on CM, is accessible from your CDs server, are you able to open solr master link from your CDs servers ?

      Like

  3. Ayman, awesome post. I have a question. We have a Sitecore implementation that uses Coveo as it’s search provider for the content of the website. However, we do use SOLR for all the other indexes required by Sitecore, including our analytic index.
    We are having issues with it now, everything was working fine before but as is getting larger we are running into issues. The current size is about 11 GB. It is mostly failing when adding/tagging contacts to lists through the ListManager API. It looks like Sitecore is doing a lot of commits and SOLR cannot keep out with it.
    What do you recommend for the solrconfig.xml settings for this index? I have reached out to Sitecore but I have not had much luck.

    Thank you in advance.
    -Camilo Varon
    Business Systems Manager

    Like

    1. Hi Camilo,
      Do you have only one index ? usually in such case, when you have huge data, you should create custom indexes, and if you have multi-sites, I prefer to have Index for each site, it I wrote about that, checkout this post, After that you can do apply SOLR replication.

      Thanks!

      Like

  4. Hi Ayman,

    Nice article.

    I am working with Sitecore 8.2 with XDB and using Solr as a search provider. I have setup Master and Slave instances and configured Cores on both the instances.

    Now I have couple of questions: 1) Which Cores need to be configured on Solr Master instance and which cores need to be configured on Slave. 2) while setting up replication, which Cores need to replicated from Master to Slave.

    Thanks in advance.

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s