24 Jun

Install Apache Solr on Ubuntu with MySQL Data Import

Posted in by at 06:06 Comments Off on Install Apache Solr on Ubuntu with MySQL Data Import

Apache Solr is an open source enterprise Search platform built in Java. Solr is written in Java and runs as a standalone full-text search server within a Servlet container such as Tomcat. Solr uses the Lucene Java search library at its core for full-text indexing and searching. This article lists all the steps that you need to perform to install the Solr with Tomcat.

Note that we have used Tomcat but you can use any other Servlet container (Jetty?) as well.

Install JDK in your system. ( You can skip this step if already have JDK in your system)

Follow the commands below to install Sun JDK in your system. You can always use Open JDK from Ubuntu software center of you want but it is preferred to use Sun JDK.
for add-apt-repository command to work
sudo apt-get install python-software-properties
it stores repository in your sources.list file
sudo add-apt-repository ppa:webupd8team/java
for updating your software to latest versions
sudo apt-get update
for installing JDK into your system.
sudo apt-get install oracle-java7-installer
After installation you can check if it was successful using command java -version

Install Tomcat.

Use following commands for installing tomcat into your system.
sudo apt-get install tomcat6 tomcat6-admin
It would install tomcat into your system.
To check if it is installed in your system go to
http://<IP_ADDRESS>:8080
OR
http://localhost:8080
It would show It works!….
If not you probably need to set java home in tomcat goto /etc/default/tomcat6
using
vim /etc/default/tomcat6
and add this line that file :
JAVA_HOME=/usr/lib/jvm/java-7-oracle/

and restart tomcat server using: /etc/init.d/tomcat6 restart and check again if it is installed or not.

Install Apache Solr with tomcat

Use following commands installing Apache Solr with tomcat. Download Apache Solr by.
wget http://apache.techartifact.com/mirror/lucene/solr/3.6.2/apache-solr-3.6.2.tgz
Now extract files using and change directory.
tar xvfz  apache-solr-3.6.2.tgz
cd  apache-solr-3.6.2
Now copy solr.war file into webapps folder of tomcat and example application in Tomcat.
cp dist/apache-solr-3.6.2.war /var/lib/tomcat6/webapps/solr.war 
cp -fr example/solr /var/lib/tomcat6/
Now change owner of Solr application to Tomcat6 using.
chown -R tomcat6:tomcat6 /var/lib/tomcat6/solr
Now restart tomcat server using,
/etc/init.d/tomcat6 restart
And you have successfully setup Solr using tomcat. You can check your installation by going to
http://localhost:8080/solr/admin/
OR
http://<IP_ADDRESS>/solr/admin/
Till now we have seen how to install and configure Apache Solr, Now it is time to import your MySQL database into Solr for searching and indexing.

Edit solrconfig.xml file

Change directory to solr in tomcat6 using and edit solrconfig.xml
cd /var/lib/tomcat6/solr
vim conf/solrconfig.xml
And add following lines after <config> tag
<requestHandler name="/dataimport" class="org.apache.solr.handler.dataimport.DataImportHandler"> 
    <lst name="defaults"> 
        <str name="config">data-config.xml</str> 
    </lst> 
</requestHandler>
Next below in <lib> tags change directory path for apache-solr-dataimporthandler library to dist/ folder in apache-solr-3.6.2 which you downloaded while installation of Apache Solr.

Create data-config.xml file

Create data-config.xml file in conf/ folder and add below code.
<dataConfig> 
    <dataSource type="JdbcDataSource" driver="com.mysql.jdbc.Driver" url="jdbc:mysql://127.0.0.1/YOUR_DB_NAME" user="root" password="YOUR_DB_PASSWORD"/> 
    <document> 
            <entity name="id"  query="select * from students" > 
                <field column="id" name="id" /> 
               <field column="fullname" name="fullname" /> 
               <field column="first_name" name="first_name" /> 
                <field column="last_name" name="last_name" />            
            </entity> 
    </document> 
 </dataConfig>
Replace values appropriately with your database credentials and in fields section match column names in your columns in your table.

Edit schema.xml file

Edit schema.xml in conf/ folder and add below lines along with other field tags.
    <field name="id"  type="text_general"  indexed="true" stored="true"/> 
    <field name="fullname"  type="text_general"  indexed="true" stored="true"/> 
    <field name="first_name" type="text_general"  indexed="true" stored="true"/> 
    <field name="last_name"  type="text_general"  indexed="true" stored="true"/> 
    <dynamicField name="*" type="ignored" />
There might be a previous field tag with name as “id” which have required property then you can skip the “id” field tag

Copy Mysql Jdbc Driver into lib/ directory

Download Mysql Jdbc Driver from : http://dev.mysql.com/downloads/connector/j/5.0.html and after extracting copy .jar file in /var/lib/tomcat6/solr/lib if lib directory is not present just create it inside solr/.

And restart tomcat server using command
/etc/init.d/tomcat6 restart

Import database into Solr

To import the database you have to first reload configuration using.
http://localhost:8080/solr/dataimport?command=reload-config
Now you can import and index your database using
http://localhost:8080/solr/dataimport?command=full-import
If everything goes fine, you can play with the data and various search options at:
http://localhost:8080/solr/admin/
OR
http://<IP_ADDRESS>/solr/admin/
If you get any error then you can check tomcat logs using :
tail -n 500 /var/lib/tomcat/logs/catalina.TODAY'S_DATE.log

If that doesn’t help, please drop us a line or leave a comment and we can help you out.

In next article we would be see how Solr can be integrated with PHP’s Yii Framework.