Misty thoughts » web

Setting up Solr for Drupal under tomcat

Kfir — Tue, 13 Jul 2010 06:37:56 +0000

Disclaimer: This method will probably not work for all servers running tomcat, it is more a personal guide then a general one.

Solr is a "popular, blazing fast open source enterprise search platform from the Apache Lucene project". Drupal search has traditionally been less then adequate. Using Solr to index and search you Drupal site is therefore quite helpful.

To start, download the Solr package (link) and the Solr drupal module (link). the current standard distribution is Solr 1.4 and Drupal 6.
Unarchive the tar files in a location that is reachable from your server.

We will assume that tomcat is installed in /opt/tomcat

In the Solr package there is an example application. That is the Solr application we are going to use. If you can write your own, you probably don't need this tutorial.

Step 1:
Copy the example application, found in the solr package under example/webapps to the tomecat webapps folder which should be /opt/tomcat/webapps. when copying the file rename it to something other the solr. this will enable you to have multiplie solr installations running under one Tomcat container. We will assume the app is renamed new_solr_inst. If you are in the tomcat root folder the command should like like this:
cp /path/to/solr/package/apache-solr-1.4.0/apache-solr-1.4.0/example/webapps/solr.war webapps/new_solr_inst.war

Tomcat will pick up the war file and open it automatically after a restart.

Step 2:
In the tomcat root folder create a folder to hold your Solr configuration and data.
mkdir new_solr_inst
Copy the example app configuration into that folder
cp -R /path/to/solr/package/apache-solr-1.4.0/apache-solr-1.4.0/example/solr/* new_solr_inst

Step 3:
You need to replace the original schema.xml and solrconfig.xml with the ones provided by the drupal module. Copy these two files to your Solr configuration folder. Assuming you are currently at the tomcat root folder
cp /path/to/solr/module/schema.xml new_solr_inst/conf
cp /path/to/solr/module/solrconfig.xml new_solr_inst/conf
The Solr app is now ready for use.

Step 4:
The last step is needed to inform Tomcat about the app and its settings. Go to conf/Catalina/localhost and create the file new_solr_inst.xml. In that file fill the following content:

Step 5:
You need to restart tomcat
/opt/tomcat/bin/shutdown.sh
/opt/tomcat/bin/startup.sh
And your Solr should be running. There are three tests we can use to check everything went well.

Tomcat has opened the war file. There is now a folder new_solr_inst in the webapps folder of Tomcat.
Solr has made a data directory in the Solr home folder (that is /opt/tomcat/new_solr_inst)
In your browser go to the Solr app http://domain:tomcat_port/new_solr_inst and you should see a welcome screen

If all is well you are ready to combine this Solr installation with your Drupal

Step 6:
Activate the Apache-Solr module and go to the Solr module settings page admin/settings/apachesolr
Assuming tomcat and drupal are running on the same server fill in localhost as the host name, fill in the port tomcat is running on and solr path should be your app name in the tomcat webapps folder. In our case is it new_solr_inst.

Save the settings. Drupal should inform you that a connection is made with Solr

Notes

It is not nessecary to put the app configuration in the Tomcat root folder. you can probably (though I have not tested this) place it anywhere you want. you do need to make sure that the environment tag in the Tomact configuration file (the one in the conf/Catalina/localhost folder) is pointing to the right place.
If you make changes to the schema you may need to dump all the indices Solr has made. you can do this via the Drupal admin interface for the Solr module.

jQuery animation problem in ie7

Kfir — Fri, 28 May 2010 11:24:30 +0000

For one of our projects, I was creating a photo bar. When the mouse was over one of the photos it was ment to increase in size and decrease back to original size when the mouse left. Unfortunately the client for which this was developed uses ie7 exclusively. This has brought two problems:
1. ie7 does not support the mouseleave event. I needed to replace it with mouseout event.
2. ie7 does not accept the position property in the jQuery animate function. Giving the position as a parameter will cause an error.

Getting session ID in Drupal 6

Kfir — Thu, 25 Feb 2010 20:49:22 +0000

Drupal saves its sessions in the sessions table. One can always query they table to get the session ID. There is however a faster way.
If the user is logged it, the session id can be found with the following code:

global $user;
$user->sid;

In any case the session id can also be read from the cookie:

$_COOKIE[session_name()];

How to truly disable drupal cache

Kfir — Fri, 29 May 2009 10:26:05 +0000

Drupal, like most CMSes uses cache intensively to speed things up. In a production site that is exactly what you what. However when developing, cache can cause you much pain.

If your an expert in drupal you probably already realized when you need to clear the cache in order to see changes and when not too. the developer module even makes it quite easy to do. However for me, and I am quite sure to quite a lot of other people it is not that clear.

"Wait", you might say. "Drupal has a disable cache setting". You would be right to say that. Only it doesn't completely disable the cache, only part of the cache, namely the page caching. To truly disable cache you need to do a bit of a hacking to the cache code. Yes I know they say to never hack core (cache is part of the core drupal release). for me this was worth it. I figure as long as you remember its there and don't use it in production you should be fine. The following piece of code needs to be added in include/cache.inc. There are two functions there that you need to edit:

cache_get
cache_set

In cache_get place the code right after the global $user. In cache_set place it at the start of the function. Here is the code that you need to add.
if(variable_get('cache', CACHE_DISABLED) == CACHE_DISABLED) { return 0; }
Kudos for this hack goes to Rolf van der Krol. Cheers mate.

Installing Apache2, PHP5 and MySQL5 on OS X Tiger

Kfir — Mon, 04 May 2009 19:45:50 +0000

Tiger comes bundled with Apache 1.3 and PHP4. While these are still used for web development they are defenetly on the way out. For this reason I have decided to set up a more up to date MAMP installation on my old Powerbook G4. While this was all done on Tiger, it will most likely also work on Leopard.

Preparations

The installation process follows mostly the unix standard. That means that you will want to have the a /usr/local folder made. If you do not have it yet, type the following in terminal:

sudo mkdir /usr/local

The advantage of using local is that whenever a system updates itself it leaves local alone. If you install directly into /usr/bin, there is a chance that a system update will overwrite your installed software with its own update. If you lost me by now, you are probably off downloading a MAMP binary now.

The first step is to retrieve the packages we will need to properly install MAMP. To keep it nice and organized make a folder that will contain the source code, if you do not yet have a designated folder.

mkdir ~/src

We can now start installing MAMP

Installation

MySQL

The first thing that will be installed is MySQL. You can get the latest dmg here. After installing MySQL from the dmg, I also recommend installing Sequelpro if you want a GUI interface to the database. For building MySQL yourself from source check this.

Apache2

Apache comes bundled with OS X. Note that the control panel will only affect the default installation. Starting and stopping your own Apache installation will need some command line work, or some scripting. The following installs Apache 2 into /usr/local/sbin/apache2, if you prefer to install into another folder change the prefix to that folder.
curl -O http://apache.hippo.nl/httpd/httpd-2.2.11.tar.gz tar xvzf http-2.2.11.tar.gz cd http-2.2.11/ ./configure --prefix=/usr/local/sbin/apache2 --enable-modules=all --enable-mods-shared=all make sudo make install cd ..
Before we continue to build PHP there are some extra libraries that are needed. If you are not interested in building PHP with GD you can probably skip most of them.

Freetype

A Free, High-Quality, and Portable Font Engine. You can learn more about it here.
curl -O http://mirrors.zerg.biz/nongnu/freetype/freetype-2.3.9.tar.gz tar xvzf freetype-2.3.9.tar.gz cd freetype-2.3.9/ ./configure make sudo make install cd ..

JPEG

Support for jpeg image format
curl -O http://www.ijg.org/files/jpegsrc.v6b.tar.gz tar xzf jpegsrc.v6b.tar.gz cd jpeg-6b/ cp /usr/share/libtool/config.sub . cp /usr/share/libtool/config.guess . ./configure --enable-shared make sudo make install cd ..

Libpng

Support for png image format
curl -O ftp://ftp.simplesystems.org/pub/libpng/png/src/libpng-1.2.36.tar.gz tar xjf libpng-1.2.36.tar.gz cd libpng-1.2.36/ ./configure make sudo make install cd ..

openSSL

Install the latest version of openSSL
curl -O http://www.openssl.org/source/openssl-0.9.8k.tar.gz tar xvzf openssl-0.9.8k.tar.gz cd openssl-0.9.8k ./config --prefix=/usr/local --openssldir=/usr/local/ssl make sudo make install ./config shared --prefix=/usr/local --openssldir=/usr/local/ssl make clean make sudo make install

PHP

Now that the extra libraries are available we can continue to build PHP. We choose to install it into /usr/local/php5. As before if you wish to install into a different location, change the prefix.
curl -O http://de3.php.net/distributions/php-5.2.9.tar.bz2 tar xjf php-5.2.9.tar.bz2 cd php-5.2.9/ ./configure --prefix=/usr/local/php5 --with-apxs2=/usr/local/sbin/apache2/bin/apxs --with-config-file-scan-dir=/usr/local/php5/php.d --with-iconv --with-openssl=/usr/local --with-openssl-dir=/usr/local/ssl --with-zlib=/usr --with-gd --with-zlib-dir=/usr --with-xmlrpc --with-iconv-dir=/usr --enable-exif --enable-sqlite-utf8 --enable-ftp --enable-sockets --enable-dbase --enable-mbstring --with-bz2=/usr --enable-fastcgi --enable-cgi --enable-zip --with-curl --with-mysql=/usr/local/mysql --with-pdo-mysql=/usr/local/mysql --with-libxml-dir=shared,/usr/local/php5 --with-jpeg-dir=/usr/local/php5 --with-png-dir=/usr/local/php5 --with-freetype-dir=/usr/local/php5 make sudo make install
You should now have a ready development system.

Finalization

To make sure that the newly installed software is used instead of the default installed one make sure that /usr/local and the bin and php5 folders within are in the path before /usr/bin.
To start the Apache server use the following command:
sudo /usr/local/sbin/apache2/bin/apachectl start
Use stop and restart to stop or restart the server.

Credit

This article is inspired by the following three great guides:
Compiling Apache and PHP (and MySQL) on Mac OS X Leopard
Apache 2 + PHP 5 + MySQL 4 under OS X 10.3.9
Installing Apache, MySQL and PHP on Leopard

Django and static files

Kfir — Fri, 26 Sep 2008 18:18:18 +0000

After a long break I have resumed my side project in django. Last night I came upon a problem. When testing a page outside django, the page was rendered correctly. When it was rendered through django the javascript files were not located.

At this point I have to admit I was a bit fullish and forgot to look at the web server's log to see if the files were correctly served. that was an hour and a half of trying to figure out why the javascript functions were not found. So after smartening up I found out that the javascript files were not found. It seemed that django kept looking for them in the wrong location. Using the example here I eventually got it all to working.

It comes down to this. in the settings.py file there is a reference to the media URL and to the file system path to the media directory. Adding the following code to the urls.py
if settings.DEBUG: urlpatterns += patterns('', (r'^tripcalc/media/(?P .*)$', 'django.views.static.serve', {'document_root': settings.MEDIA_ROOT}), )
Will make sure that while django is in debug mode, serving of the static files will be done via django. for production sites it is better to let a dedicated web server (such as Apache) serve these static files.
And that took care of it. I can now continue with the development. And it also enabled me to call the CSS file, so style is also shown. hurray.

Installing Django on Dreamhost

Kfir — Mon, 19 May 2008 10:29:44 +0000

Dreamhost appears on the django site as one of the django friendly hosting services. Unfortunately, dreamhost does not officially support django. It does not have mod_python installed. Django is instead deployed using FastCGI. Hopefully sometime in the future mod_python will be added. There are a few good guides I have found, that explain how to setup django on a dreamhost account:

Jeff Croft has a good guide on his blog

Gordon Tillman also has a good informative page

The dreamhost wiki also has a guide

Between the three of them you can probably find all the information needed for installing django for use on a dreamhost account. I will not repeat what they explain but instead add some from my own experience.

Python

Dreamhost, at the moment of writing, is running python 2.4. Luckily it is possible for you to locally install python. I highly recommend it as it will enable you to setup the python environment exactly the way you want it, and it will make it easier to upgrade to future versions of python.

cd ~/soft wget http://www.python.org/ftp/python/2.5.2/Python-2.5.2.tgz tar xvfz Python-2.5.2.tgz cd Python-2.5.2 ./configure --prefix ~/install/dir --enable-shared make make install

Where ~/install/dir is the directory you want python installed in. I followed dreamhost's Unix account setup guide and installed it under run. I recommend you do to, as it is easier to have full control over you /usr/local.
Also adding setuptools makes future installs easier

cd ~/soft wget http://peak.telecommunity.com/dist/ez_setup.py ~/path/to/yourpython/python ez_setup.py

This will add the easy_install script which will simplify adding packages to your own python install. The final step is adding the new MySQLdb package

cd ~/soft svn co https://mysql-python.svn.sourceforge.net/svnroot/mysql-python/trunk/MySQLdb MySQLdb easy_install MySQLdb

This is assuming easy_install is in your PATH. If it is not it needs to be added.

Your now ready to install django using your very own Python installation. That is detailed enough so I will move to two problems I met after the installation.

Post Installation Problems

I met with two problems that had frustrated me for a few hours. To save you the future installer some pain here they are in case you experience something similar

syncdb after admin activation: This should have been very obvious to me but for some reason it escaped me. After enabling the admin page you must run django-admin.py syncdb in your project home page. What happened was that I ran it before I enabled the admin application. This lead to the creation of the needed tables in the MySQL database, but no tables for the admin application. After enabling the admin application, more tables need to be created to accommodate the new application data. The errors I got gave me the impression that there was an error in the MySQLdb egg, so I reinstalled it, then tried to find some workaround. Eventually I realized that I'm just missing the admin tables.

.htacess: This was the real mind bender. I kept getting an internal server error saying that it reached maximum internal allowed redirects. It was obviously a configuration error so I compared my .htaccess with that of the guides and it looked the same. So I looked else, but I kept coming back to the conclusion it has to be in the .htaccess. Yet no matter how often I looked at it I couldn't find what was wrong. I need to point out that I am no expret when it comes to apache and that maybe if I knew more about it this would have been simple, but I didn't so I got some gray hairs before I realized I'm missing a space between the - and the [L]. A bloody white space! I was feeling furious and incredibly stupid at the same time.

Django is now up and running, and I like it so far. I sure is a lot nicer to work with then with JSP and servlets. Enjoy your python

Javascript reintroduction

Kfir — Fri, 25 Apr 2008 19:10:05 +0000

AJAX is somewhat of a tricky subject for me. I am a huge fan of good UI and of good user experience. There is no doubt in my mind that since the introduction of javascript, and later on AJAX the general experience in using the internet has improved. For me, watching Adobe's new Photoshop express is something close to a miracle. I remember the days that some image rollover was the coolest thing on the web.
That said there are some downsides to this. Nowadays to check your webmail, you most likely will need quite a modern computer and a relative new browser. The internet raised as a means to share data, and it looks to me like that feature is being pushed a bit to the back to make room for the shiny, new, apps.

There is a lot to be said for online apps. I like the idea that my office programs are always available, on every system with internet access. Its more platform independent than any virtualization technology, easier sharing then any mesh and is probably cheaper. Then again, if all I want is to find some simple information about the address of some business, or restaurant, there is no reason for me not to be able to do it on a 10 year old computer running OS/2 with netscape 1. Ok I might have gotten carried away there, but a 10 year old system with a 10 year old OS and a 10 year old browser can probably handle basic HTML. Which means it can display web pages. So it should be good enough.
Except its not. Because every damn site has bells and whistles. And the people who are the most to blame are us, the users. We want hype, cool looking pages. Hell if I get a basic HTML page I go away without even looking at the content. What goes through my mind is either "This is some crap some 40 year old did in his garage after reading the w3c tutorial" or "this cant be the right spot". So everyone got bling bling. And we end up with browsers which are turning into an OS. I am quite happy with X thank you. I don't need to run Safari OS on top of that.

Ok so I got carried away again, but you get my point.

Now that my thoughts on AJAX specifically, and javascript in general are out of the way I can turn to the subject at hand.

javascript rocks!

Last time I made web pages was hmm, almost 8 years ago. A lot have changed then. It is a wonderful thing to see how it is actually possible to make usable UI on the web with javascript. Since I am now developing java2 EE apps for a living I got into javascript. Its wonderful. And it seems a lot of people think that because there are about as much js and AJAx framweorks as there are commands in js, if not more. I knew about DOJO since it was mentioned several time on the Django page. Trying to choose one seemd to me next to impossible. So I did the only thing I could. I asked an expert. I ended up with Prototypes and Scriptalicous. The name alone would have convinced me to us scriptalicous. The fact that Mark Pilgrim did part of the developing just strengthened me in my decision. I have been using he framework for about a week now and I like it.

To sum this post up, javascript brings along a lot to the table, but it also enforces a better table. I am a UI design enthusiast. I love my javascript.

Marsedit

Kfir — Fri, 25 Apr 2008 18:01:07 +0000

Offline blogging may sound like a bit of an odd idea, but the idea did stick with me. After all, wordpress is not exactly the world's best word processor. I just grabbed Marsedit and am going to see how this works for me. On other news, Im really turning into a twitter fan.

some nice maps

Kfir — Tue, 16 Oct 2007 06:50:26 +0000

I came across this site today. they create some nice graphic maps. For example this map shows the development of religions in the world. Its funny that Judaism is shown, but non of the religion of central or south america which were probably a lot larger, though they don't exist anymore, at least not in any real numbers.