Solr Enterprise Solutions Prove Worthless Without Connectors

Apache Solr is the technology behind enterprise search for major websites including eBay. Carsabi also utilizes it and they published the article "Optimizing Solr (Or How to 7x Your Search Speed)," which reveals some insights into Solr from their experience.

After reaching 1.8 million listings per month, the article reveals their basic installation slowed down in a major way. This prompted the company to optimize. 

According to the article, these are the specs on their former Solr solution:

Very briefly, our stack has gone through a few iterations which may be sufficient for your corpus volume – no sense in over-engineering. Postgres tables had to be denormalized at 100k vehicles, and we switched to WebSolr’s extremely convenient Solr solution at 300k – their Heroku plugin will create an installation in minutes for just $20/month. This worked very well until about 1M listings, at which point even their beefiest plan was returning results with >800ms latency.

The base technology of Solr is solid, but it is only a foundation. Solr does not even come with connectors. Of course, many vendors provide their own versions of Solr that come with the connectors needed to ensure true enterprise search accomodating files of all varieties--thanks to open source technology. 

Megan Feil, April 2, 2012