Best Practice of Field Collapsing in SOLR 1.4

Posted by Dominik on Stack Overflow See other posts from Stack Overflow or by Dominik
Published on 2010-04-08T06:25:39Z Indexed on 2010/04/08 6:33 UTC
Read the original article Hit count: 263

Filed under:

solr

I need a way to collapse duplicate (defined in terms of a string field with an id) results in solr. I know that such a feature is comming in the next version (1.5), but I can't wait for that. What would be the best way to remove duplicates using the current stable version 1.4?

Given that finding duplicates in my case is really easy (comparison of a string field), should it be a Filter, should I overwrite the existing SearchComponent or write a new Component, or use some external libraries like carrot2?

The overall result count should reflect the shortened result.

Related posts about solr

Faceted search with Solr on Windows

as seen on ASP.net Weblogs - Search for 'ASP.net Weblogs'
With over 10 million hits a day, funda.nl is probably the largest ASP.NET website which uses Solr on a Windows platform. While all our data (i.e. real estate properties) is stored in SQL Server, we're using Solr 1.4.1 to return the faceted search results as fast as we can.And yes, Solr is very… >>> More
Severe errors in solr configuration: Error loading class 'solr.TrieDateField'

as seen on Server Fault - Search for 'Server Fault'
Installed Solr on my Windows XP PC. Tomcat seems to be working fine. Cannot get Solr to work. I noticed the TrieDateField is declared in a file called schema.xml in the SolrHome directory. Any thoughts? The Url http://localhost:8080/solr/ returns: HTTP Status 500 - Severe errors in solr configuration… >>> More
Solr error; Anybody know what this means?

as seen on Stack Overflow - Search for 'Stack Overflow'
I am installing solr on my VPS (Ubuntu 9.10) via PuTTY. First, I thought about installing Solr with Tomcat, but then after installing tomcat, I changed my mind and went for the Jetty which comes with Solr. Now that I have setup everything on my Server, and try to start the "start.jar" file, I get… >>> More
Solr DataImportHandler configuration

as seen on Stack Overflow - Search for 'Stack Overflow'
I want to get data from mysql database with the help of DataImportHandler so i can create indexes. Now I've configured my Solr instance so that it works on Tomcat (the example admin page), but if I try to change the sorlconfig.xml file i'll get the error message. I'm working with Solr 3.6 So my configuration… >>> More
Java exception when the traffic grow up

as seen on Server Fault - Search for 'Server Fault'
I have an error with java/solr when the traffic grows up. It seems Solr tries to cast a java.lang.Object to a org.apache.solr.common.util.ConcurrentLRUCache$CacheEntry SEVERE: java.lang.ClassCastException: [Ljava.lang.Object; cannot be cast to [Lorg.apache.solr.common.util.ConcurrentLRUCache$CacheEntry; … >>> More

Developer IT