What method should be used for searching this mysql dataset?

Posted by GeoffreyF67 on Stack Overflow See other posts from Stack Overflow or by GeoffreyF67
Published on 2010-04-18T21:40:05Z Indexed on 2010/04/18 21:43 UTC
Read the original article Hit count: 307

Filed under:

mysql

|

large

|

dataset

|

php5

|

search

I've got a mysql dataset that contains 86 million rows.

I need to have a relatively fast search through this data.

The data I'll be searching through is all strings.

I also need to do partial matches.

Now, if I have 'foobar' and search for '%oob%' I know it'll be really slow - it has to look at every row to see if there is a match.

What methods can be used to speed queries like this up?

G-Man

© Stack Overflow or respective owner

Related posts about mysql

How to remove MySQL completely with config and library files on ubuntu 12.04 gnome 3.0

as seen on Ask Ubuntu - Search for 'Ask Ubuntu'
I tried everything till now: sudo apt-get remove mysql-server mysql-client mysql-common sudo apt-get purge mysql-server mysql-client mysql-common sudo apt-get autoremove and even more commands... But whenever I am trying to locate mysql. I get a no. of files related to mysql command: shell>>… >>> More
mysql: Cannot load from mysql.proc. The table is probably corrupted

as seen on Server Fault - Search for 'Server Fault'
Mysql was started: /usr/bin/mysqld_safe --datadir=/srv/mysql/myDB --log-error=/srv/mysql/logs/mysqld-myDB.log --pid-file=/srv/mysql/pids/mysqld-myDB.pid --user=mysql --socket=/srv/mysql/sockets/mysql-myDB.sock --port=3700 but when I'm trying to do something: ERROR 1548 (HY000) at line 1: Cannot… >>> More
Why is there a /etc/init.d/mysql file on this Slackware machine? How could it have gotten there?

as seen on Server Fault - Search for 'Server Fault'
A client of my IT-consulting service owns a web-development shop. He's been having problems with a Slackware 12.0 server running MySQL 5.0.67. The machine was set up by the client's sysadmin, who left on bad terms. My client no longer employs a sysadmin. As far as I can tell, the only copy of MySQL… >>> More
mysql: Bind on unix socket: Permission denied

as seen on Server Fault - Search for 'Server Fault'
Can't start mysql with: sudo /usr/bin/mysqld_safe --datadir=/srv/mysql/myDB --log-error=/srv/mysql/logs/mysqld-myDB.log --pid-file=/srv/mysql/pids/mysqld-myDB.pid --user=mysql --socket=/srv/mysql/sockets/mysql-myDB.sock --port=3700 120222 13:40:48 mysqld_safe Starting mysqld daemon with databases… >>> More
MySQL – Learning MySQL Online in 6 Hours – MySQL Fundamentals in 320 Minutes

as seen on SQL Authority - Search for 'SQL Authority'
MySQL is one of the most popular database language and I have been recently working with it a lot. Data have no barrier and every database have their own place. I have been working with MySQL for quite a while and just like SQL Server, I often find lots of people asking me if I have a tutorial which… >>> More

Related posts about large

Windows Seven, Large and Extra Large icons missing.

as seen on Super User - Search for 'Super User'
Some of my Large icons and extra large icons in windows 7 have gone blank. Nothing shows up in their place. The media icons are fine, thumbnails are created, tho My Computer can't show an icon for my drives when i ask it to show them in Large and Extra Large formats. Also I can't change my computer… >>> More
AJAX get data from large HTML page as the large HTML page loads

as seen on Stack Overflow - Search for 'Stack Overflow'
Not entirely sure whether this has a name but basically I have a large HTML page that is generated from results in a db. So viewing the HTML page (which is a report) in a browser directly does not display all contents immediately but displays what it has and additional HTML is added as the results… >>> More
Searching a large list of words in another large list

as seen on Stack Overflow - Search for 'Stack Overflow'
I have a list of 1,000,000 strings with a maximum length of 256 with protein names. Every string has an associated ID. I have another list of 4,000,000,000 strings with a maximum length of 256 with words out of articles and every word has an ID. I want to find all matches between the list of protein… >>> More
Creating a multi-column rollover image gallery with HTML 5

as seen on ASP.net Weblogs - Search for 'ASP.net Weblogs'
I know it has been a while since I blogged about HTML 5. I have two posts in this blog about HTML 5. You can find them here and here.I am creating a small content website (only text,images and a contact form) for a friend of mine.He wanted to create a rollover gallery.The whole concept is that we… >>> More
MySQL Binary Storage using BLOB VS OS File System: large files, large quantities, large problems.

as seen on Stack Overflow - Search for 'Stack Overflow'
Hi Guys, Versions I am running (basically latest of everything): PHP: 5.3.1 MySQL: 5.1.41 Apache: 2.2.14 OS: CentOS (latest) Here is the situation. I have thousands of very important documents, ranging from customer contracts to voice signatures (recordings of customer authorisation for contracts)… >>> More