Sphinx Results Take Huge Time To Show (Slow Index)

137 Views Asked by At

I'm new to Sphinx, i have simple table tbl_urls with two columns (domain_id,url) i created my index as below to get domain id and number of urls for any giving keyword

source src2
{
  type                  = mysql
  sql_host              = 0.0.0.0
  sql_user              = spnx
  sql_pass              = 123
  sql_db                = db_spnx
  sql_port              = 3306 # optional, default is 3306
  sql_query             = select id,domain_id,url from tbl_domain_urls
  sql_attr_uint         = domain_id
  sql_field_string      = url
}
index url_tbl
{
    source   = src2
    path     =/var/lib/sphinx/data/url_tbl
}
indexer
{
    mem_limit               = 2047M
}
searchd
{
    listen                  = 0.0.0.0:9312
    listen                  = 0.0.0.0:9306:mysql41
    listen                  = /home/charlie/sphinx-3.4.1/bin/searchd.sock:sphinx
    log                     = /var/log/sphinx/sphinx.log
    query_log               = /var/log/sphinx/query.log
    read_timeout            = 5
    max_children            = 30
    pid_file                = /var/run/sphinx/sphinx.pid
    max_filter_values       = 20000
    seamless_rotate         = 1
    preopen_indexes         = 0
    unlink_old              = 1
    workers                 = threads # for RT indexes to work
    binlog_path             = /var/lib/sphinx/data
    max_batch_queries       = 128
}

problem is the time taken to show results is over one min


SELECT domain_id,count(*) as url_counter
FROM ul_tbl WHERE MATCH('games')
group by domain_id limit 1000000 OPTION max_matches=1000000;show meta;

+-----------+-------+
| domain_id | url   |
+-----------+-------+
|      9900 |   444 |
|     41309 |    48 |
|     62308 |   491 |
|     85798 |   401 |
|       595 |  4851 |


13545 rows in set (3 min 22.56 sec)

+---------------+--------+
| Variable_name | Value  |
+---------------+--------+
| total         | 13545  |
| total_found   | 13545  |
| time          | 1.406  |
| keyword[0]    | games  |
| docs[0]       | 456667 |
| hits[0]       | 514718 |
+---------------+--------+

table tbl_domain_urls 100,821,614 rows
dedicated server HP Proliant 2xL5420 16GB RAM 2x1TB HDD

I need your support to optimize my QUERY or config settings, i need the results in the lowest time possible, i really appreciate any new idea to test

Note: I tried distributed index to use multiple core for processing without any noticable results

0

There are 0 best solutions below