ServiceStack Redis search is very slow. How to optimize?

Question

I'm using ServiceStack.Redis to implement a demo project. It contains two POCOs i.e Albums and its Songs.

Below is the search results measured using a stopwatch instance:

Time elapsed searching 5804 items is 00:00:00.1243984 <-- Albums
Time elapsed searching 138731 items is 00:00:02.0592068 <-- Songs

As you can see the search for the songs is taking too much time. I'm displaying the results in a WPF application wherein the search term is also entered. The lag is a no-go for redis.

Below is the code used for searching:

IEnumerable<int> songsFromRedis =
    songRedis.GetAll()
    .Where(song => song.Title != null 
        &&  song.Title.ToLowerInvariant().Contains(searchText))
    .OrderBy(song => song.Title)
    .Select(x => x.AlbumId);

If we cannot make it any faster, would ElasticSearch help ?

Answer 1

Any search oriented database would help. Even mysql fulltext search, which is notoriusly slow, would be a lot better here.

Elasticsearch is one good alternative, Sphinx another good one. ES has the easy scalability and ease of use, sphinx has the performance win and otherwise most of the common features but is a bit more work to learn and to scale.

Answer 2

The issue is how you're using Redis, i.e. songRedis.GetAll() downloads the entire dataset, deserializes all entities into C# objects and performs the search on the client.

You should never download and query an entire dataset across the network on the client (i.e with any datastore), even a full server-side table-scan query would perform much better since only the filtered results are returned to the client and not the entire dataset. Ideally even full server-side table-scans should be avoided and any queries should be made via an index.

Redis doesn't have support for indexes built-in, but when needed you can use a SET to manually create indexes between entities in Redis.