Hardware vs. efficient design
Friday, October 27th, 2006The search engine has a common problem, which is Quality vs. Speed. The more relevant the search results get, the longer they take to produce. One solution, which I was going to go with, was to throw more hardware at it! Though this would work, it is avoiding the issue, which could be summed up as poor design.
So do I go with the quicker approach of adding more hardware OR do I rewrite the engine from scratch to meet my needs (engine was first designed to handle only 1 million sites). Well what I plan to do is a little of both. Over the next couple of weeks I will be tweaking the engine to produce better quality results for cached searches and tweaking the non-cached results to return at a fast rate. Adding extra hardware to make sure I have redundancy and better caching should prove beneficial too.
The only pothole I will fall into is that a non-cached result could be rubbish and scare off the user. This will be something I will have to monitor. So far the lack of speed has managed to scare many people off!