You might hitting some soft limits - mainly RAM. Check the file sizes of the files in data/index DokuWiki need to fit each of the files into RAM (not all at once but each on their own) and it might need about twice or even more RAM than the file size. So compare that with your memory_limit setting for PHP.
You might want to consider switching to a "real" search engine. The elasticsearch plugin comes to mind.