Not logged in. · Lost password · Register
Forum: General Help and Support Features and Functionality RSS
Mysql
Avatar
bbolman #1
Member since Oct 2006 · 18 posts
Group memberships: Members
Show profile · Link to this post
Subject: Mysql
Sorry if this has been asked before, but, is there any way to get Dokuwiki to use a Mysql DB instead of texts files? I love DokuWiki, but the text files run so much slower than systems that use mysql (like Mediawiki) do. I also have so many pages that backing up the files takes forever. Is there some sort of Dokuwikisql version or something along those lines? Thanks for any help.
Avatar
andi (Administrator) #2
Member since May 2006 · 2447 posts · Location: Berlin Germany
Group memberships: Administrators, Members
Show profile · Link to this post
Quote by bbolman:
is there any way to get Dokuwiki to use a Mysql DB instead of texts files?

No.

Quote by bbolman:
I love DokuWiki, but the text files run so much slower than systems that use mysql (like Mediawiki) do.

Can you backup this by any benchmark data? Unless you can I'd say that this is not true.

Quote by bbolman:
I also have so many pages that backing up the files takes forever.

What? That makes no sense. Why should this take longer than to do a database dump?
Read this if you don't get any useful answers.
Lies dies wenn du keine hilfreichen Antworten bekommst.
Avatar
ChrisS #3
Member since Sep 2006 · 91 posts
Group memberships: Members, Wiki Managers
Show profile · Link to this post
Quote by andi:
Quote by bbolman:
I love DokuWiki, but the text files run so much slower than systems that use mysql (like Mediawiki) do.

Can you backup this by any benchmark data? Unless you can I'd say that this is not true.

Significant improvements were made in the speed of DW for the last release - 2006-11-06.  If you don't have that release, you should upgrade immediately.

If you get your cache settings right and your server is using a modern file system, on the same hardware, DW will be quicker than media wiki.  Using the latest version of DW, 2006-11-06, you can set "cachetime" to something quite long - 1 month or more.  DW is smart enough to work out if the cached copy needs refreshing before cachetime is reached if something has changed in the file or any of its dependencies.

On the same hardware:
- DW serving a page from cache is much faster than MediaWiki.
- DW generating a page from instructions is about the same as MediaWiki
- DW generating a page from raw wiki text is slower than MediaWiki.  The amount it is slower is dependent on the page content.

I forget the figures for backlinks and searching.  If searching is an issue, upgrade to a recent snapshot.  Further improvements have been made in DW's searching algorithm since the last release was made.

FYI - its a fallacy that DB's are faster than files.  The db data comes from a file.  Fwiw, the impact of a DB on DW would be negligible except possibly in searching/backlinks.  In order to generate a page in mediawiki or DW the webserver needs to load and process far more php files than wiki data files.  The time of access for the file system or the DBMS to retrieve the data is insignificant when compared to the other activities the wiki engine needs to carry out in order to display the page.
Avatar
bbolman #4
Member since Oct 2006 · 18 posts
Group memberships: Members
Show profile · Link to this post
Well I must admit I was just going off assumption on the speed front, but if you go to my site (www.romapedia.com) it takes a really long time to load. Is this my server's fault? Is it perhaps the fault of the template I'm using? Once you navigate around for a little while it goes much quicker, but this initial load (especially on slower connections) really annoys my users. Is there something I can do to speed that process up?

What I meant is that to backup files I just download them all, but there are hundreds of text files and so this takes quite a while. Is there a faster way I don't know about?
Avatar
ChrisS #5
Member since Sep 2006 · 91 posts
Group memberships: Members, Wiki Managers
Show profile · Link to this post
In reply to post #3
Quote by ChrisS on 2007-02-01, 10:24:
<snip>
you can set "cachetime" to something quite long - 1 month or more. 
<snip>

So, what is it set to?

Quote by bbolman on 2007-02-04, 21:46:
What I meant is that to backup files I just download them all, but there are hundreds of text files and so this takes quite a while. Is there a faster way I don't know about?
Zip ;)
Avatar
pchan #6
Member since Apr 2007 · 111 posts · Location: Lille, France
Group memberships: Members
Show profile · Link to this post
Subject: Large Scale Wiki
Hi, I 'm thinking of proposing DokuWiki for my company Wiki. I'm worried about data being stored in text files:
If my Wiki grows to 10 or 100's of thousands of pages, how will performance be like?
Thanks for this great software!
Avatar
rivabem #7
Member since May 2007 · 2 posts
Group memberships: Members
Show profile · Link to this post
In reply to post #2
Quote by andi on 2007-02-01, 09:29:
Quote by bbolman:
is there any way to get Dokuwiki to use a Mysql DB instead of texts files?

No.

By no, you mean "it's architecturally impossible"?

I need to use Oracle for storage, I'd rather not but requirements are tough. :(

I gave a fast look on DokuWiki source and found some specialized functions for io, but also noticed there are some functions like "file_exists" or "realpath" in other parts of the source.

It would be interesting if the retrieving/saving process was completely modular, so one could write plugins to different storage systems or to enhance something he'd lke.
Avatar
andi (Administrator) #8
Member since May 2006 · 2447 posts · Location: Berlin Germany
Group memberships: Administrators, Members
Show profile · Link to this post
Quote by rivabem:
I gave a fast look on DokuWiki source and found some specialized functions for io, but also noticed there are some functions like "file_exists" or "realpath" in other parts of the source.

It would be interesting if the retrieving/saving process was completely modular, so one could write plugins to different storage systems or to enhance something he'd lke.

No it's not very modular in that regard, in fact it is highly optimized to use the file system in a most optimal way. Simply replacing IO functions to store stuff in a DB would be a really bad idea and possibly make things slower, not faster.

BTW. requiring to use Oracle for a tool that does not use a database is plain stupid. Do you store your text processing files in Oracle as well?
Read this if you don't get any useful answers.
Lies dies wenn du keine hilfreichen Antworten bekommst.
Avatar
rivabem #9
Member since May 2007 · 2 posts
Group memberships: Members
Show profile · Link to this post
Quote by andi:
Quote by rivabem:
It would be interesting if the retrieving/saving process was completely modular, so one could write plugins to different storage systems or to enhance something he'd lke.

No it's not very modular in that regard, in fact it is highly optimized to use the file system in a most optimal way. Simply replacing IO functions to store stuff in a DB would be a really bad idea and possibly make things slower, not faster.

BTW. requiring to use Oracle for a tool that does not use a database is plain stupid. Do you store your text processing files in Oracle as well?

We do not require oracle for DokuWiki itself, but we need a wiki and we'd like to use DokuWiki, it´s pretty good.

The intention of the database is not making it faster, we're sure it wil not work that way. Our need is integration with other tools we have that cannot access/create a remote text file neither can access http server and use it as a service.

We may do some helping scripts, such as one to read jpg and pdf from the database to link them from the Wiki, or even a cronjob to synchronize the Oracle database to the Wiki database for searching, that's easy. But we were willing to develop something not that specific and more useful to the community.

And surprise yourself, most collaborative tools like Oracle Collaboration and Microsoft Sharepoint do save lots of files on the database. They have even a Explorer plugin for drag and drop! No, I don't like it, but...
Avatar
pchan #10
Member since Apr 2007 · 111 posts · Location: Lille, France
Group memberships: Members
Show profile · Link to this post
The disadvantage of using a myriad of files is that the hard disk is completely fractionned. The HH is logically divided into segments of (say 512 bytes, 1 K or more). Even if you have 182 bytes of information, the whole segment is used up and unavailable for anything else. If the content takes more than a segment, the HH puts them over several segments and links them; to get all the data, the HH head must locate and load all the individual segments and takes a heavy performance hit.
Unless you defragment the HH from time to time.

That's why people spend so much time and effort to write DB systems.  (or buy them or use Open source systems).

Another typical advantage of using DB is that highly complex queries can be written so that data can be found in one command and avoids several round turns to the server. However, this argument is not relevant for PHP which runs on the server. 

I think rivabem is right about replacing the io on files by db access. To be very objective, that is one way to further develop dokuWiki in the future. The core system should use plugins for IO, so that users can choose if they want simplicity or compacity/performance.

Maybe DokuWiki developpers have other visions of the future ?
Avatar
cihans #11
Member since May 2007 · 6 posts
Group memberships: Members
Show profile · Link to this post
In reply to post #3
Hi Chris and andi,

What are your opinions on my project, explained here :

http://forum.dokuwiki.org/thread/904
Avatar
pchan #12
Member since Apr 2007 · 111 posts · Location: Lille, France
Group memberships: Members
Show profile · Link to this post
In reply to post #10
Some more arguments for using a database:

1° It allows processing on one or more separate servers with replicated databases, such that lots of (quasi) simultaneous requests can be handled. Say, hundreds or thousands (or more) of pages seen per second, like the
more popular Wikipedia.
2° If one disk or server breaks down, backups can be put into operation within minutes/seconds (according to some advertisements from Oracle). Maybe some RAID disks can do that too, so this argument is not strong.

Of course, these are not the objectives of DokuWiki which aims at building highly functional WIKIs with low-load usages. There isn't any upgrade possible or planned, should the load hopefully/eventually goes up. You'd have to change... WIKI !
chi #13
Member since Jun 2006 · 1851 posts · Location: Munich Germany
Group memberships: Members, Super Mods, Wiki Managers
Show profile · Link to this post
Quote by pchan:
1° It allows processing on one or more separate servers with replicated databases, such that lots of (quasi) simultaneous requests can be handled. Say, hundreds or thousands (or more) of pages seen per second, like the
more popular Wikipedia.

Databases are IMO not necessary for load balancing setups. You could achieve that in various other ways too.

Quote by pchan:
2° If one disk or server breaks down, backups can be put into operation within minutes/seconds (according to some advertisements from Oracle). Maybe some RAID disks can do that too, so this argument is not strong.

High availability setups aren't db related either. There are also various other ways to achieve that. And IMHO here's what makes DokuWiki in this case stronger than DB driven systems. Even if the server breaks down the data is stll readable as it doesn't needs some db daemon running in order to access it ;-).

Quote by pchan:
Of course, these are not the objectives of DokuWiki which aims at building highly functional WIKIs with low-load usages. There isn't any upgrade possible or planned, should the load hopefully/eventually goes up. You'd have to change... WIKI !

IMHO The only argument that speaks for a database would be to store the index/meta/search relevant parts of the data to get better response times in these areas. And eventually this could be the case in the future - since sqlite is part of php5 ;-).
Please add [SOLVED] to the initial thread subject if you feel your question has been answered.
If my answer doesn't make sense maybe your question didn't either - just visit http://facepalm.org.
bmn (Former member) #14
No profile available.
Link to this post
I was wondering if there have been any updates on this in the last year and a half. Would it be any easier to add support for a database backend now?
Close Smaller – Larger + Reply to this post:
Verification code: VeriCode Please enter the word from the image into the text field below. (Type the letters only, lower case is okay.)
Smileys: :-) ;-) :-D :-p :blush: :cool: :rolleyes: :huh: :-/ <_< :-( :'( :#: :scared: 8-( :nuts: :-O
Special characters:
Go to forum
Imprint
This board is powered by the Unclassified NewsBoard software, 20120620-dev, © 2003-2011 by Yves Goergen
Current time: 2014-04-24, 00:11:54 (UTC +02:00)