Short description Search all wikis in a farm, or search a different farm wiki that you're currently looking at.
I maintain a wiki farm of about 6-7 subwikis. We usually have a wiki per program, with a Main InfoWiki that contains useful, general information. It would good in the search box to be able to either search all the farm wikis or a syntax like InfoWiki:<search> to search just the InfoWiki
The underlying problem is that users type in a search term for a page they think should exist. When they don't find it, they may realize that they should look in another wiki in the farm, but they have to change to that wiki just to get to the search field.
I did some searching in MoinMoin for cross farm searching and didn't find any information.
-- KlausHeinisch 2006-04-29 16:35:50 That's a feature, that i would need in the near Future: Importing a TWiki-Farm with 4 Subwikis to MoinMoin, i need to realize some dedicated instances of MoinMoin. And i am looking for the "FarmSearch-Feature" in this MoinMoin-Wiki...
-- OliverSiemoneit 2006-11-13 21:30:45 I have found only this: WikiRpc should allow to query also other wikis in a farm and allow to retrieve search results from there. May the existing macros TextSearch and FullSearch could be modified accordingly and get a switch to search the whole farm or only a specific wiki. See also MacroMarket/WikiList for some code that could be interesting for rewriting these macros.
-- OliverSiemoneit 2006-11-16 20:52:13 This is a first version of a farm search macro. It the second bit of code I have ever written for Moin. It's not the best solution (bad performance), needs improvment, but might works. However this macro needs testing and I don't have a wikifarm or want to set up one. So please help.
Download farmfullsearch.py and save it to the macro directory of one of your farm wikis.
Add to some wikipage the code [[FarmFullSearch(Test)]] and save the page. The macro does call via WikiRpc the wiki http://wikifarm.koumbit.net/ and does retrieve search results for "Test" there. This should work. I have tested this. I could not test searching with WikiRpc within a farm.
So please open now farmfullsearch.py and change in line 75 the url http://wikifarm.koumbit.net/ into a url of another farm wiki in your farm as it is written in the farmconfig file. Save the macro and test again. I'm not sure whether WikiRpc does also work in a farm, i.e. WikiRpc on the same machine.
If WikiRpc does also work within a farm, you can try this version. Theoretically if WikiRpc does work within in farm, this marco should search all farm wikis. Only farmfullsearch action is now missing so that you can also use the input box with the search button on a page. Please do test the second version of the fullfarmsearch macro either und let me know your results.
Thanks for getting this started! You are missing a from farmconfig import wikis line. Also I don't see where looping over the wikis in the farm happens. I will continue to test when I have more time (currently it does not work on my testfarm). Does anybody know if this XMLRPC-approach is at all compatible with external authentication (we use Apache/SSPI)? I fear it is not. -- DavidLinke
Hi David! I have provided above a version (farmfullsearch2.py) that should work on a farm and does loop over the wikis. But I could not test it. -- OliverSiemoneit 2006-11-17 13:55:05
Oliver, I did a quick test: It works nice for wikis that don't required authentication. For wikis where authentication is required an error is raised. You should put the lines searchresults = srcwiki.searchPages(searchterm) and allpages = srcwiki.getAllPages() into a try..except clause to handle that. If the error is catched it is also required to reset searchresults to an empty list at the beginning of the loop over the farmwikis; else the same result is returned for the wiki that cannot be accessed.
-- OliverSiemoneit 2006-11-17 15:52:09 Nice to hear that it is also working on a farm.
Concerning authentication you can find some information on WikiRpc. Do also have a look at at moinmoin/script/old/xmlrpx-tools in the distribution code especailly the file wikibackup.py for some examples.
- "Basic Authentication" can be handled. However, there seems to be no way to handle SSPI (AKA: NTLM).
Search performance is bad at the moment because I get a complete list off all the pagenames of the wiki, which I actually don't need! I'm only interested in the total number of pages in the remote wiki, which I get at the moment by len(allpages). I don't use the huge list of pagenames! If we could find another solution for that, search time will go down by more than half of the time! Just think of big wikis with >5000 pages...
- Please do compare the total number of pages you get with remote search by len(allpages) and the total number of pages you get displayed when searching directly in the built-in-search-function of the other wiki. These numbers do differ! I don't know why...
WikiRpc does not support to search into titles - so currently no FarmTitleSearch is possible.
-- OliverSiemoneit 2006-11-17 19:55:57 Here are the missing actions files for both version of the fullfarmsearch macro: a_farmfullsearch.py a_farmfullsearch2.py . Maybe someone else can finish the macros and actions since I don't have a wikifarm and thus cannot test further developement steps as suggested e.g. by David.
- Hi Oliver. I succesfully tested version2 of macro and action on my testfarm. I also added the suggested changes:
Related Topics
See CommonUserAccounts for having one common user account for different farm wikis to emulate "TWiki Webs".
For farm support by themes see DavidLinke/Sinorca4Moin
Also, see the FeatureRequests/OverlayPagesForFarms for a solution that allows a wiki web to be overlayed/shared with other webs in the farm. It doesn't use RPC, it only works if the wiki farms are on the same machine (web server).
Comment
it's always better to show a diff and not a description what to do on a specific line. diff -u old new because code gets changes. -- ReimarBauer 2006-11-16 21:46:31
-- OliverSiemoneit 2006-11-16 22:17:06 Hi Reimar! Thanks for the hint! I don't have a diff on my windows system right now, but I will have to look how I can produce theses diff-files. Do you have any suggestion? Is there some tool for win, which renders the same diff output as I can often find here on this wiki? By the way: Is there also a tool to do it the other way round? Have a diff-file, have and old version and produce an new patched version of the file by patch diff old --> new automatically?
You may try http://winmerge.org/ (nice GUI). Commandline tools for windows like diff and patch are e.g. included in msys (http://www.mingw.org/). Patching and diffing is also handled by nearly all version management systems. -- DavidLinke 2006-11-17 10:35:38
For local farms (sharing the same server / file system), supporting indexed interwiki farm search in xapian search will be the best mid term.
This may also be the way to work around the problem of NTLM-authentication with XMLRPC. E.g. in my case (intranet) I have the same users for all wikis (AD domain users). If a user has passed authentication for one of the wikis the username could be used to search all wikis. This scenario is probably quite common. I guess coding a search would be easier if all wikis of the farm would have common user accounts. Maybe the latter can be added as a config option (the suggested solution is a hack especially on Windows systems where JunctionPoints would be required which are somewhat dangerous to use). -- DavidLinke 2006-11-18 16:43:40