V
32  
Tags
Attachments
KOHA
Zebra indexing


Zebra start i stop

mglavica@koha-upgrade:~$ sudo /usr/share/koha/bin/koha-zebra-ctl.sh start
Starting Zebra Server

mglavica@koha-upgrade:~$ sudo /usr/share/koha/bin/koha-zebra-ctl.sh stop
Stopping Zebra Server

Obnavljanje zapisa u indexu

Za to bi trebao služiti Zebraqueue Daemon, ali ne radi kako treba (koha-zebraqueue-ctl.sh). Umjesto toga index se rebuilda svako malo uz pomoć crona

Pokrenuti kao user koha.

KOHA_CONF=/etc/koha/koha-conf.xml
PERL5LIB=/usr/share/koha/lib

# m h  dom mon dow   command
*/4 * * * *   ps ax | grep -v grep | grep rebuild_zebra.pl || /srv/koha/misc/migration_tools/rebuild_zebra.pl -b -z >/dev/null

Dodavanje polja u pretraživanje

etc/zebradb/marc_defs/marc21/biblios/record.abs
mapping of MARC fields to indexes

etc/zebradb/biblios/etc/bib1.att
list of search indexes and their corresponding Z39.50 use attributes

etc/zebradb/ccl.properties
for searching purposes

UTF-8 to ASCII mappings

http://lists.indexdata.dk/pipermail/zebralist/2007-August/001707.html

Dijakritički znakovi

Situacija: u bazu su povučeni zapisi iz NSK (Voyager) u kojima su naša slova s dijaktiticima zabilježena kao dva znaka (slovo bez dijakritika + dijaktitik).

Nije lijepo da takvi znakovi završe u Koha bazi, ali kad su već unutra, treba ih nekako učiniti pretraživima.
U datoteku /etc/koha/zebradb/etc/word-phrase-utf.chr dodati ekvivalente:

git

# Characters to be considered equivalent for searching purposes
equivalent č(cˇ)
equivalent Č(Cˇ)
equivalent ć(c´)
equivalent Ć(C´)
equivalent š(sˇ)
equivalent Š(Sˇ)
equivalent ž(zˇ)
equivalent Ž(Zˇ)

Sad to dobro radi kad se pretražuje s "normalnim" znakovima. Ostaje problem kod browsanja preko liknova - strgani zapisi nađu samo strgane (pr. http://10.60.0.253/cgi-bin/koha/opac-detail.pl?biblionumber=241488, Pavao Pavličić).

Zebra register i shadow

Veličinu zebra register i shadow treba podesiti sukladno veličini baze. Za oko 120.000 zapisa, dovoljno je 4G. Za FF je sada podešeno 16GB.

Z39.50 server

Dokumentacija

Ali sve je uzalud jer je pravi odgovor na http://www.indexdata.com/zebra/doc/zebrasrv.html

The first two servers, "server1" and "server2", can be reached by both listener addresses - since no listenref attribute is specified.

Pravo rješenje

test-yaz-zebra.sh

Server

služi za razmjenu zapisa

Konfiguracija

Dodati novi public server prema uputama na http://wiki.koha-community.org/wiki/Troubleshooting_Koha_as_a_Z39.50_server

Dokumentacija

Najkorisnije:

Ostalo:

FFZG

yaz-client

dpavlin@koha-dev:~$ yaz-client 
Z> open localhost:9999/biblios
Connecting...OK.
Sent initrequest.
Connection accepted by v3 target.
ID     : 81
Name   : Zebra Information Server/GFS/YAZ
Version: 3.0.52 e687cb7eb87c841f0d1a374174d51d30371f2d97
Options: search present delSet triggerResourceCtrl scan sort extendedServices namedResultSets
Elapsed: 1.605266
Z> find human
Z> show 1
Z> quit
See you later, alligator.

Use Koha's CCL queries:

dpavlin@koha-dev:~$ yaz-client -c /etc/koha/zebradb/ccl.properties localhost:9999/biblios
Connecting...OK.
Sent initrequest.
Connection accepted by v3 target.
ID     : 81
Name   : Zebra Information Server/GFS/YAZ
Version: 4.1.7 c43e561fdfe12989a6040163dae0e28ba0a02453
Options: search present delSet triggerResourceCtrl scan sort extendedServices namedResultSets
Elapsed: 0.001445
Z> querytype ccl2rpn
Z> find (kw,wrdl,rtrn=human ) not Suppress=1
Sent searchRequest.
Received SearchResponse.
Search was a success.
Number of hits: 1935, setno 1
SearchResult-1: term=    human cnt=1935, term=    1 cnt=21
records returned: 0
Elapsed: 0.025643
Z> 

Debug Zebra problems

enable all logging

# add all to http://www.loc.gov/z3950/agency/defns/bib1diag.html
ZEBRAOPTIONS="-v none,fatal,warn,all"

dpavlin@koha-dev:~$ sudo /etc/init.d/koha-zebra-ctl.sh restart
Restarting the Zebra Server

dpavlin@koha-dev:~$ sudo tail -f /var/log/koha/koha-zebradaemon-output.log | grep ERROR
18:45:58-29/03 zebrasrv(1) 605aecf5 [request] Init ERROR 1011 ID:81 Name:ZOOM-C/YAZ Version:4.1.7 c43e561fdfe12989a6040163dae0e28ba0a02453

Lookup error codes at http://www.loc.gov/z3950/agency/defns/bib1diag.html



Problemi

Neki zapisi nestaju u katalogu

Nepoznat obrazac u zapisima koji fale.

Nagađanje 1:

da li su problematični oni zapisi koji imaju različit biblionumber i biblioitemnumber?
To je dozvoljena kombinacija, ne bi trebalo smetati.
Problem je u nečem drugom, no ipak smo odlučili te zapise poindeksirati ponovo, ovako:

mysql> create temporary table foo as select max(id) as id from zebraqueue join biblioitems on biblioitems.biblionumber=zebraqueue.biblio_auth_number and biblionumber <> biblioitemnumber group by biblio_auth_number ;
mysql> update zebraqueue set done = 0 where id in (select id from foo) ;

Nakon toga pokrenuti rebuild_zebra.pl.

I dalje postoje zapisi koji se ne vide u katalogu.

Ovo javi kad se poindeksiraju svi zapisi s rebuild_zebra.pl -b -v -r:

00:24:15-02/02 zebraidx(1490) [log] Records: 238000 i/u/d 238000/0/0
00:24:19-02/02 zebraidx(1490) [log] MARC: Bad directory
00:24:19-02/02 zebraidx(1490) [warn] MARC: Base address does not follow directory
00:24:19-02/02 zebraidx(1490) [warn] MARC: Bad offsets in data. Skipping rest
00:24:19-02/02 zebraidx(1490) [warn] Record didn't contain match fields in (bib1,Local-number)
00:24:19-02/02 zebraidx(1490) [log] error grs.marcxml.record /tmp/gdqck1EhgW/biblio/exported_records 234318986
00:24:31-02/02 zebraidx(1490) [log] Merge 0.4% completed; 38 minutes remaining

Nagađanje 2:

Potrgan MARC

  • napraviti dump MARC zapisa iz Kohe (uz pomoć WebPAC2)
  • provjeriti s marclint

Duplanje zapisa u zebri

mglavica@koha:/etc/koha/zebradb$ diff -u /srv/koha/etc/zebradb/biblios/etc/bib1.att biblios/etc/bib1.att 
--- /srv/koha/etc/zebradb/biblios/etc/bib1.att 2010-02-20 21:41:34.000000000 +0100
+++ biblios/etc/bib1.att       2010-02-22 05:48:19.000000000 +0100
@@ -56,7 +56,7 @@
 att 51   Number-music-publisher
 att 52   Number-db
 att 53   Number-local-call
-att 54   ln
+att 54   language
 att 55   Code-geographic
 att 56   Code-institution
 att 57   Name-and-title

Out of memory


 

Upload Files

Click "Browse" to find the file you want to upload. When you click "Upload file" your file will be uploaded and added to the list of attachments for this page.

Maximum file size: 50MB

 
 
 
File Name Author Date Uploaded Size

Save Page As

Enter a meaningful and distinctive title for your page.

Page Title:

Tip: You'll be able to find this page later by using the title you choose.

Page Already Exists

There is already a page named XXX. Would you like to:

Save with a different name:

Save the page with the name "XXX"

Append your text to the bottom of the existing page named: "XXX"

Upload Files

Click "Browse" to find the file you want to upload. When you click "Add file" this file will be added to the list of attachments for this page, and uploaded when you save the page.

 
 
 
Add Tags

Enter a tag and click "Add tag". The tag will be saved when you save the page.

Tag: 

Suggestions: