Interestring projects

TODO: test them out and write full description on this wiki :-)

• FreeLSS is a laser scanning program for the Raspberry Pi. It allows a Raspberry Pi to function as the core to a complete turn table laser scanning system.
• https://github.com/hairu/freelss
• PWM on the Raspberry pi - done properly (in hardware, stable)
• https://github.com/sarfata/pi-blaster
• https://github.com/ali1234/raspi-teletext
• https://github.com/jgarff/rpi_ws281x

• u-boot on sdcard for nfs boot: http://elinux.org/RPi_U-Boot

• RF transmitter for Raspberry Pi https://github.com/F5OEO/rpitx

development

• distcc setup http://openframeworks.cc/setup/raspberrypi/Raspberry-Pi-DISTCC-guide.html
• RPi (Raspbian arm) chroot on x86 Linux http://dev.xff.lt/b/archives/2015/01/11/rpi_raspbian_arm_chroot_on_x86_linux/
• https://wiki.debian.org/QemuUserEmulation https://wiki.debian.org/RaspberryPi/qemu-user-static

OV7670

• http://www.raspberrypi.org/phpBB3/viewtopic.php?uid=25856&f=44&t=11839&start=0

DHT11/DHT22

• http://www.tortosaforum.com/raspberrypi/dht11driver.htm
• http://learn.adafruit.com/dht-humidity-sensing-on-raspberry-pi-with-gdocs-logging/overview

BlinkM + i2c

• http://www.raspberrypi.org/forums/viewtopic.php?t=11969

CEC - control your TV

http://www.raspberrypi.org/forum/viewtopic.php?f=35&t=15749

root@raspberrypi:/home/pi# git clone https://github.com/Pulse-Eight/libcec.git
root@raspberrypi:/home/pi# cd libcec/
root@raspberrypi:/home/pi/libcec# apt-get install autoconf automake libtool liblockdev1-dev libudev-dev libraspberrypi-dev
root@raspberrypi:/home/pi/libcec# sh -x bootstrap
root@raspberrypi:/home/pi/libcec# ./configure --with-rpi-include-path=/opt/vc/include/ --with-rpi-lib-path=/opt/vc/lib/

##############################################################################

libCEC version 2:1:0 configured

Compilation flags:
  CXXFLAGS :       -g -O2 -fPIC -Wall -Wextra -Wno-missing-field-initializers  -Wno-psabi
  libCEC LDFLAGS : -lbcm_host -llockdev -lrt -ldl -lpthread  -L/opt/vc/lib/ -lvcos -lvchiq_arm -ludev  
  client LDFLAGS : -lrt -ldl -lpthread 

Configured features:
  Pulse-Eight CEC Adapter :             yes
  Pulse-Eight CEC Adapter detection :   yes
  Raspberry Pi support :                yes
  TDA995x support :                     no

You can now build libCEC by running:
  make

##############################################################################

make install

TV on

pi@raspberrypi ~ $ echo "on 0" | cec-client -s

Hints from forum

screen saver

http://www.raspberrypi.org/phpBB3/viewtopic.php?p=84831#p84831

# prevent screensaver from powering down display
setterm -blank 0 -powerdown 0 > /dev/tty0

# clear the display and turn off the flashing cursor
clear > /dev/tty0
setterm -cursor 0 > /dev/tty0

# turn the cursor back on when done with omxplayer
setterm -cursor 1 > /dev/tty0

YouTube transcoding in VLC

http://www.raspberrypi.org/phpBB3/viewtopic.php?p=163658#p163658

cvlc 'youtube.com/watch?v=XXXXXXXXXX' :sout='#transcode{vcodec=h264,vb=0,fps=0,scale=0,height=720,acodec=mp4a,ab=128,channels=2,samplerate=44100}:http{mux=ffmpeg{mux=flv},dst=:8080/youtube}' :sout-keep

Education materials

• http://pi.cs.man.ac.uk/download/Raspberry_Pi_Education_Manual.pdf
• http://www.ocr.org.uk/qualifications/by-subject/computing/raspberry-pi/
• http://www.cl.cam.ac.uk/projects/raspberrypi/

Raspberry Pi Accessories Starter Kit

http://www.conrad.com/ce/en/product/409208/Raspberry-Pi-Accessories-Starter-Kit?ref=searchDetail

• 1x Rapberry sticker

• Power supply for the Raspberry
• Sturdy acrylic housing, which includes all possible connections for this Raspberry and perfectly protects
• 4 GB SD card to store your programs
• https://learn.adafruit.com/adafruit-raspberry-pi-lesson-1-preparing-and-sd-card-for-your-raspberry-pi
• USB Micro Card Reader
• USB cable (for power supply or the power supply of a PC)
• 3 m Ethernet cable
• https://learn.adafruit.com/adafruits-raspberry-pi-lesson-6-using-ssh
• https://learn.adafruit.com/adafruit-raspberry-pi-lesson-7-remote-control-with-vnc
• TTL USB cable
• https://learn.adafruit.com/adafruits-raspberry-pi-lesson-5-using-a-console-cable/test-and-configure
• PI Cobbler (KIT hereby connect the RP with other circuit boards)
• Large breadboard circuit
• Colored board cable connector

• 5x 560 ohm resistors (for LEDS)
• 3x LED's (red, green, blue)
• https://learn.adafruit.com/raspberry-pi-e-mail-notifier-using-leds
• https://learn.adafruit.com/debugging-with-the-raspberry-pi-webide/debug-a-blinking-led
• 5x 10K resistors (for buttons)
• 3x pushbutton
• https://learn.adafruit.com/playing-sounds-and-using-buttons-with-raspberry-pi/bread-board-setup-for-input-buttons
• 1x photocell
• https://learn.adafruit.com/basic-resistor-sensor-reading-on-raspberry-pi/basic-photocell-reading
• 1x 1uf capacitor (for photocell)

GPU

• FFT
• http://www.raspberrypi.org/accelerating-fourier-transforms-using-the-gpu/
• https://github.com/raspberrypi/userland/tree/master/host_applications/linux/apps/hello_pi/hello_fft/qasm
• SHA-256
• http://rpiplayground.wordpress.com/
• https://github.com/elorimer/rpi-playground/tree/master/QPU
• Machine learning - the Deep Belief image recognition SDK
• http://petewarden.com/2014/06/09/deep-learning-on-the-raspberry-pi/
• http://scientistnobee.wordpress.com/2014/06/20/machine-learning-with-raspberry-pi/

Arduino

• Connect using I2C http://www.pihomeserver.fr/en/2013/08/13/raspberry-pi-home-server-arduino-lier-les-deux-via-bus-i2c/

JTAG

• https://github.com/synthetos/PiOCD/wiki/Using-a-Raspberry-Pi-as-a-JTAG-Dongle

Arduino ProMini 3.3V 8MHz

Disable serial port console

edit /boot/cmdline.txt and remove all ttyAMA0 leaving:

pi@raspberrypi ~ $ cat /boot/cmdline.txt
dwc_otg.lpm_enable=0 console=tty1 root=/dev/mmcblk0p2 rootfstype=ext4 elevator=deadline rootwait bcm2708.w1_gpio_pin=4

also, comment out ttyAMA0 in /etc/inittab

connect Arduino serial

RX->TX
TX->RX

https://github.com/dpavlin/avrdude-rpi/tree/fix_channel_in_use

pin definitions in GPIO.RPi are physical pins on board not BCM or wiring ip!

flasing

dpavlin@blue:~/Arduino/RF433_Sockets$ scp /tmp/build*.tmp/*.hex pi@rpi.tv:/tmp/
RF433_Sockets.cpp.hex                                        100%   19KB  18.6KB/s   00:00

pi@raspberrypi ~ $ cat avrdude.sh 
#!/bin/sh
hex=`ls -t /tmp/*.hex | head -1`
echo "flash hex: $hex"
sudo /home/pi/avrdude-rpi/avrdude-autoreset -C /home/pi/avrdude.conf -v -v -patmega328p -carduino -P/dev/ttyAMA0 -b57600 -D -Uflash:w:$hex:i 

pi@raspberrypi ~ $ ./avrdude.sh
flash hex: /tmp/RF433_Sockets.cpp.hex

avrdude: Version 6.1, compiled on Sep 17 2014 at 05:41:35
         Copyright (c) 2000-2005 Brian Dean, http://www.bdmicro.com/
         Copyright (c) 2007-2014 Joerg Wunsch

         System wide configuration file is "/home/pi/avrdude.conf"
         User configuration file is "/root/.avrduderc"
         User configuration file does not exist or is not a regular file, skipping

         Using Port                    : /dev/ttyAMA0
         Using Programmer              : arduino
         Overriding Baud Rate          : 57600
done with autoreset
strace: |/home/pi/avrdude-rpi/autoreset: Broken pipe
strace: |/home/pi/avrdude-rpi/autoreset: Broken pipe
         AVR Part                      : ATmega328P
         Chip Erase delay              : 9000 us
         PAGEL                         : PD7
         BS2                           : PC2
         RESET disposition             : dedicated
         RETRY pulse                   : SCK
         serial program mode           : yes
         parallel program mode         : yes
         Timeout                       : 200
         StabDelay                     : 100
         CmdexeDelay                   : 25
         SyncLoops                     : 32
         ByteDelay                     : 0
         PollIndex                     : 3
         PollValue                     : 0x53
         Memory Detail                 :

                                  Block Poll               Page                       Polled
           Memory Type Mode Delay Size  Indx Paged  Size   Size #Pages MinW  MaxW   ReadBack
           ----------- ---- ----- ----- ---- ------ ------ ---- ------ ----- ----- ---------
           eeprom        65    20     4    0 no       1024    4      0  3600  3600 0xff 0xff
           flash         65     6   128    0 yes     32768  128    256  4500  4500 0xff 0xff
           lfuse          0     0     0    0 no          1    0      0  4500  4500 0x00 0x00
           hfuse          0     0     0    0 no          1    0      0  4500  4500 0x00 0x00
           efuse          0     0     0    0 no          1    0      0  4500  4500 0x00 0x00
           lock           0     0     0    0 no          1    0      0  4500  4500 0x00 0x00
           calibration    0     0     0    0 no          1    0      0     0     0 0x00 0x00
           signature      0     0     0    0 no          3    0      0     0     0 0x00 0x00

         Programmer Type : Arduino
         Description     : Arduino
         Hardware Version: 2
         Firmware Version: 1.16
         Vtarget         : 0.0 V
         Varef           : 0.0 V
         Oscillator      : Off
         SCK period      : 0.1 us

avrdude: AVR device initialized and ready to accept instructions

Reading | ################################################## | 100% 0.01s

avrdude: Device signature = 0x1e950f
avrdude: safemode: lfuse reads as 0
avrdude: safemode: hfuse reads as 0
avrdude: safemode: efuse reads as 0
avrdude: reading input file "/tmp/RF433_Sockets.cpp.hex"
avrdude: writing flash (6770 bytes):

Writing | ################################################## | 100% 1.98s

avrdude: 6770 bytes of flash written
avrdude: verifying flash memory against /tmp/RF433_Sockets.cpp.hex:
avrdude: load data flash data from input file /tmp/RF433_Sockets.cpp.hex:
avrdude: input file /tmp/RF433_Sockets.cpp.hex contains 6770 bytes
avrdude: reading on-chip flash data:

Reading | ################################################## | 100% 1.49s

avrdude: verifying ...
avrdude: 6770 bytes of flash verified

avrdude: safemode: lfuse reads as 0
avrdude: safemode: hfuse reads as 0
avrdude: safemode: efuse reads as 0
avrdude: safemode: Fuses OK (E:00, H:00, L:00)
strace: |/home/pi/avrdude-rpi/autoreset: Broken pipe
strace: |/home/pi/avrdude-rpi/autoreset: Broken pipe
strace: |/home/pi/avrdude-rpi/autoreset: Broken pipe
strace: |/home/pi/avrdude-rpi/autoreset: Broken pipe
strace: |/home/pi/avrdude-rpi/autoreset: Broken pipe

avrdude done.  Thank you.

strace: |/home/pi/avrdude-rpi/autoreset: Broken pipe

DS1307 Tiny RTC

• http://www.instructables.com/id/Set-up-Real-Time-Clock-RTC-on-Raspberry-Pi/
• http://electronics.stackexchange.com/questions/98361/how-to-modify-ds1307-rtc-to-use-3-3v-for-raspberry-pi

Check if module has 5V pull-up I2C resistors! If it does (like Tiny RTC has R2 and R3) you will have to remove them.

root@raspberrypi:/home/pi# apt-get install i2c-tools

root@raspberrypi:/home/pi# modprobe i2c-dev

root@raspberrypi:/home/pi# i2cdetect -y 1
     0  1  2  3  4  5  6  7  8  9  a  b  c  d  e  f
00:          -- -- -- -- -- -- -- -- -- -- -- -- -- 
10: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- 
20: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- 
30: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- 
40: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- 
50: 50 -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- 
60: -- -- -- -- -- -- -- -- 68 -- -- -- -- -- -- -- 
70: -- -- -- -- -- -- -- --                         

root@raspberrypi:/home/pi# modprobe rtc-ds1307

root@raspberrypi:/home/pi# echo ds1307 0x68 > /sys/class/i2c-adapter/i2c-1/new_device

root@raspberrypi:/home/pi# hwclock --systohc -D --noadjfile --utc
hwclock from util-linux 2.20.1
Using /dev interface to clock.
Assuming hardware clock is kept in UTC time.
Time elapsed since reference time has been 0.722720 seconds.
Delaying further to reach the new time.
Setting Hardware Clock to 11:53:41 = 1429876421 seconds since 1969
ioctl(RTC_SET_TIME) was successful.

DS18B20

root@raspberrypi:/home/pi# grep w1 /boot/config.txt 
dtoverlay=w1-gpio

# reboot

root@raspberrypi:/home/pi# cat /sys/bus/w1/devices/28-0000043c5507/w1_slave 
50 05 4b 46 7f ff 0c 10 1c : crc=1c YES
50 05 4b 46 7f ff 0c 10 1c t=85000

Hmm, you will also want to remove R1 pullup to make it work?

avrdude

Using Raspberry Pi as Arduino ISP programmer

• https://github.com/onandoffables/avrdude-linuxgpio

root@raspberrypi:/home/pi# avrdude -c linuxgpio -p atmega328p

avrdude: AVR device initialized and ready to accept instructions

Reading | ################################################## | 100% 0.00s

avrdude: Device signature = 0x1e950f

avrdude: safemode: Fuses OK (E:05, H:DA, L:FF)

avrdude done.  Thank you.

logic analyzer

• bare metal: https://github.com/tuxyme/metal-pi
• Panalyzer - a RaspberryPi based Logic Analyzer https://github.com/richardghirst/Panalyzer

rotary encoder

http://www.bobrathbone.com/raspberrypi_rotary.htm

SWD

http://www.disk91.com/2015/technology/programming/swd-programming-using-a-raspberrypi/
https://github.com/disk91/PySWD

pi@raspberrypi ~ $ git clone https://github.com/disk91/PySWD
pi@raspberrypi ~ $ cd PySWD/

PIC

• rpp - PIC Programmer using GPIO http://holdenc.altervista.org/rpp/
• http://mujweb.cz/tlc/rpp/
• https://www.raspberrypi.org/forums/viewtopic.php?f=44&t=14689&start=75
• https://github.com/oh7bf/PiPIC
• https://github.com/WallaceIT/picberry

esp8089

• https://hackaday.io/project/8678-rpi-wifi-hat
• https://github.com/al177/esp8089

PS/2 GPIO

• http://www.deater.net/weave/vmwprod/hardware/pi-ps2/
• https://github.com/deater/vmw-meter/blob/master/pi-ps2/linux-kernel/pi-ps2gpio.c

GPIO header annotations

• https://github.com/splitbrain/rpibplusleaf/tree/master

Signal generator

• https://github.com/JamesP6000/WsprryPi

pi@rpi2 ~/WsprryPi $ sudo ./wspr --test-tone 1000000
A test tone will be generated at frequency 1.000000 MHz

Using local mbox device file with major 249.
Transmitting test tone on frequency 1.000000 MHz
Press CTRL-C to exit!

• http://github.com/gryrmln/RPi_Wobbulator

LCD panels

http://blog.reasonablycorrect.com/raw-dpi-raspberry-pi/

Pi Zero USB OTG

libcomposite

• http://isticktoit.net/?p=1383

PATA IDE emulation

• https://www.retrotronics.org/home-page/netpi-ide/

DPI LCD screen

• http://blog.reasonablycorrect.com/raw-dpi-raspberry-pi/
• https://www.raspberrypi.org/forums/viewtopic.php?p=1023780#p1023780

i2c clock stretching

• https://github.com/raspberrypi/linux/issues/254#issuecomment-217047171

power

• what about power it by USB http://forums.parallella.org/viewtopic.php?f=12&t=1128
• 2A to micro-usb connector next to the rj45 Ethernet http://forums.parallella.org/viewtopic.php?f=50&t=1081
• How to minimize Epiphany standby power? http://forums.parallella.org/viewtopic.php?uid=590&f=10&t=1069&start=0

• Critical issue with non-conforming USB powered hubs. http://forums.parallella.org/viewtopic.php?f=10&t=841
• USB port change http://forums.parallella.org/viewtopic.php?f=10&t=1072
• USB power fix http://forums.parallella.org/viewtopic.php?f=50&t=1510

• Where can I draw 5V for a fan ? http://forums.parallella.org/viewtopic.php?f=23&t=1049&start=10

Update boot flash

This is required to get CR10 led working

• http://forums.parallella.org/viewtopic.php?f=49&t=983
• https://github.com/parallella/parallella-hw/tree/master/boards/parallella-I/firmware

enviroment before u-boot update:

zynq-uboot> env print
baudrate=115200
bitstream_image=parallella.bit.bin
bootcmd=run modeboot
bootdelay=0
devicetree_image=devicetree.dtb
devicetree_size=0x20000
ethact=zynq_gem
ethaddr=04:4f:8b:00:10:13
fdt_high=0x20000000
initrd_high=0x20000000
ipaddr=192.168.0.99
kernel_image=uImage
kernel_size=0x500000
modeboot=run qspiboot
qspiboot=echo Configuring PL and Booting Linux...;mmcinfo;fatload mmc 0 0x4000000 ${bitstream_image};fpga load 0 0x4000000 0x3dbafc;fatload mmc 0 0x3000000 ${kernel_image};fatload mmc 0 0x2A00000 ${devicetree_image};bootm 0x3000000 - 0x2A00000
serverip=192.168.0.101
stderr=serial
stdin=serial
stdout=serial

Environment size: 655/131068 bytes

flasing output:

zynq-uboot> mmcinfo
Device: SDHCI
Manufacturer ID: 3
OEM: 5344
Name: SU08G 
Tran Speed: 50000000
Rd Block Len: 512
SD version 2.0
High Capacity: Yes
Capacity: 7.4 GiB
Bus Width: 4-bit
zynq-uboot> fatload mmc 0 0x4000000 parallella.7020.flash.bin
reading parallella.7020.flash.bin

4351800 bytes read
zynq-uboot> sf probe 0 0 0
SF: Detected N25Q128 with page size 64 KiB, total 16 MiB
zynq-uboot> sf erase 0 0x1000000
SF: Successfully erased 16777216 bytes @ 0x0
zynq-uboot> sf write 0x4000000 0 0x$filesize
SF: program success 4351800 bytes @ 0x0

# I skipped set ethaddr because it was already set correctly

zynq-uboot> set AdaptevaSKU SKUA101040
zynq-uboot> saveenv
Saving Environment to SPI Flash...
SF: Detected N25Q128 with page size 64 KiB, total 16 MiB
Erasing SPI flash...SF: Successfully erased 131072 bytes @ 0x4e0000
Writing to SPI flash...SF: program success 131072 bytes @ 0x4e0000
done

GPIO

dpavlin@parallella:~/parallella-utils$ make gpiotest
gcc -o gpiotest gpiotest.c para_morse.c para_gpio.c -Wall -lrt

dpavlin@parallella:~/parallella-utils$ sudo ./gpiotest 
[sudo] password for dpavlin: 
GPIOTEST - Basic test of para_gpio

Initializing...
Success, pausing 5 seconds
Setting direction (forces output to 0!)...
Pausing 5 seconds
Reading multiple times
> 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 
Toggling pin 100000 times...
Took 1.432 seconds, 69818 updates/sec
Write / Read test, 100000 times...
Took 2.618 seconds, 76384 ops/sec
Setting output to 0
Verifying level... OK
Disabling output and monitoring input
Gave up waiting for the input to transition
Setting output to 1
Verifying level... OK
Disabling output and monitoring input
Input flipped after 0 cycles
Closing

FPGA

• Zynq documentation http://forums.parallella.org/viewtopic.php?f=51&t=1114

• Command-line programming http://forums.parallella.org/viewtopic.php?uid=590&f=51&t=1161&start=0

sudo mknod /dev/xdevcfg c 250 0
sudo -s
cat parallella.bit.bin > /dev/xdevcfg
exit

# programming time
cat /sys/devices/amba.1/f8007000.devcfg/prog_done

• Routing additional Zynq hard block I/O to PEC_FPGA http://forums.parallella.org/viewtopic.php?f=51&t=1154

• Source http://forums.parallella.org/viewtopic.php?uid=590&f=10&t=1069&start=0

• does the board contains thermal sensor(s) ? http://forums.parallella.org/viewtopic.php?f=23&t=930

/sys/bus/iio/devices/iio:device0/in_temp0_raw
/sys/bus/iio/devices/iio:device0/in_temp0_offset
/sys/bus/iio/devices/iio:device0/in_temp0_scale

T = ((in_temp0_raw + in_temp0_offset) * in_temp0_scale) / 1000)

script:

#!/bin/bash
raw=`cat /sys/bus/iio/devices/iio:device0/in_temp0_raw`
offset=`cat /sys/bus/iio/devices/iio:device0/in_temp0_offset`
scale=`cat /sys/bus/iio/devices/iio:device0/in_temp0_scale`

c_temp=`echo "scale=1;(($raw + $offset) * $scale) / 1000" | bc`
f_temp=`echo "scale=1;(($c_temp * 9) / 5) + 32" | bc`

echo
echo "Zynq Temp: $c_temp C / $f_temp F"
echo

Modified to display temperature in loop: temp.sh

headless stream

http://forums.parallella.org/viewtopic.php?f=48&t=1213

removed html traces from device tree

serial

• serial port http://forums.parallella.org/viewtopic.php?f=10&t=845

hardware

• Eagle library available http://forums.parallella.org/viewtopic.php?f=11&t=559
• KiCad daughter card template http://forums.parallella.org/viewtopic.php?f=11&t=1204 https://github.com/TiZed/Parallella_Template
• Parallella Case With 12V Fan http://www.thingiverse.com/thing:273701
• Parallella clip case https://github.com/kyllikki/designs/tree/master/Parallella
• ARM JTAG http://forums.parallella.org/viewtopic.php?f=10&t=167
• uboot network problems http://forums.parallella.org/viewtopic.php?f=48&t=1667&p=10416&hilit=devicetree#p10416

software

device tree

sudo apt-get install device-tree-compiler

# Compiling a device tree blob from source (.dts-->.dtb):
dtc -I dts -O dtb -o devicetree.dtb zynq-parallella1-headless.dts

# To get source from compiled device tree (.dtb-->.dts):
dtc -I dtb -O dts -o devicetree.dts devicetree.dtb

/dev

see how to modify devicetree at http://elinux.org/Parallella_Speed_Up_Ubuntu

Debian

http://elinux.org/Parallella_Debian

Epiphany

• cgminer https://github.com/LordRafa/cgminer

Working configuration

(for me at least :-)

/etc/ppp/peers/huawei-e220

dpavlin@llin:~$ cat /etc/ppp/peers/huawei-e220
# /etc/ppp/peers/huawei-e220 pppd script
# pppd call huawei-e220

debug
kdebug 3
nodetach
#persist

/dev/ttyUSB0
460800
idle 7200
noipdefault
defaultroute
usepeerdns

connect "/usr/sbin/chat -vf /etc/chatscripts/huawei-e220.chat"  

# avoid compression:
noccp
#nobsdcomp
novj
# t-mobile specific?
#ipcp-restart 8
#ipcp-max-configure 50
#ipcp-accept-local
#ipcp-accept-remote
#ipcp-max-failure 20

lcp-echo-failure 0
lcp-echo-interval 0

modem
crtscts

require-pap
refuse-chap

show-password
user YOUR_USERNAME@DOMAIN
password YOUR_PASSWORD

passive

/etc/chatscripts/huawei-e220.chat

dpavlin@llin:~$ cat /etc/chatscripts/huawei-e220.chat
TIMEOUT 3
ABORT BUSY 
ABORT 'NO CARRIER' 
ABORT VOICE 
ABORT 'NO DIALTONE' 
ABORT 'NO DIAL TONE' 
ABORT 'NO ANSWER' 
ABORT DELAYED
"" ATZ
OK ATQ0V1E1S0=0&C1&D2
OK AT+COPS?
OK AT+CGDCONT=1,"ip","carnet.vip.hr"
OK ATDT*99#
CONNECT ""

Links

• http://wwwu.uni-klu.ac.at/agebhard/HuaweiE220/
• another tutorial: http://ske.sourceforge.net/html/projects/huawei/huawei_tre.html
• interesting compilation of configuration files and stat tool: http://oozie.fm.interia.pl/pro/huawei-e220/
• Ubuntu, GUI: http://sistemac-portal.carnet.hr/node/335
• source: http://www.kanoistika.sk/bobovsky/archiv/umts/
• ObWikipedia: http://en.wikipedia.org/wiki/Huawei_E220

Turn off pin

$ cu -l /dev/ttyUSB0
at+clck="sc",0,"PIN!"
OK

libusb driver program

Compilation

Get source from http://www.kanoistika.sk/bobovsky/archiv/umts/huaweiAktBbo.c

cc     huaweiAktBbo.c   -o huaweiAktBbo -lusb

hal problem

If you modem is restarting like mad, it's problem with hal.

• http://wiki.archlinux.org/index.php/Huawei_E220

CD image update

Since device can emulate USB CDROM, I will try to update image on it to provide bios update image instead of useless (to me) Windows drivers.

dmesg

dpavlin@t61p:~$ uname -a
Linux t61p 2.6.30-1-686-bigmem #1 SMP Mon Aug 3 17:32:39 UTC 2009 i686 GNU/Linux
dpavlin@t61p:~$ dmesg
[ 3405.568120] usb 4-1: new full speed USB device using uhci_hcd and address 2
[ 3405.726188] usb 4-1: New USB device found, idVendor=12d1, idProduct=1003
[ 3405.726196] usb 4-1: New USB device strings: Mfr=1, Product=2, SerialNumber=0
[ 3405.726203] usb 4-1: Product: HUAWEI Mobile
[ 3405.726208] usb 4-1: Manufacturer: HUAWEI Technologies
[ 3405.726384] usb 4-1: configuration #1 chosen from 1 choice
[ 3405.792930] Initializing USB Mass Storage driver...
[ 3405.796244] usb-storage: probe of 4-1:1.0 failed with error -1
[ 3405.796289] usbcore: registered new interface driver usb-storage[ 3405.796296] USB Mass Storage support registered.
[ 3405.952077] usb 4-1: USB disconnect, address 2
[ 3407.432107] usb 4-1: new full speed USB device using uhci_hcd and address 3
[ 3407.590171] usb 4-1: New USB device found, idVendor=12d1, idProduct=1003
[ 3407.590180] usb 4-1: New USB device strings: Mfr=1, Product=2, SerialNumber=0
[ 3407.590187] usb 4-1: Product: HUAWEI Mobile
[ 3407.590192] usb 4-1: Manufacturer: HUAWEI Technologies
[ 3407.590373] usb 4-1: configuration #1 chosen from 1 choice
[ 3407.603280] usb-storage: probe of 4-1:1.2 failed with error -1[ 3407.650695] usbcore: registered new interface driver usbserial
[ 3407.650725] USB Serial support registered for generic
[ 3407.650819] usbcore: registered new interface driver usbserial_generic
[ 3407.650824] usbserial: USB Serial Driver core
[ 3407.663176] USB Serial support registered for GSM modem (1-port)
[ 3407.663293] option 4-1:1.0: GSM modem (1-port) converter detected
[ 3407.663458] usb 4-1: GSM modem (1-port) converter now attached to ttyUSB0
[ 3407.663478] option 4-1:1.1: GSM modem (1-port) converter detected
[ 3407.663576] usb 4-1: GSM modem (1-port) converter now attached to ttyUSB1
[ 3407.663609] usbcore: registered new interface driver option
[ 3407.663614] option: v0.7.2:USB Driver for GSM modems

power

root@t61p:~/t61p/usb_modeswitch/usb_modeswitch-1.0.2# lsusb
Bus 004 Device 003: ID 12d1:1003 Huawei Technologies Co., Ltd. E220 HSDPA Modem / E270 HSDPA/HSUPA Modem
Bus 004 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub

root@t61p:~/t61p/usb_modeswitch/usb_modeswitch-1.0.2# echo suspend >  /sys/bus/usb/devices/4-1/power/level
root@t61p:~/t61p/usb_modeswitch/usb_modeswitch-1.0.2# dmesg

links

• http://www.draisberghof.de/usb_modeswitch/

E171

Found modem         : E171
Model               : Huawei E171
IMEI                : 354807044078459
Serial NR.          : 4BA5TA10B2706637
Firmware            : 11.126.85.01.143
Compile date / time : Jun 21 2010 20:27:27
Dashboard version   : UTPS11.300.05.15.143_MAC11.301.09.01.143
Chipset             : Qualcomm MSM6290
Voice feature       : disabled
SIM Lock status     : unlocked
Wrong codes entered : 0 (unlock attempts left : 10)

E1752

Found modem         : E1752
Model               : Huawei E1752
IMEI                : 353145034241600
Serial NR.          : O47NAA19A1600110
Firmware            : 11.126.13.00.00
Compile date / time : Jul 05 2010 10:16:42
Dashboard version   : UTPS11.300.05.22.56_MAC11.300.08.19.56
Chipset             : Qualcomm MSM6290
Voice feature       : disabled
SIM Lock status     : unlocked
Wrong codes entered : 0 (unlock attempts left : 10)

AT^CVOICE=0 Enable Voice
AT^CVOICE=1 Disable Voice
AT^CVOICE=? Check status
=========
AT^U2DIAG=0 – switch the device in modem mode only
AT^U2DIAG=1 – device in modem mode + CD-ROM
AT^U2DIAG=255 – modem mode + CD-ROM + Card Reader
AT^U2DIAG=256 – modem mode + Card Reader

SQL snippets for PostgreSQL

Size of transaction log

select sum((pg_stat_file('pg_xlog/' || file)).size)
from pg_ls_dir('pg_xlog') as file
where file ~ '^[0-9A-F]';

Size of tables in current database

select relname,pg_relation_size(oid) as size
from pg_class
where relkind = 'r' and relname not like 'pg_%' order by size desc;

Number of rows in all tables

http://stackoverflow.com/questions/2596670/how-do-you-find-the-row-count-for-all-your-tables-in-postgres

SELECT 
  nspname AS schemaname,relname,reltuples::int
FROM pg_class C
LEFT JOIN pg_namespace N ON (N.oid = C.relnamespace)
WHERE 
  nspname NOT IN ('pg_catalog', 'information_schema') AND
  relkind='r' 
ORDER BY reltuples DESC;

Script to display database size

Script: pg_size

all databases

$ pg_size

all databases, grouped by user

$ pg_size -u

Active locks

select * from pg_locks where pid not in (select procpid from pg_stat_activity);

Graphics

Taken from http://lca2007.linux.org.au/att_data/Miniconfs(2f)PostgreSQL/attachments/advanced.pdf

# CREATE AGGREGATE array_accum (anyelement)
(
    sfunc = array_append,
    stype = anyarray,
    initcond = '{}'
);

# select d, count(*),
       array_to_string(array_accum('+'::text),'') as graph
       from hits group by 1 order by 1 asc;
     d       | count | graph
------------+-------+----------
 2007-01-13 |      8 | ++++++++
 2007-01-14 |      7 | +++++++
 2007-01-15 |      4 | ++++
(3 rows)

Current query activity

select datid,datname,procpid,current_query,query_start from pg_stat_activity where current_query not like '<IDLE>' ;

To actually see queries you also have to edit postgresql.conf to include:

stats_command_string = yes

Quickly convert database table to SQLite

pg_dump --inserts --no-owner --no-privileges --table hr_edu_orgs a3c | grep -v ^SET | sqlite3 var/a3c.sqlite

There are many on-line resources about RFID. However, most of them are not well suited for beginners. So, if you just got RFID reader (3M in this case) and want high-level overview of what it is and what it can do, you are out of luck.

Until now, that is. This guide should help you decide if RFID is right thing for your library and when you make choice to implement it, how to do it.

This document will describe my experience with 3M 810 reader using RFID 501: RFID Standards for Libraries RFID_501.pdf

RFID tag

Best way to think about RFID tags is like contact-less readable barcode.

Our particular tags come in two form: RFID stickers (to be placed on books) and plastic credit-card like cards (we use them for patrons).

Have in mind that established practice is to have different RFID systems for books and patrons (we are use same one). When we asked about using same system for books and patrons, we got reply: "we don't have experience with it".
In practice, we have problem with 3M selfcheck software in some special conditions where book reader have patron card in range it gets network connection error with SIP server.
Since normal configuration of selfcheck stations is to have two different systems for patrons and books this problem doesn't show up for other users.

Each tag has unique serial number (SID) assigned by manufacturer and used in RFID collision detection protocol. It looks like hexadecimal number starting with letter E0 like this: E00401003123AA26.

It's best to think of SID as unique identifier of physical tag.
Your information system will have it's own ID (barcode?) for some item.
There are cases in which you might want to change physical tag sticker on book because it's damaged. In that case, you will change SID of that item, but not barcode (which is data programmed on tag itself).

Our initial idea was to use data programmed on chip for everything, and just ignore SIDs, but we found out that there is class of RFID devices which can read ONLY SID from chip (in our case it's photocopying system).
3M software does record SIDs to log file when programming chips, but that's all. It essential ignores it for all practical intends and purposes.

Chips have 7 blocks of user data on it, each block with 4 bytes which enables us to store 28 bytes of user specified data on each tag.

blank tag

3M Manufacturing Blank

Easiest case is blank tag, in which all data on chip is 0x55

0	55 55 55 55		blank tag
1	55 55 55 55
2	55 55 55 55
3	55 55 55 55
4	55 55 55 55
5	55 55 55 55
6	00 00 00 00 

Generic blank

Generic blank seems to erase only first three blocks with zeros:

00	00 00 00 00
01	00 00 00 00
02	00 00 00 00

while rest of tag will be unchanged including rest of data on tag

programmed tag

Tags programmed with 3M software have following data layout on them:

0	04 is 00 tt		i [4 bit] = number of item in set	[1 .. i .. s]
					s [4 bit] = total items in set
					tt [8 bit] = item type

1	dd dd dd dd		dd [16 bytes] = barcode data
2	dd dd dd dd
3	dd dd dd dd
4	dd dd dd dd

5	bb bl ll ll		b [12 bit] = branch [unsigned]
				l [20 bit] = library [unsigned]
6	cc cc cc cc		c [32 bit] = custom signed integer

This basically means that your barcode or identifier of item or patron can have up to 16 characters (by default numeric, but you can extend that to handle alphanumeric and special character if you need that) and three integer values: branch 0 .. 4095, library 0 .. 1048575 and custom data -2147483648 .. 2147483647.

You might want to use those values to uniquely identify your library and branch so that your RFID tags in books won't collide with other libraries.
If you leave decision just to providers of equipment, you might end up with 300000 tags which have plain and simple 0 in those fields. Guess which value will have tags of next library which that provider will have? My guess would be 0 also.

Writing correct numbers in that fields is not enough. If you want to use 3M software, you will also have to setup it to ignore all other tags which doesn't match your library and branch.

security

There is also single byte called AFI or security which can be changed without accessing content of chip. This byte is also readable by more primitive RFID devices like doors to check if book have been checked out from library.

3M is using 0xD7 (215) value for secured items (door will beep) and 0xDA (218) as unsecured. It seems that all other values are ignored.
(I would guess that other manufacturers are using different values)

As I mentioned before, since we don't have any special values in branch, library or custom field, we have situations in which patron cards get secured when patron walks by checkout counter and 3M software is left in checkout mode.
This triggers door to ring when patron passes which is not ideal.

disable tag

3M software have option to disable tags. Initial examination showed that it's simply programming of tag with following content:

00	ff 00 00 00
01	00 00 00 00
02	00 00 00 00
03	00 00 00 00
04	00 00 00 00
05	00 00 00 00
06	00 00 00 00

and security set to d7 (this might be value from tag before disabling it, I'll have to re-check this)

While 3M software will ignore tags programmed with this content, there is not permanent disabling of tag since it can be programmed using other software.

RFID reader device

Reader consists of several part:

• black pad - reader antenna
• reader - small box with micro controller and usb port
• software

Reader is recognized as USB serial device with it's own protocol on serial port. We are mostly interested in it's protocol and our ability to use reader and tags with our custom software.

At first, I assumed that protocol with RFID readers is some kind of standard.
After extensive search on Internet I wasn't able to find any documentation about this particular protocol (I even tried to compare it with existing open source implementations just to be sure).

So, only solution was to do clean-room reverse engineering, and using that technique I developed perl module which can talk with RFID reader which is available at http://svn.rot13.org/index.cgi/RFID

After initial reverse engineering of protocol I rewrote support for 3M and CPR reader which is available at https://github.com/dpavlin/Biblio-RFID

Related blog posts

More information

If this was too geeky for you here is some additional materials:

• RFID for libraries FAQ
• RFID - overview of protocols, librfid implementation and passive sniffing

ISO standard

• ISO 15962.2004 - object identifier structure
• ISO 15693 - RFID (layer 2)
• ISO 18000 Part 3 Mode 1 - 13.56MHz
• ISO/IEC JTC1/SC17/WG8

ISO/IEC 14443, Proximity cards

The Standard series ISO/IEC 14443 consists of 4 parts, which are:

ISO/IEC 14443-1 Physical characteristics

ISO/IEC 14443-2 Radio frequency power and signal interface

ISO/IEC 14443-2/AMD2 Amendment 2: Bit rates of fc/64, fc/32 and fc/16

ISO/IEC 14443-3 Initialization and anticollision

ISO/IEC 14443-3 Amendment 1: Bit rates of fc/64, fc/32 and fc/16

ISO/IEC 14443-4 Transmission protocol

ISO/IEC 15693, Vicinity cards

ISO/IEC 15693-1 Physical characteristics

ISO/IEC 15693-2 Air interface and initialisation

ISO/IEC 15693-3 Anticollision and transmission protocol

ISO/IEC 10373-6, 10373-7, Test methods for the contactless integrated circuit(s) cards

ISO/IEC 10373-6 Proximity cards

ISO/IEC 10373-6/AMD1 Amendment 1: Additional PICC test methods

ISO/IEC 10373-6/AMD2 Amendment 2: Improved RF test methods

ISO/IEC 10373-7 Vicinity cards

ISO/IEC 10536, Close-coupled cards

ISO/IEC 10536-1 Physical characteristics

Welcome to scratch pad of usefull shell snippets

remove all php4 packages with apt-get

dpkg -l | grep php4 | awk '{ print $2 }' | xargs apt-get remove -y

use socat as simple serial program

socat - /dev/ttyS0,b9600,echo=0,raw

list first ten files from srchives

ls *.tar.gz *.tgz | xargs -i sh -c "ls -hs '{}' ; echo ; tar tvfz '{}' | head -10 ; echo"

ls *.tar.bz2 *.tbz | xargs -i sh -c "ls -hs '{}' ; echo ; tar tvfj '{}' | head -10 ; echo"

Yes, it sucks that tar doesn't know how to handle compression formats automatically...

tail recent logs and put output in file

Useful to take a quick view into last lines, reload page (and capture newly created errors/infos) and store that in single file for easy review (in vim :-)

tail -f /var/log/apache2/*.log | tee ~/log

port forwarder using socat

socat -d -d TCP4-LISTEN:1280,fork TCP4:192.168.212.5:80

remove all filesystem cache

echo 3 > /proc/sys/vm/drop_caches

D-Link DNS-323 network attached storage

Overview

dlink-DFDADE:~# uname -a
Linux dlink-DFDADE 2.6.12.6-arm1 #30 Mon Aug 18 14:19:14 CST 2008 armv5tejl GNU/Linux

dlink-DFDADE:~# free
            total      used      free    shared   buffers    cached
Mem:        61904     54808      7096         0     11824     30592
-/+ buffers/cache:     12392     49512
Swap:     1060208         0   1060208

dlink-DFDADE:~# cat /proc/cpuinfo 
Processor      : ARM926EJ-Sid(wb) rev 0 (v5l)
BogoMIPS       : 331.77
Features       : swp half thumb fastmult edsp java 
CPU implementer : 0x41
CPU architecture: 5TEJ
CPU variant    : 0x0
CPU part       : 0x926
CPU revision   : 0
Cache type     : write-back
Cache clean    : cp15 c7 ops
Cache lockdown : format C
Cache format   : Harvard
I size         : 32768
I assoc        : 1
I line length  : 32
I sets         : 1024
D size         : 32768
D assoc        : 1
D line length  : 32
D sets         : 1024

Hardware       : MV-88fxx81
Revision       : 0000
Serial         : 0000000000000000

RAID

Personalities : [linear] [raid0] [raid1] 
md0 : active raid0 sda2[0] sdb2[1]
      2925532672 blocks 64k chunks
      
unused devices: <none>
dlink-DFDADE:~# hdparm -tT /dev/sda /dev/sdb /dev/md0

/dev/sda:
 Timing cached reads:   210 MB in  2.01 seconds = 104.48 MB/sec
 Timing buffered disk reads:   72 MB in  3.00 seconds =  24.00 MB/sec

/dev/sdb:
 Timing cached reads:   212 MB in  2.01 seconds = 105.47 MB/sec
 Timing buffered disk reads:  104 MB in  3.03 seconds =  34.32 MB/sec

/dev/md0:
 Timing cached reads:   208 MB in  2.01 seconds = 103.48 MB/sec
 BLKGETSIZE failed: File too large

dlink-DFDADE:~# dd_rescue /dev/sda /dev/null
dd_rescue: (info): ipos:   1044480.0k, opos:   1044480.0k, xferd:   1044480.0k
                   errs:      0, errxfer:         0.0k, succxfer:   1044480.0k
             +curr.rate:    37169kB/s, avg.rate:    32123kB/s, avg.load: 60.7%

dlink-DFDADE:~# dd_rescue /dev/md0 /dev/null
dd_rescue: (info): ipos:    729536.0k, opos:    729536.0k, xferd:    729536.0k
                   errs:      0, errxfer:         0.0k, succxfer:    729536.0k
             +curr.rate:    37176kB/s, avg.rate:    37502kB/s, avg.load: 76.5%

Debian

• http://www.cyrius.com/debian/orion/d-link/dns-323/install.html

2.6.26-2-orion5x

raid0

Linux dlink-DFDADE 2.6.26-2-orion5x #1 Thu Aug 20 05:04:03 UTC 2009 armv5tel GNU/Linux
Personalities : [raid1] [raid6] [raid5] [raid4] [raid0] 
md1 : active raid0 dm-1[1] dm-0[0]
      2097024 blocks 64k chunks
      
md0 : active raid1 sda4[0] sdb4[1]
      513984 blocks [2/2] [UU]
      
unused devices: <none>

/dev/sda:
 Timing cached reads:   212 MB in  2.01 seconds = 105.39 MB/sec
 Timing buffered disk reads:   72 MB in  3.05 seconds =  23.58 MB/sec

/dev/sdb:
 Timing cached reads:   218 MB in  2.01 seconds = 108.26 MB/sec
 Timing buffered disk reads:   88 MB in  3.00 seconds =  29.33 MB/sec

/dev/md1:
 Timing cached reads:   208 MB in  2.01 seconds = 103.46 MB/sec
 Timing buffered disk reads:   60 MB in  3.07 seconds =  19.57 MB/sec

Linux dlink-DFDADE 2.6.26-2-orion5x #1 Thu Aug 20 05:04:03 UTC 2009 armv5tel GNU/Linux
Personalities : [raid1] [raid6] [raid5] [raid4] [raid0]
md1 : active (auto-read-only) raid1 dm-1[1] dm-0[0]
      1048512 blocks [2/2] [UU]
        resync=PENDING

md0 : active raid1 sda4[0] sdb4[1]
      513984 blocks [2/2] [UU]

unused devices: <none>

/dev/sda:
 Timing cached reads:   220 MB in  2.00 seconds = 109.73 MB/sec
 Timing buffered disk reads:   82 MB in  3.01 seconds =  27.22 MB/sec

/dev/sdb:
 Timing cached reads:   216 MB in  2.02 seconds = 107.12 MB/sec
 Timing buffered disk reads:   90 MB in  3.09 seconds =  29.14 MB/sec

/dev/md1:
 Timing cached reads:   208 MB in  2.02 seconds = 103.19 MB/sec
 Timing buffered disk reads:   52 MB in  3.05 seconds =  17.06 MB/sec

2.6.30

http://www.cyrius.com/journal/debian/orion/d-link/dns-323/dns-323-fan-control

dlink-DFDADE:~/mdadm# ./test.sh 
+ ./remove.md1.sh
+ mdadm --manage --stop /dev/md1
mdadm: error opening /dev/md1: No such file or directory
+ mdadm --manage --remove /dev/md1
mdadm: error opening /dev/md1: No such file or directory
+ yes
+ mdadm --create --verbose /dev/md1 --level=0 --raid-devices=2 --force /dev/vga/raid.a /dev/vgb/raid.b
mdadm: chunk size defaults to 64K
mdadm: /dev/vga/raid.a appears to be part of a raid array:
    level=raid1 devices=2 ctime=Sat Sep  5 14:54:52 2009
mdadm: /dev/vgb/raid.b appears to be part of a raid array:
    level=raid1 devices=2 ctime=Sat Sep  5 14:54:52 2009
Continue creating array? mdadm: array /dev/md1 started.
+ ./hdparm-test.sh
+ test -d out
+ uname -a
+ cat /proc/mdstat
+ hdparm -tT /dev/sda /dev/sdb /dev/md1
++ date +%Y%m%d_%H%M%S
+ tee out/20090905_151032
Linux dlink-DFDADE 2.6.30-1-orion5x #1 Tue Aug 18 04:19:30 UTC 2009 armv5tel GNU/Linux
Personalities : [raid1] [raid0] 
md1 : active raid0 dm-0[1] dm-1[0]
      2097024 blocks 64k chunks
      
md0 : active raid1 sda4[0] sdb4[1]
      513984 blocks [2/2] [UU]
      
unused devices: <none>

/dev/sda:
 Timing cached reads:   246 MB in  2.01 seconds = 122.14 MB/sec
 Timing buffered disk reads:  132 MB in  3.00 seconds =  43.93 MB/sec

/dev/sdb:
 Timing cached reads:   242 MB in  2.01 seconds = 120.27 MB/sec
 Timing buffered disk reads:  138 MB in  3.01 seconds =  45.87 MB/sec

/dev/md1:
 Timing cached reads:   234 MB in  2.01 seconds = 116.15 MB/sec
 Timing buffered disk reads:  130 MB in  3.03 seconds =  42.85 MB/sec

+ ./remove.md1.sh
+ mdadm --manage --stop /dev/md1
mdadm: stopped /dev/md1
+ mdadm --manage --remove /dev/md1
+ yes
+ mdadm --create --verbose /dev/md1 --level=1 --raid-devices=2 --spare-devices=0 --force /dev/vga/raid.a /dev/vgb/raid.b
mdadm: /dev/vga/raid.a appears to be part of a raid array:
    level=raid0 devices=2 ctime=Sat Sep  5 15:10:31 2009
mdadm: /dev/vgb/raid.b appears to be part of a raid array:
    level=raid0 devices=2 ctime=Sat Sep  5 15:10:31 2009
mdadm: size set to 1048512K
Continue creating array? mdadm: array /dev/md1 started.
+ ./hdparm-test.sh
+ test -d out
+ uname -a
++ date +%Y%m%d_%H%M%S
+ cat /proc/mdstat
+ hdparm -tT /dev/sda /dev/sdb /dev/md1
+ tee out/20090905_151114
Linux dlink-DFDADE 2.6.30-1-orion5x #1 Tue Aug 18 04:19:30 UTC 2009 armv5tel GNU/Linux
Personalities : [raid1] [raid0] 
md1 : active (auto-read-only) raid1 dm-0[1] dm-1[0]
      1048512 blocks [2/2] [UU]
        resync=PENDING
      
md0 : active raid1 sda4[0] sdb4[1]
      513984 blocks [2/2] [UU]
      
unused devices: <none>

/dev/sda:
 Timing cached reads:   116 MB in  2.01 seconds =  57.84 MB/sec
 Timing buffered disk reads:  118 MB in  3.00 seconds =  39.32 MB/sec

/dev/sdb:
 Timing cached reads:   114 MB in  2.01 seconds =  56.79 MB/sec
 Timing buffered disk reads:  140 MB in  3.03 seconds =  46.27 MB/sec

/dev/md1:
 Timing cached reads:   234 MB in  2.02 seconds = 116.07 MB/sec
 Timing buffered disk reads:  110 MB in  3.01 seconds =  36.49 MB/sec

updates

Here are some quick notes about installation on clean Debian etch machine

This page is aimed towards system administrators, if you want information how to customize design of page take a look at SocialText CSS design (which is included on bottom of this page).

Helper packages

sudo apt-get install wv xpdf-utils unrtf unzip links
sudo cpan MP3::Tag

Install source

svn co https://repo.socialtext.net:8999/svn/socialtext/trunk socialtext
cd socialtext/nlw

# checkout lastest Socialtext::Resting client
svn export --force https://repo.socialtext.net:8999/svn/socialtext-clients/trunk/rest/perl/Socialtext-Resting/lib/Socialtext/ lib/Socialtext/

./configure --socialtext-open 1 --apache-proxy 0 --force-ssl-login 1
make
sudo bash
make install
update-rc.d -f apache-perl remove
update-rc.d st-apache defaults
update-rc.d ceqlotron defaults
/etc/init.d/st-apache start
/etc/init.d/ceqlotron start

Update existing checkout

cd socaialtext
svn update
cd nlw
./configure --socialtext-open 1 --apache-proxy 0 --force-ssl-login 1
make
make test
sudo make install
sudo /etc/init.d/st-apache stop
sudo /etc/init.d/ceqlotron restart
sudo /etc/init.d/st-apache start

Add new admin user

sudo -u www-data st-admin create-user --email dpavlin@rot13.org --password wikidream \
  --first-name Dobrica --last-name Pavlinusic
sudo -u www-data st-admin add-workspace-admin --email dpavlin@rot13.org

Configuring mailer

postfix

Install and configure postfix for mail delivery

apt-get install postfix
vi /etc/postfix/main

Important lines are following:

alias_maps = hash:/etc/aliases, hash:/etc/aliases.deliver
alias_database = hash:/etc/aliases, hash:/etc/aliases.deliver

They add /etc/aliases.deliver (which SocialText generates with names of wiki workspaces) to configuration.

Next, we need to create /etc/aliases.deliver.db which is binary version of database needed for postfix. It's extremly important that aliases.deliver file is owned by www-data because postfix picks user which will do e-mail delivery based on owner of aliases file. If your owner of /var/www/socialtext is some other user than www-data you will have to modify following command appropriately:

# make www-data owner of file
chown www-data:root /etc/aliases.deliver
# allow root group to write in /etc/
chmod g+w /etc/
# create /etc/aliases.deliver.db
sudo postalias /etc/aliases.deliver

Now restart postfix as root user

/etc/init.d/postfix restart

crontab

Add following to /etc/crontab to refresh aliases database for postfix regularly:

*/10 *  * * *   root    postalias /etc/aliases.deliver

Handling e-mail errors

sudo tail /var/log/mail.log | grep st-admin

dsn=5.3.0, status=bounced (Command died with status 13: "/usr/local/bin/st-admin deliver-email --workspace bio". Command output: open nlw-stats.log: Permission denied at /usr/local/share/perl/5.8.8/Socialtext/Statistics.pm line 103. END failed--call queue aborted. )

This problem is reported on socialtext-devel list, but no response so far.

# sudo vi +24 /usr/local/share/perl/5.8.8/Socialtext/Statistics.pm
Readonly my $LOG_NAME => 'nlw-stats.log';
Readonly my $LOG_PATH => $ENV{APACHE_LOG_DIR}
    ? "$ENV{APACHE_LOG_DIR}/$LOG_NAME"       
    : "/tmp/$LOG_NAME";

Customize template

Remove "Did you know?" box from dashboard

sudo vi /usr/share/nlw/template/view/homepage

Or from your socialtext/nlw checkout (so that make install will install correct one):

vi share/template/view/homepage

Fix RSS feed errors

If you are using {fetchrss http://blog.rot13.org/index.xml} and having problems with RSS feeds, try to re-installing two perl modules:

sudo cpan XML::RSS XML::Atom

Change e-mail address of workspace

This configuration change will feed responses to e-mails generated by wiki back into wiki. You might not want to do that! :-)

st-admin set-workspace-config --workspace rot13 \
  email_notification_from_address '"random unstructured stuff" <rot13@wiki.rot13.org>'

Access simple version of page

Every SocialText wiki has a lite version available at http://wiki.example.com/lite/page/:workspace/

For this wiki it's http://saturn.ffzg.hr/lite/page/rot13

Configure sane resource usage

Since SocialText wiki is designed to run on dedicated box, it's resource requiements are not for faint-hearted users. However, if you are installing it on machine with constraint hardware (or virtual machine) you might want to decrease number of apache-perl server which are created on startup and kept running. Each apache child takes about 80Mb of RAM, so lowering number of children will have beneficial effect on your memory usage.

If you open /etc/apache-perl/st-httpd.conf you will see comment that describes command used for generating this configuration. So, let's edit command limits, re-create config and restart apache:

vi +485 dev-bin/gen-config
dev-bin/gen-config --sitewide --apache-proxy=0 --hostname=saturn.ffzg.hr
/etc/init.d/st-apache reload

Convert RSS feed URLs to absolute

If you need to fetch RSS feeds from remote URLs and those providers create only relative URLs in feeds (like svnweb does) that this patch might help you: FetchRSSPlugin.pm.diff

It does dirty magic to prepend hostname before URL in feeds making them work from Socialtext wiki which is on different host.

Problems with 8-bit UTF-8 encoded characters in URLs

It seems that trunk of socialtext

Repository Root: https://repo.socialtext.net:8999/svn/socialtext
Repository UUID: 593ce9bc-2028-0410-bb98-c566b643a948
Revision: 6127
Node Kind: directory
Schedule: normal
Last Changed Author: jjp
Last Changed Rev: 6028
Last Changed Date: 2007-12-10 19:38:21 +0100 (Mon, 10 Dec 2007)

has problems with CGI.pm 3.31. As a temporary work-around use backpan.sh to download CGI.pm 3.29 which is known to work.

Enable html wafl in workspace

This enables .html markup which enables embedded html into wiki page

st-admin set-workspace-config --workspace razmjenavjestina allows_html_wafl 1

Upgrade to 2.2

There are various problems with this upgrade.

Socialtext::Resting

For a start, you will have to install Socialtext::Resting clinet in the middle of make install because import of help workspace will fail.

You can get it from https://repo.socialtext.net:8999/svn/socialtext-clients/trunk/rest/perl/Socialtext-Resting

Missing directory

mkdir /etc/apache-perl/socialtext-conf.d

Missing skins

http://code.google.com/p/socialtext-skins/wiki/SkinningIssues

Problem not yet solved.

cd /usr/share/nlw/css/
cp -r st rot13
st-admin set-workspace-config --workspace rot13 skin_name rot13
vi rot13/screen.css

cd /usr/share
svn co file:///data/st-custom/share nlw.svn
cd nlw.svn
find . | cpio -pdu ../nlw/
cd - && rm -Rf nlw.svn
cd nlw
svn info

Quick guide based on stunnel for mysql - server and client to add https to existing http server on demand.

Installation

apt-get install stunnel4

Create certificate

cd /etc/stunnel
openssl req -x509 -nodes -days 365 -newkey rsa:1024 -keyout https.pem -out https.pem

Configuration file

/etc/stunnel/httpd.conf

cert = /etc/stunnel/https.pem
sslVersion = SSLv3
debug = 7

[https]
accept  = 443
connect = 80
TIMEOUTclose = 0

Start tunnel

stunnel4 /etc/stunnel/https.conf

Use vpnc instead of Cisco VPN client

Source

svn co http://svn.unix-ag.uni-kl.de/vpnc/trunk vpnc
cd vpnc
make
sudo make install

Support template

pcf2vpnc Support_template.pcf > /etc/vpnc/foobar.conf

Hash password security, eh?

Start connection

sudo /usr/local/sbin/vpnc --no-detach --debug 2 foobar

Fix routes

I didn't like that all my traffic was going over VPN

sudo route del default
route add default gw 192.168.1.1
route add -net 192.168.4.0 netmask 255.255.255.0 dev tun0