How to start connexion without installation, just out of the SVN copy.

Pre-caution

Branch 0.4.7 is un-tested, un-stable; some parts or modules do not function at all. It is because of major core changes in the Connexion engine. Please, do not use 0.4.7 for anything else than testing.

Download a connexion engine

svn co svn://radlinux.org/radlinux/branches/0.4.7

Install required python modules

Connexion requires python 2.4 or 2.5. For ALT Linux, packages list is:

python-base
python-module-Crypto
python-module-dns
python-module-pyasn1
python-module-pysnmp4
python-modules
python-modules-compiler
python-modules-ctypes
python-modules-email
python-modules-logging

Start with public key security

Generate DSA keys

For every instance in the cluster, there should be two DSA keys, private and public. To create keys, use cxkey.py:

$ cd shell
$ mkdir keys
$ cd keys
$ export PYTHONPATH="../lib/cxcore/:../lib/cxnet/:../lib/cxutil/"
$ ../../lib/cxnet/cxnet/cxkey.py -f inst1
$ ../../lib/cxnet/cxnet/cxkey.py -f inst2
$ ls -l
total 16
-rw-r--r-- 1 peet peet 2732 Nov 28 22:03 inst1.private
-rw-r--r-- 1 peet peet 2652 Nov 28 22:03 inst1.public
-rw-r--r-- 1 peet peet 2732 Nov 28 22:03 inst2.private
-rw-r--r-- 1 peet peet 2652 Nov 28 22:03 inst2.public

Launch an instance

$ cd shell
$ export PYTHONPATH="../lib/cxcore/:../lib/cxnet/:../lib/cxutil/"
$ ./connexion.py \
           -x \
           -w sh:../modules/ \
           -W base:../modules-basedict/ \
           -W ha:../modules-heartbeat/ \
           -N inst1 \
           -P keys/inst1.private \
           -K inst1:keys/inst1.public \
           -K inst2:keys/inst2.public

Parameters:

• -D — logging level
• -N — instance name, must be unique along the cluster (default: hostname)
• -c — communication port for cluster transport protocol (default: 40323)
• -m — interface address: which interface use to join multicast group (default: all but loopback)
• -P — private DSA key
• -K - mapping of public DSA key to instance name
• -x — run debug shell
• -w — unprivileged modules (<namespace>:<path>)
• -W — privileged modules (<namespace>:<path>)

Launch as many instances as you want to test, but do not forget to use unique names.

Note, that there should be public key mappings for every node in the cluster. An instance should know about own public key too.

Start with pre-shared symmetric key

Generate PSK

Shared key should be generated with cxkey.py too:

$ cd shell
$ export PYTHONPATH="../lib/cxcore/:../lib/cxnet/:../lib/cxutil/"
$ ../lib/cxnet/cxnet/cxkey.py -t psk+AES -b 256

Launch an instance

$ cd shell
$ export PYTHONPATH="../lib/cxcore/:../lib/cxnet/:../lib/cxutil/"
$ ./connexion.py \
           -x \
           -w sh:../modules/ \
           -W base:../modules-basedict/ \
           -W ha:../modules-heartbeat/ \
           -k cxkey.private

Parameter -k forces Connexion to use pre-shared key. With PSK, there should be no instance/key mappings.

Debug shell commands

Each instance launched with -x runs debug shell. It should be used only for core monitoring. Debug shell has no autocomplete and no readline support. Available (tested for 0.4.7) commands are listed in help output.

Quick testing

Assume we launched two instances of Connexion on different machines. Simple test can be done with console client:

$ cd interfaces/console
$ ./console.py -e "exec uptime"
 13:37:00 up  1:53,  1 user,  load average: 0.00, 0.00, 0.00
 13:35:58 up 105 days, 22:24,  5 users,  load average: 0.00, 0.02, 0.00
$ ./console.py -e "exec uname -a"
Linux debian 2.6.18-6-amd64 #1 SMP Wed Oct 15 10:07:11 UTC 2008 x86_64 GNU/Linux
Linux localhost.localdomain 2.6.25-std-def-alt6 #1 SMP Thu Jul 3 18:44:31 MSD 2008 x86_64 GNU/Linux

DNS listing in the debug shell (note RRSIG records):

inst1 # dns


_cx._udp.local.
        in      ptr     record[ptr,in,_cx._udp.local.,3600/3594.21791309,inst1._cx._udp.local.]
        in      rrsig   RRSIG: [ptr] _cx._udp.local.
        in      ptr     record[ptr,in,_cx._udp.local.,3600/3594.36257495,inst2._cx._udp.local.]
        in      rrsig   RRSIG: [ptr] _cx._udp.local.

inst1._cx._udp.local.
        in      srv     record[srv,in,inst1._cx._udp.local.,3600/3288.25403125,inst1._cx._udp.local.:40323]
        in      rrsig   RRSIG: [srv] inst1._cx._udp.local.
        in      txt     record[txt,in,inst1._cx._udp.local.,3600/3288.25430908,'']
        in      rrsig   RRSIG: [txt] inst1._cx._udp.local.
        in      a       192.168.0.1 (reachable)
        in      rrsig   RRSIG: [a] inst1._cx._udp.local.
        in      a       10.0.0.6 (reachable)
        in      rrsig   RRSIG: [a] inst1._cx._udp.local.

inst2._cx._udp.local.
        in      srv     record[srv,in,inst2._cx._udp.local.,3600/3277.9897998,inst2._cx._udp.local.:40324]
        in      rrsig   RRSIG: [srv] inst2._cx._udp.local.
        in      a       192.168.0.1 (reachable)
        in      rrsig   RRSIG: [a] inst2._cx._udp.local.
        in      a       10.0.0.6 (reachable)
        in      rrsig   RRSIG: [a] inst2._cx._udp.local.

inst1 #

Another sample:

bala # lookup bala._cx._udp.local.
address: 10.0.120.17:36001
properties:
        state:  running
        role:   _cx._udp. testbed at bala

bala # lookup test._cx._udp.local.
address: 10.0.120.17:9001
properties:
        state:  stale
        role:   _cx._udp. testbed at test

One host (test._cx._udp.local.) is marked as stale. Possibly, the network node is unreachable or an instance is dead.

Issues

In 0.4.7 events still do not work. So, one should not plug-in modules-events. Also, ap-cluster command does not work too (because of events). Do not work filters, branch calls or scripts. Does not work egress configuration (HTB, TBF, SFQ) and network filter (iptables).