Documentation in this arena being slim, I felt it would be helpful to write a short article on my configuration. FastCGI examples are generally documented in about two or three blog posts on the subject, all of which have become dated. For the basic ground work take a look at streamhacker.

Firstly, the fastcgi process is unaware whether you are using HTTP or HTTPS, another fastcgi_param must be passed to solve the following error:

searching for changes
ssl required

fastcgi_param   HTTPS           on;

If you are having the following error, and you have ensured that HTTPS is functioning and allow_push = username is specified in the repo’s .hg/hgrc:

searching for changes
27 changesets found
abort: authorization failed

Then the fastcgi process isn’t receiving the username from nginx. Ensure that the following lines are added to your nginx config:

fastcgi_param   AUTH_USER       $remote_user;
fastcgi_param   REMOTE_USER     $remote_user;

Nginx is also liable to spit out a:

413 Request Entity Too Large

on your first major commit. This is because the max_body_size is set low by default and is unacceptable for a mercurial application. Ensure to add and modify this line to your needs:

client_max_body_size    100M;

my entire nginx vhost.conf

server {
    listen      443;
    server_name hg.DOMAIN.COM;

    ssl                  on;
    ssl_certificate      server.crt;
    ssl_certificate_key  server.key;
    ssl_session_timeout  5m;
    ssl_protocols  SSLv2 SSLv3 TLSv1;
    ssl_ciphers  ALL:!ADH:!EXPORT56:RC4+RSA:+HIGH:+MEDIUM:+LOW:+SSLv2:+EXP;
    ssl_prefer_server_ciphers   on;

        # Increase transfer size to accommodate large pushes
        client_max_body_size    100M;

        location / {
                auth_basic "Restricted Access";
                auth_basic_user_file /repo/sites/hg.DOMAIN.COM/mercurial_passwd;
                fastcgi_pass    127.0.0.1:9001;
                fastcgi_param   PATH_INFO       $fastcgi_script_name;
                fastcgi_param   AUTH_USER       $remote_user;
                fastcgi_param   REMOTE_USER     $remote_user;
                fastcgi_param   QUERY_STRING    $query_string;
                fastcgi_param   REQUEST_METHOD  $request_method;
                fastcgi_param   CONTENT_TYPE    $content_type;
                fastcgi_param   CONTENT_LENGTH  $content_length;
                fastcgi_param   SERVER_PROTOCOL $server_protocol;
                fastcgi_param   SERVER_PORT     $server_port;
                fastcgi_param   SERVER_NAME     $server_name;
                fastcgi_param   HTTPS           on;
        }
        location ~ /\.ht {
            deny  all;
        }
}

hg clone ssh://user@host/path/to/repo

Is simple enough…

But what about when you have a repo called “big repo”. Took me a while to figure out (with ssh verbosity) that a combination of the regular special character ‘\’ and quotes is required for hg to call the right ssh command.

hg clone "ssh://user@host/path/to/big\ repo"

A simple test this evening between two FreeBSD 8.0-RELEASE machines. OpenVPN 2.1.1 compile from ports (source).

gen2 i386, Pentium 4 2.0GHz 2.0GB circa 2003

[root@gen2 /usr/ports/security/openvpn]# make install
real	4m11.491s
user	1m46.402s
sys	0m26.863s

blackbox i386, Athlon Thunderbird 1.0GHz 512MB circa 2001

[root@blackbox /usr/ports/security/openvpn]# make install
real	6m9.970s
user	3m15.515s
sys	0m43.739s

I recently migrated my RAID-5 home storage array to RAIDZ using ZFS on FreeBSD 8.0-RELEASE. FreeBSD’s latest release includes ZFS v13 with many tuning patches, making ZFS not only more stable but easier to tune.

It is noted that on 64-bit systems with over 2GB RAM, default auto-tuning settings will be quite stable. However in any other configuration some tuning is still necessary to achieve a stable system.

On 32-bit you still have to compile a custom kernel per these instructions. If it is your first time compiling a custom kernel, don’t worry it is a very short procedure. Kernel KVA_PAGES (virtual address space) must be increased to allow for ZFS to use more kernel memory. The setting for this value depends on other userland applications that will be running on the server, and how much memory they require. KVA_PAGES value must be a multiple of 4 and that is how you can determine the kernel memory allocation, in my case:

KVA_PAGES=384 -> 384 * 4 -> 1536MB (1.5GB)

System configuration:

FreeBSD/i386 8.0 (Pentium 4)
2GB RAM
965GB ZFS RAID-Z over 3 drives

The general guideline I followed for setting tuning values was:

vm.kmem_size -> 1/2 of total RAM
vm.kmem_size_max -> 1/2 of total RAM
vfs.zfs.arc_max -> 1/2 of vm.kmem_size_max

However when testing at high transfer load (1000mbit LAN) with arc_max set to 512MB I still experienced kernel panics. I still suggest starting at 1/2 of kmem_size and working down from there.

Final stable tuning values:

# /boot/loader.conf
vm.kmem_size="1024M"
vm.kmem_size_max="1024M"
vfs.zfs.arc_max="256M"

# /etc/sysctl.conf
kern.maxvnodes="400000"

# Custom Kernel Options
options    KVA_PAGES=384    # increase kernel virtual address space for ZFS

Damn, perl is ugly by today’s standards, but I found this old script kicking about so I decided to dust it off and make it useable. This happens alot with asterisk code posted on the various wikis.

Using the newer Text::CSV_XS library and the latest asterisk column structure I’ve made it work as below to import about 4500 rows from cdr_csv on a production system. Enjoy! Glad to be back.

Pre-requisites:

DBD::Pg
Text::CSV_XS

#!/usr/bin/perl -w
# Asterisk 1.6 CDR_CSV import to PGSQL
# Herbert Molenda <me at herbertm.ca> - 2009-12-09
# updated from original by Stéphane HENRY stephane.henry (=at=) heberge.net - 2007-03-20
 
use strict;
use DBI; 
use Text::CSV_XS;
 
# this script import asterisk cdr log Master.csv into a postgresql table
 
my $cdr_log_file = $ARGV[0]; 
my $pg_host = $ARGV[1]; 
my $pg_db = $ARGV[2]; 
my $pg_table = $ARGV[3]; 
my $pg_user = $ARGV[4]; 
my $pg_pwd = $ARGV[5]; 
my $csv = Text::CSV_XS->new({ quote_char => '"', always_quote => 1 });
 
&usage if (!$ARGV[5]); 
 
 
# Connect to database 
print "Connecting to database...\n\n"; 
 
my $dsn="DBI:Pg:dbname=$pg_db;host=$pg_host;port=5432";
my $dbh=DBI->connect($dsn,$pg_user,$pg_pwd);
 
if ($dbh) {
	print "Successfully connected to $dsn\n";
	open cdr_log, "< $cdr_log_file" or die "Cannot open cdr_log_file\n";
	while (<cdr_log>) {
            if ($csv->parse($_)) { 
		my (@fields) = $csv->fields(); 
		my $insert_str = "insert into $pg_table (src, dst, dcontext, clid, channel, dstchannel, lastapp, lastdata, calldate, duration, billsec, disposition, uniqueid, userfield) values (\'".$fields[1]."\', \'".$fields[2]."\', \'".$fields[3]."\', \'".$fields[4]."\', \'".$fields[5]."\', \'".$fields[6]."\', \'".$fields[7]."\', \'".$fields[8]."\', \'".$fields[9]."\', \'".$fields[12]."\', \'".$fields[13]."\', \'".$fields[14]."\', \'".$fields[16]."\', \'".$fields[17]."\');"; 
		print $insert_str."\n";
		my $sth = $dbh->prepare($insert_str); 
		$sth->execute(); 
		$sth->finish(); 
	    } else {
	        my $err = $csv->error_input;
        	print "Failed to parse line: $err";
	    }
 
	} 
	$dbh->disconnect(); 
	close (cdr_log); 
}
else{
	die("Problem connecting to : $dsn\n");
}
 
print "\n\nEnd.\n"; 
exit; 
 
 
sub usage() { 
       print_header(); 
       print "\nUsage: perl cdr_csv2pgsql.pl <cdr_log_file> <pg_hostname> <database> <table> <username> <password>\n"; 
       die; 
}; 
 
sub print_header() { 
       print "\ncdr_csv2pgsql.pl - Asterisk 1.6 CDR_CSV import to PGSQL\n"; 
} </password></username></table></database></pg_hostname></cdr_log_file></me>

Getting connected:

Configure the connection as you would normally: hostname, username, password, domain (usually blank). Now lets get to the advanced settings that are not so well documented.

Default settings will likely result in LCP (host refused authentication) errors, which indicate that your authentication request is being denied by the server. Windows VPNs can be tricky to connect to, but with the right settings there’s no issue.

Dec  7 10:16:50 chater pppd[3919]: CHAP authentication succeeded
Dec  7 10:16:50 chater pppd[3919]: MPPE 128-bit 40-bit stateless compression enabled
Dec  7 10:16:50 chater pppd[3919]: LCP terminated by peer

Lets disable compression algorithms we know can’t be used in this case. To connect to Windows PPTP servers you need to enable MPPE encryption, easy; but problems above arise if you leave it to networkmanager to determine whether to use 40-bit or 128-bit. Force 128-bit encryption and make sure your advanced configuration looks like this:

nm-pptp vpn advanced configuration

Just in time for remembrance day! Reading an article today reveals another epic scam by our provincial politicians. With public support for our military finally gaining ground, it seems anyone will take advantage to make an extra buck. A popular way seems to be claiming that buying expensive, useless stuff is going to support various CF charities; the claim is not completely untrue but it is quite misleading when the fact of the matter is:

…the charity [Canadian Forces Personnel Assistance Fund] gets 1.43% of the sale of the $314 plates and 2.89% on the $77.75 set.

Not much more to say on that one. Buy a personalized plate if you want, but don’t buy it thinking you’re actually making a difference to military families.

A not-so-well known feature of Linux (aside from server administrators and gentoo tweekers probably) is the kernel’s I/O scheduler. I believe as of 2.6.18, CFQ (Complete Fair Queuing) has become the default for most distributions as it is well suited to low-latency desktop use. Previously anticipatory (which is better on slower drives) seemed to be the preferred choice. As computing power increased and hard drive seek times decreased other, more aggressive, schedulers became more efficient. I am not going to go into detail on how these schedulers work since it is quite a good read on wikipedia.

If you are curious here are some commands to check what is running:

Replace sda with your block device:

# cat /sys/block/sda/queue/scheduler 
noop anticipatory deadline [cfq] 

Changing to a different scheduler

1. Some distributions compile all the schedulers into the kernel and let you select using a kernel option at boot. Add the following to your kernel line in grub.conf (cfq,as,deadline,noop):

 elevator=cfq

2. The recommended way seems to be using a grub kernel setting at boot, but it can also be done hastily by simply echoing the scheduler to the /sys object.

# echo deadline > /sys/block/sda/queue/scheduler

Device Drivers -->
          Block Devices -->
                   IO Schedulers -->
                         CFQ I/O Scheduler

It seems the default OpenSUSE 10.3 system installs all 4 schedulers, and defaults to CFQ. This works great for me in desktop applications, however if you are running a very task-specific server (such as database or file serving only) one of the other schedulers tailored for that task would be better. Similarly on older hardware (laptops, slow drives <7200rpm), a CFQ scheduler may be causing too much head movement and degrade performance noticeably.

Figure 1.1 Care of RedHat tests conducted for an Oracle database server.

Figure 1.2 Another Oracle database benchmark comparing all 4 schedulers from OracleSponge

Further discussion

evolution-scalix from Gnome SVN:

checking Checking for Evolution version 2.10... not found
checking Checking for Evolution version 2.8... not found
checking Checking for Evolution version 2.6... not found
checking Checking for Evolution version 2.4... not found
configure: error: No compatible Evolution version found or missing development headers

We know that the connector trunk is stuck back in the days of Evo 2.10, so unless you are still around with <2.8 where it works flawlessly you need to do a little tweaking to get up and running. Some people claim that the older versions are more stable anyway, so that may be a valid choice.

Note: Further discussion and eventual resolution at trunk openSUSE 10.3 Evo 2.12 [RESOLVED EVO 2.12 STABLE]

Most providers these days won’t offer DID announcement if you have several virtual numbers under the same account (separate logins are fine, but gets messy with register statements). The solution is to extract the TO: line from the SIP invite header and use that to discriminate DIDs.

Provider SIP message (viewable from asterisk by using the sip debug command at the CLI):

< -- SIP read from 216.18.125.7:5065: 
BYE sip:s@192.168.2.4 SIP/2.0
To: <sip:xxxxxxxxxxx@216.18.125.3;user=phone>;tag=as6370c3aa
From: "CID NAME"<sip :xxxxxxxxxx@sip.babytel.ca>;tag=650a2e00
...
</sip>

The dialplan below is designed to be used in an AMP/FreePBX scenario inside the extensions_custom.conf file. You can enter your additional DIDs/virtual numbers in here and to be parsed before proceeding with the other dialplans configured in the web admin.

Since my provider’s TO: header has a bunch of junk surrounding the number, I use asterisk’s string functions to create a substring and evaluate against the DID number only. Makes the code easier to read, especially when further additions are made. Note: The offset may differ for you, depending on how the TO line is sent in the header above.

Modified dialplan to include header DIDs:

[ext-did-custom]
exten => s,1,Set(DID_TO="${SIP_HEADER(TO)}")
exten => s,2,GotoIf($["${DID_TO:6:11}" = "xxxxxxxxxxx"]?ext-local,205,1)