Documentation in this arena being slim, I felt it would be helpful to write a short article on my configuration. FastCGI examples are generally documented in about two or three blog posts on the subject, all of which have become dated. For the basic ground work take a look at streamhacker.

Firstly, the fastcgi process is unaware whether you are using HTTP or HTTPS, another fastcgi_param must be passed to solve the following error:

searching for changes
ssl required

fastcgi_param   HTTPS           on;

If you are having the following error, and you have ensured that HTTPS is functioning and allow_push = username is specified in the repo’s .hg/hgrc:

searching for changes
27 changesets found
abort: authorization failed

Then the fastcgi process isn’t receiving the username from nginx. Ensure that the following lines are added to your nginx config:

fastcgi_param   AUTH_USER       $remote_user;
fastcgi_param   REMOTE_USER     $remote_user;

Nginx is also liable to spit out a:

413 Request Entity Too Large

on your first major commit. This is because the max_body_size is set low by default and is unacceptable for a mercurial application. Ensure to add and modify this line to your needs:

client_max_body_size    100M;

my entire nginx vhost.conf

server {
    listen      443;
    server_name hg.DOMAIN.COM;

    ssl                  on;
    ssl_certificate      server.crt;
    ssl_certificate_key  server.key;
    ssl_session_timeout  5m;
    ssl_protocols  SSLv2 SSLv3 TLSv1;
    ssl_ciphers  ALL:!ADH:!EXPORT56:RC4+RSA:+HIGH:+MEDIUM:+LOW:+SSLv2:+EXP;
    ssl_prefer_server_ciphers   on;

        # Increase transfer size to accommodate large pushes
        client_max_body_size    100M;

        location / {
                auth_basic "Restricted Access";
                auth_basic_user_file /repo/sites/hg.DOMAIN.COM/mercurial_passwd;
                fastcgi_pass    127.0.0.1:9001;
                fastcgi_param   PATH_INFO       $fastcgi_script_name;
                fastcgi_param   AUTH_USER       $remote_user;
                fastcgi_param   REMOTE_USER     $remote_user;
                fastcgi_param   QUERY_STRING    $query_string;
                fastcgi_param   REQUEST_METHOD  $request_method;
                fastcgi_param   CONTENT_TYPE    $content_type;
                fastcgi_param   CONTENT_LENGTH  $content_length;
                fastcgi_param   SERVER_PROTOCOL $server_protocol;
                fastcgi_param   SERVER_PORT     $server_port;
                fastcgi_param   SERVER_NAME     $server_name;
                fastcgi_param   HTTPS           on;
        }
        location ~ /\.ht {
            deny  all;
        }
}

hg clone ssh://user@host/path/to/repo

Is simple enough…

But what about when you have a repo called “big repo”. Took me a while to figure out (with ssh verbosity) that a combination of the regular special character ‘\’ and quotes is required for hg to call the right ssh command.

hg clone "ssh://user@host/path/to/big\ repo"

A simple test this evening between two FreeBSD 8.0-RELEASE machines. OpenVPN 2.1.1 compile from ports (source).

gen2 i386, Pentium 4 2.0GHz 2.0GB circa 2003

[root@gen2 /usr/ports/security/openvpn]# make install
real	4m11.491s
user	1m46.402s
sys	0m26.863s

blackbox i386, Athlon Thunderbird 1.0GHz 512MB circa 2001

[root@blackbox /usr/ports/security/openvpn]# make install
real	6m9.970s
user	3m15.515s
sys	0m43.739s

I recently migrated my RAID-5 home storage array to RAIDZ using ZFS on FreeBSD 8.0-RELEASE. FreeBSD’s latest release includes ZFS v13 with many tuning patches, making ZFS not only more stable but easier to tune.

It is noted that on 64-bit systems with over 2GB RAM, default auto-tuning settings will be quite stable. However in any other configuration some tuning is still necessary to achieve a stable system.

On 32-bit you still have to compile a custom kernel per these instructions. If it is your first time compiling a custom kernel, don’t worry it is a very short procedure. Kernel KVA_PAGES (virtual address space) must be increased to allow for ZFS to use more kernel memory. The setting for this value depends on other userland applications that will be running on the server, and how much memory they require. KVA_PAGES value must be a multiple of 4 and that is how you can determine the kernel memory allocation, in my case:

KVA_PAGES=384 -> 384 * 4 -> 1536MB (1.5GB)

System configuration:

FreeBSD/i386 8.0 (Pentium 4)
2GB RAM
965GB ZFS RAID-Z over 3 drives

The general guideline I followed for setting tuning values was:

vm.kmem_size -> 1/2 of total RAM
vm.kmem_size_max -> 1/2 of total RAM
vfs.zfs.arc_max -> 1/2 of vm.kmem_size_max

However when testing at high transfer load (1000mbit LAN) with arc_max set to 512MB I still experienced kernel panics. I still suggest starting at 1/2 of kmem_size and working down from there.

Final stable tuning values:

# /boot/loader.conf
vm.kmem_size="1024M"
vm.kmem_size_max="1024M"
vfs.zfs.arc_max="256M"

# /etc/sysctl.conf
kern.maxvnodes="400000"

# Custom Kernel Options
options    KVA_PAGES=384    # increase kernel virtual address space for ZFS

Damn, perl is ugly by today’s standards, but I found this old script kicking about so I decided to dust it off and make it useable. This happens alot with asterisk code posted on the various wikis.

Using the newer Text::CSV_XS library and the latest asterisk column structure I’ve made it work as below to import about 4500 rows from cdr_csv on a production system. Enjoy! Glad to be back.

Pre-requisites:

DBD::Pg
Text::CSV_XS

#!/usr/bin/perl -w
# Asterisk 1.6 CDR_CSV import to PGSQL
# Herbert Molenda <me at herbertm.ca> - 2009-12-09
# updated from original by Stéphane HENRY stephane.henry (=at=) heberge.net - 2007-03-20
 
use strict;
use DBI; 
use Text::CSV_XS;
 
# this script import asterisk cdr log Master.csv into a postgresql table
 
my $cdr_log_file = $ARGV[0]; 
my $pg_host = $ARGV[1]; 
my $pg_db = $ARGV[2]; 
my $pg_table = $ARGV[3]; 
my $pg_user = $ARGV[4]; 
my $pg_pwd = $ARGV[5]; 
my $csv = Text::CSV_XS->new({ quote_char => '"', always_quote => 1 });
 
&usage if (!$ARGV[5]); 
 
 
# Connect to database 
print "Connecting to database...\n\n"; 
 
my $dsn="DBI:Pg:dbname=$pg_db;host=$pg_host;port=5432";
my $dbh=DBI->connect($dsn,$pg_user,$pg_pwd);
 
if ($dbh) {
	print "Successfully connected to $dsn\n";
	open cdr_log, "< $cdr_log_file" or die "Cannot open cdr_log_file\n";
	while (<cdr_log>) {
            if ($csv->parse($_)) { 
		my (@fields) = $csv->fields(); 
		my $insert_str = "insert into $pg_table (src, dst, dcontext, clid, channel, dstchannel, lastapp, lastdata, calldate, duration, billsec, disposition, uniqueid, userfield) values (\'".$fields[1]."\', \'".$fields[2]."\', \'".$fields[3]."\', \'".$fields[4]."\', \'".$fields[5]."\', \'".$fields[6]."\', \'".$fields[7]."\', \'".$fields[8]."\', \'".$fields[9]."\', \'".$fields[12]."\', \'".$fields[13]."\', \'".$fields[14]."\', \'".$fields[16]."\', \'".$fields[17]."\');"; 
		print $insert_str."\n";
		my $sth = $dbh->prepare($insert_str); 
		$sth->execute(); 
		$sth->finish(); 
	    } else {
	        my $err = $csv->error_input;
        	print "Failed to parse line: $err";
	    }
 
	} 
	$dbh->disconnect(); 
	close (cdr_log); 
}
else{
	die("Problem connecting to : $dsn\n");
}
 
print "\n\nEnd.\n"; 
exit; 
 
 
sub usage() { 
       print_header(); 
       print "\nUsage: perl cdr_csv2pgsql.pl <cdr_log_file> <pg_hostname> <database> <table> <username> <password>\n"; 
       die; 
}; 
 
sub print_header() { 
       print "\ncdr_csv2pgsql.pl - Asterisk 1.6 CDR_CSV import to PGSQL\n"; 
} </password></username></table></database></pg_hostname></cdr_log_file></me>

Getting connected:

Configure the connection as you would normally: hostname, username, password, domain (usually blank). Now lets get to the advanced settings that are not so well documented.

Default settings will likely result in LCP (host refused authentication) errors, which indicate that your authentication request is being denied by the server. Windows VPNs can be tricky to connect to, but with the right settings there’s no issue.

Dec  7 10:16:50 chater pppd[3919]: CHAP authentication succeeded
Dec  7 10:16:50 chater pppd[3919]: MPPE 128-bit 40-bit stateless compression enabled
Dec  7 10:16:50 chater pppd[3919]: LCP terminated by peer

Lets disable compression algorithms we know can’t be used in this case. To connect to Windows PPTP servers you need to enable MPPE encryption, easy; but problems above arise if you leave it to networkmanager to determine whether to use 40-bit or 128-bit. Force 128-bit encryption and make sure your advanced configuration looks like this:

nm-pptp vpn advanced configuration

Just in time for remembrance day! Reading an article today reveals another epic scam by our provincial politicians. With public support for our military finally gaining ground, it seems anyone will take advantage to make an extra buck. A popular way seems to be claiming that buying expensive, useless stuff is going to support various CF charities; the claim is not completely untrue but it is quite misleading when the fact of the matter is:

…the charity [Canadian Forces Personnel Assistance Fund] gets 1.43% of the sale of the $314 plates and 2.89% on the $77.75 set.

Not much more to say on that one. Buy a personalized plate if you want, but don’t buy it thinking you’re actually making a difference to military families.

“Why on earth would you delete a Facebook account?!?! ITS THE BESTEST THING EVER!

You have some catching up to do, the rest of you skip down:

1. SOPHOS Facebook Study – good facts, interesting experiment
2. Facebook Privacy Policy – I don’t suggest reading the whole thing, its quite lame.
3. Facebook’s Privacy Trainwreck – You remember people freaking out when the News Feeds came in?
4. Facebook: Threats to Privacy – H.Jones & J.Soltren, MIT in 2005!

And the list goes on and on, I’ve had enough. Google around yourself if you still feel cool with Facebook. All you need is “privacy” and “facebook” in one query. “Facebook” and “biometrics” is another good one, see if it produces the results you’re looking for.

Deactivation

If you’d like to go through the ordeal of asking Facebook, you can do yourself a favour and check out many blog posts that have already tried. Basically the process boils down to this:

1. Reactivate your facebook account (if necessary)
2. In Profile -> Edit, go through and delete every piece of information, field by field. prepare to cry
3. Wall post by Wall post delete every one. cry
4. Photos, not so bad go to Edit Album and check all the delete boxes for the photos then delete album.
5. Remove yourself from all networks (clever little leave network link at the bottom of the Networks page). This will take care of groups and events.
6. Delete all friends.

Dear Facebook Privacy Staff,
I have removed all of my profile content. Please remove my account from your database and confirm when this is complete.

account: email login

Regards,

Chater

The Response

Hi Herb,

Your information no longer appears on the site, and we have removed your login email address from our system.
Thanks,

Paul

User Operations

Facebook

Other Comments

What’s with this?

Please enter your full birthday. We are required to retrieve this information by law, but you will have control over whether others see it.

You’re not required by law to do anything. Sure in the U.S there is the COPPA, but (and I quote below) if you go read the laws you might realize it only has to do with a person’s age. It is meant to protect children’s privacy, which is fair enough; I wouldn’t want any kiddies using Facebook either.

(1) CHILD.ââ�¬â��The term “child” means an individual under the age of 13¹.

Sounds to me like all that has to be done here is verify my age. Why don’t they just ask for that?

The wording that is used is interesting as well. Why is Facebook retrieving information from you? For whom? As if I am worried about the noobs with accounts on there viewing my info; well, aside from the identity theives and fraudsters, who can make a killing with Facebook (ref SOPHOS), I don’t really care, its the corporation that we should be worried about. COPPA Section 1302.1

Attempts to create an account:

Checking input data......Done
WWWAcct 12.0 (c) 1997-2007 cPanel, Inc.......Done
Running pre creation script (/scripts/prewwwacct)......Done
Adding User...Unable to add group deuce

The script actually manages to create the group, which can be verified in /etc/groups.

# cat /etc/group
---
deuce:x:32070:

Which you have to manually remove to get the script to work again.

# /usr/sbin/groupdel deuce

As mentioned in the above post, (paraphrasing the Cpanel techs) the problem was that nscd is running.

# /sbin/service nscd stop

Account gets created without a hitch (assuming that group is removed manually). Easily reproducible.

A not-so-well known feature of Linux (aside from server administrators and gentoo tweekers probably) is the kernel’s I/O scheduler. I believe as of 2.6.18, CFQ (Complete Fair Queuing) has become the default for most distributions as it is well suited to low-latency desktop use. Previously anticipatory (which is better on slower drives) seemed to be the preferred choice. As computing power increased and hard drive seek times decreased other, more aggressive, schedulers became more efficient. I am not going to go into detail on how these schedulers work since it is quite a good read on wikipedia.

If you are curious here are some commands to check what is running:

Replace sda with your block device:

# cat /sys/block/sda/queue/scheduler 
noop anticipatory deadline [cfq] 

Changing to a different scheduler

1. Some distributions compile all the schedulers into the kernel and let you select using a kernel option at boot. Add the following to your kernel line in grub.conf (cfq,as,deadline,noop):

 elevator=cfq

2. The recommended way seems to be using a grub kernel setting at boot, but it can also be done hastily by simply echoing the scheduler to the /sys object.

# echo deadline > /sys/block/sda/queue/scheduler

Device Drivers -->
          Block Devices -->
                   IO Schedulers -->
                         CFQ I/O Scheduler

It seems the default OpenSUSE 10.3 system installs all 4 schedulers, and defaults to CFQ. This works great for me in desktop applications, however if you are running a very task-specific server (such as database or file serving only) one of the other schedulers tailored for that task would be better. Similarly on older hardware (laptops, slow drives <7200rpm), a CFQ scheduler may be causing too much head movement and degrade performance noticeably.

Figure 1.1 Care of RedHat tests conducted for an Oracle database server.

Figure 1.2 Another Oracle database benchmark comparing all 4 schedulers from OracleSponge

Further discussion