Alan Ford's ECS Homepage
Moved here.
I'm currently in my final year (of four) of the MEng Computer Science course at ECS at the University of Southampton, UK.
One day I'll arrange this into several pages, however in the mean time, scroll down to find Perl and Java tips and tricks, some documents and presentations I've created, and software and patches I've written.
I can be emailed at (mangled) <ajf101 AT ecs.soton.ac.uk>, or <ajf101 AT zepler.net>, which will last long after I graduate. Also, I'm 3033272 on ICQ and alan@whirlnet.co.uk on MSN.
CSLib
CSLib is ECS's student-supported systems. If you aren't using it on the lab Linux machines - why not?! :-).
I used to be the CSLib admin - but this honour has now passed to James Goldsmith. I still know what's going on and can fix things if they break, however, so feel free to shout at me in the labs (see mugshot).
Perl Useful Bits
This section is some useful but obscure bits of perl I've gathered over time, that I find useful and hopefully so will other people.
- Safe command execution and reading
To get the output from a command, it's easy to do `command args`, but that is a bad security hole if the arguments are variable, since that's just a shell execution. To do it safely, do:open(PROG, "-|") or exec ("command", "arg1", "arg2"); while(<PROG>) { ... } close (PROG);For the curious, this works by forking (-|). Open returns the PID of the forked program - however if this is the child, open will have returned zero, thus executing the "or" part. And since exec does not return, that's all that's run.
To write to a command, using safe arguments:open(PROG, "|-") or exec ("command", "arg1", "arg2"); print PROG "Hello"; close(PROG);Will also work fine.
You do not need to do anything fancy with IPC::Open2 and IPC::Open3 for bi- and tri-directional pipe communication respectively, since you can supply the command as an array of the command and arguments, as is possible (and used above) with system and exec. - Date formatting
This is just neat, to produce the date in YYYY-MM-DD format.$date = sprintf("%04d-%02d-%02d", sub {($_[5]+1900, $_[4]+1, $_[3])}->(localtime)); - Encoding strings to URL format
Apply this regular expression to a string to encode all non-word characters to their %HEX equivalents for use in URLs:s/([^\w])/"%" . uc(sprintf("%2.2x",ord($1)))/eg - Splitting a path into directory and filename
Very simple regexp, will set $1 to the path, and $2 to the filename.m/(.+)\/([^\/]+)$/;
- Daemonising your script
Very simple - just fork before you want the daemon to begin, and exit if you were the parent (if you have the pid set). It's common courtesy to log your pid in /var/run if possible, so you can be easily killed!$pid = fork; exit if $pid; open (PID, ">/var/run/foo.pid"); print PID $$; close (PID);
- Sorting Stuff
sort is much more powerful than it first appears. If you do:@list2 = sort @list1;
Then the list will be sorted asciibetically. This doesn't work for numerical sorting. Instead, for that, you want:@list2 = sort { $a <=> $b } @list1;Then the list will be sorted according to the predicate given, in this case, from smallest to largest. The $a and $b could of course be switched to sort in the other direction. Any other comparison can be given in this, for example, this is really neat to numerically sort a hash by the value of the pair:%hash2 = sort { $hash{$a} <=> $hash{$b} } keys %hash - Redirecting STDOUT/STDERR
This is useful for many things, including if you only want the exit status from a program called with system(), and don't want its output to appear.open OLDOUT, ">&STDOUT"; open OLDERR, ">&STDERR"; open STDOUT, ">/dev/null"; open STDERR, ">/dev/null"; # Do system call etc here close STDOUT; close STDERR; open STDOUT, ">&OLDOUT"; open STDERR, ">&OLDERR";
This works by saving the old STDOUT/STDERR file handles in temporary handles, while they are redirected to /dev/null. When you're done, close the new file handles and re-open them with the temporary handles. - Beautiful Obfuscation
What does this code return?[$a => $b] -> [$b <= $a]
Clues: The first square brackets create an anonymous array reference. => is basically a synonym for comma (=> is the hash notation, while the comma is array notation - but hashes are arrays anyway). -> dereferences an array reference. <= is a boolean comparison operator, that's all, it will return 0 or 1.
So, this code returns the lowest of the two values. Urgh!
Java Useful Bits
These are a few bits of Java that are not obviously documented, but can be handy in programs.
MD5 Checksums in Java
Java provides a java.security.MessageDigest class, but this is how to make it actually do a MD5 sum of a String:
public static String MD5Sum (String s) {
MessageDigest MD5;
try { MD5 = MessageDigest.getInstance("MD5"); }
catch (Exception e) { return null; }
String ans="", temp;
byte[] digest = MD5.digest(s.getBytes());
for (int i=0;i<digest.length;i++) {
temp = Integer.toHexString(0xFF & digest[i]);
ans += (temp.length() == 1 ? "0" + temp : temp);
}
return ans;
}
NIS/YP Directory Lookups in Java
Java can integrate with a NIS database via JNDI, it's just not documented. You'll need the special JNDI NIS service provider.
The following example will connect to NIS domain "foo" on the server "nis.domain.co.uk", and extract the value attributed to the given key (user) from the "system/passwd" NIS map.
public static String NISPasswd (String user) throws Exception {
Hashtable env = new Hashtable();
env.put(Context.INITIAL_CONTEXT_FACTORY, "com.sun.jndi.nis.NISCtxFactory");
env.put(Context.PROVIDER_URL, "nis://nis.domain.co.uk/foo");
Context ctx = new InitialContext(env);
Context passwd = (Context)ctx.lookup("system/passwd");
String value = passwd.lookup(user).toString();
return value;
}
You can also do things like interate over the passwd data, line-by-line, rather than lookup a particular user. More details on this can be found on the JavaDoc for the Context interface (see the listBindings method).
Documents
Modified Kernel Packet Travelling Diagram
I have an interest in Linux routing and QoS, and my third year project was heavily involved with this topic. Stef Coene's KPTD is a brilliant resource for seeing how the Linux kernel processes packets. This KPTD has become rather crowded, with both kernel 2.2 and 2.4 and IMQ information, which is not directly relevant. So, I drew my own handy cut-out-and-keep (well, full A4 printout) version.
Introduction to 802.11 and Meshed Networking
This presentation was done for the 4th year computer networks course. I was rather pleased with it as a summary however so thought I'd stck it up here!
Things to Think About when Renumbering an IPv6 Network
This Internet-Draft came out of work I did during summer 2004, and have continued since, with Tim Chown, Mark Thompson, and Stig Venaas, related to the 6net project.
- Current version (-01), 21 February 2005:
- Older version (-00), 18 October 2004:
IPv6 Multihoming
This was my fourth year Individual Research Project, summarising the current state and possible future developments in the field of IPv6 multihoming, one of the last major hurdles with IPv6 development.
Random Linux Software Stuff
These are rather minor, yet useful, creations of mine come about over recent times.
isup
Is a machine up? This attempts to make a TCP connection on a port (default 7, can specify something else). If it can connect, or the connection is refused, it infers the machine is up and returns success. Otherwise, it returns an error code. I've found it useful in shell scripts that need to know if a machine is up or not. In addition, it can optionally print out a message saying if the machine is up or not, and if it is not, give the error message. The latest versions (v0.3+) also can be used for connection testing (whether a port is available or not) as well as whether the machine is actually up. Version 0.4 added support for a specified timeout. Version 0.5 added IPv6 support, although due to the removal of several kludges the source code size actually decreased!
- Version 0.5, with IPv6 support:
- Old versions:
- Version 0,4, with timeout support:
- Version 0.3.1, with support for named ports:
- Version 0.3, with support for connection availability testing:
- isup.c (source)
- Version 0.2, with human-readable output support:
- Version 0.1, initial release:
getfn
This will search through the system passwd file, or NIS passwd file if it exists, for a given username or part of username or name. The results are presented in a way it can easily be used as an external mutt query program.
The latest version of this program is 0.6, and should now be generic enough to work in places other than ECS (although ECS is still the compile-time default domain name, this can be changed on the command line and by changing the DEFINE in the source). It now has several useful command-line options. If NIS is available, it will directly search the NIS passwd file, which is far far quicker than doing getpwent searches - these are only used as a last resort if NIS is unavailable, or if the program is not compiled with NIS support (see comments in source for compile instructions).
There used to be two versions of this program, still retained here for historical reasons. The 0.2 series is the old version that used to be used in CSLib, since it is designed to work fast and efficently, but only for ECS addresses. The 0.5 series was supposed to be a more generic version, not requiring NIS and having flexible domain definitions. Versions from 0.6 onwards, however, handle this much better. Version 0.7 added support for additional addresses to be kept in an external file.
- getfn 0.7 (source)
- getfn.1 (HTML)
- Version 0.6, merged releases and general tidy-up:
- Old versions (added case-insensitive searching):
- getfn 0.2.3 (source)
- getfn 0.5.3 (source)
- getfn.1 (HTML)
- Even older versions (split versions, not very flexible):
- getfn 0.2.2 (source)
- getfn 0.5.2 (source)
- getfn.1 (man page) (HTML)
smtppush
This is a simple command-line SMTP relay program intended to be used as a lightweight replacement for sendmail or other MTA. It simply relays all mail to the defined host (via the -S option), failing if unable to relay the email.
This program was originally written by Michael Elkins, the original author of the mutt e-mail client, to send directly to a SMTP server. Since it was no longer maintained, and in fact had pretty much disappeared off the face of the Internet, yet I still found it useful, I decided to continue maintaining it, with new features where appropriate.
My first modification, version 0.3: uses getaddrinfo, for IPv6 support; supports multiple addresses associated with hostnames; supports iteration over multiple servers until one is available; and adds the -l command-line option for allowing more flexibility over the HELO value.
- smtppush 0.3 (source)
- smtppush 0.3 (manual page) (HTML)
- Original version by Michael Elkins:
- smtppush 0.2 (original source tarball)
- smtppush 0.2 (source only)
randombg
This will look for all your X background pictures, by default in ~/backgrounds, and pick a random one to use. It requires xv to work. It should pick the resolution of one monitor and use that as the scaled size for the background image.
freq
This will take a list of items on standard input, and calculate the frequency with which each one occurs. Command line arguments - -s to sort the list by frequency (default is alphabetically), -h to print the list as a histogram.
mailbox-stats
This will process a unix mbox mailbox, counting how many mails you received each day, and producing a histogram thingy.
Perl/Tk NS Lookup (ptknslookup)
I wrote this many years ago to learn how to use Perl/Tk. It's still quite a nice GUI interface to DNS lookups, and is a fair example of simple Perl/Tk programming.
Various Patches
- I wrote quite a few bits of the abook address book program.
- I've contributed several patches to the gaim instant messaging client. Some, however, are not included for various reasons, and are here instead:
- This patch allows you to configure how long before you become "idle".
- This patch (older version) adds a range of MSN tweaks, including contact list status, monitoring signon and idle times.
- This patch is an updated version of somebody else's patch posted on the gaim patch tracker, that logs when buddies were last seen online.
- This patch for cal.c in util-linux adds support for giving the cal(1) program a range of months to print calendars for. Actually very useful!
- This patch for tty.c in GNU Shellutils (now part of GNU Coreutils), adds support for -n, to give the shortened name of a tty.
- rdesktop allows connection
from a unix machine to a Windows Terminal server. A patch was posted on the
rdesktop-devel mailing list, against rdesktop 1.2.0, to add basic paste support
from X to the terminal server, by converting the clipboard contents to
keystrokes, on a middle-button click. I have made a patch to be applied
after the main patch to make this only work if there is something on the
clipboard. If there is not, then the middle click will be sent as normal.
- The original patch by Steve Fosdick.
- My additional patch.
- Kernel Patches (these are generally laughable)
- Fancied an ASCII art logo at the top of your dmesg? Now you can.
- Want to know when you add a dummy network device?
- Got a Dell Inspiron, want the shortcut buttons to work without using those high-level command-line tools? Inspiron Shortcut Buttons.
- Can't remember those damn SysRq shortcuts? Now you can get more advanced help with thier use.
- Want numlock to be on at bootup? Use this patch and the new "numlock" kernel command-line option.
OpenSSL Tricks
These are just a couple of handy incantations to make OpenSSL do useful stuff.
- Talking HTTPS
If you run the following command, OpenSSL will do all the certificate negotiation for you and provide this information. It then acts like telnet/netcat etc as a client, so you can type away at HTTP requests.openssl s_client -connect secure.ecs.soton.ac.uk:443
- Certificate Generation
There are two steps to generating a self-signed certificate for a web site:
- Generate a Private Key
Use the following command, specifying the strength of the key in bits (e.g. 1024):openssl genrsa -out server.key 1024
- Generate a Self-Signed Certificate
Run the following command, using the private key as generated previously. It will prompt you for the necessary owner details for the certificate. -days specifies the number of days the certificate will be valid for.openssl req -x509 -new -days 365 -key server.key -out server.crt
- Generate a Private Key