Example shell-script for using STATIST in batch-mode

[sfronzek@gmx.net]

Hi statist-list,

I have started using the batch-mode functionality of STATIST to calculate
simple statistical numbers such as the standard deviation for a large number of
files. Maybe some of you find the shell-script below useful so I post it to
this list. The task of it is to extract some data from a relatively large
file and calculate the standard deviation for it.

The script is by no means a masterpiece of shell-programming, but I hope to
give an example of how STATIST can be used effectively in batch-mode.

Examples of the data files for which I made the shell-script can be found
from http://www.knmi.nl/samenw/eca/htmls/index3.html. 

Copy the script below to std.sh and the batch-commands to a file
STAT_batch.txt (note the empty lines!).

Best regards,
Stefan.



>>>>>>>>BEGINNING OF FILE std.sh<<<<<<<<<<
#! /usr/bin/sh

# std.sh <data_file>
# Calculates the standard deviation for each month of every year
# from a table with daily values by calling STATIST in batch-mode.
# The data file is expected to be comma-separated table with one line per
# day, the date denoted as yyyymmdd in the second column, and the data for
# which to calculate the standard deviation in forth column. The output
# goes to standard out as well as separate files for each month.

echo std.sh: Processing file $1

# list of months for which to do the calculations
monthlist="01 02 03 04 05 06 07 08 09 10 11 12";

for month in $monthlist ;
do
	# first year to calculate
	year=1951;

	# set the file name for output
	outfile=$1$month.dat;

	# loop through the years until 1980
	while [ $year -lt 1980 ];
	do
		# grep out the values for calculation: all days of one month in one year
		grep $year$month $1 |	\
		# use the 4th column = mean temperature
		cut -d, -f4 |		\
		# substitute the missing-value denotion -9999 with the STATIST denotion
'M'
		sed '1,$ s/-9999/    M/' > tmp.dat;
		# call STATIST in batch-mode to calculate the standard deviation
		stddev=$(statist tmp.dat -silent -noplot < STAT_batch.txt | grep "Standard
deviation" | cut -d: -f2);
		# print out: first to standard out, then to separate files for each month
		echo $year$month	$stddev;
		echo $year$month	$stddev >> $outfile;
		# increment the year counter
		year=`expr $year + 1`;
	done
done

# remove the tmp file
rm tmp.dat;
>>>>>>>>END OF FILE std.sh<<<<<<<<<<


>>>>>>>>START OF FILE STAT_batch.txt<<<<<<<<<<
4
1
a




0
>>>>>>>>END OF FILE STAT_batch.txt<<<<<<<<<<

-- 
GMX - Die Kommunikationsplattform im Internet.
http://www.gmx.net