Linux system stressing

After reading this post, I’ve start having a look at a few system stressing tools. Some useful links:

openMosix stressing
Linux Test Project
stress project

IBM resource: Stress-testing the Linux kernel

There’s plenty of information on every link with uses and examples.

Advertisement

Linux buffer cache state

Following Faidon’s comment on an earlier post, I came across  this informative site concerning Linux’s buffer cache state. In a nutshell, the following code will release all the cached data of a specified file.

#define _XOPEN_SOURCE 600
#include <unistd.h>
#include <fcntl.h>
int main(int argc, char *argv[]) {
    int fd;
    fd = open(argv[1], O_RDONLY);
    fdatasync(fd);
    posix_fadvise(fd, 0,0,POSIX_FADV_DONTNEED);
    close(fd);
    return 0;
}

There are some useful samples and examples on the mentioned web page. posix_fadvise description here.

lcfg-xen-1.0.1

Finally got to release 1! Not many new features but more or less stable as it seems. The only addition to this is the support for defining and configuring groups of virtual machines, forming eventually virtual clusters.
New resources are introduced that can replace the individual resources for every virtual machine that is member of the virtual group.

!xen.virtualgroup		mADD(cluster1)
!xen.members_cluster1  		mADD(node1 node2 node3)
!xen.maxmem_cluster1    	        mSET(2048)
!xen.memory_cluster1    	        mSET(1024)
!xen.groupvcpus_cluster1   	mSET(4)
!xen.groupcpus_cluster1     	mSET(4)

The resource definition above will configure each member of the virtual group according to the specified resources.

Network configuration can also be performed per group basis:

!xen.groupnettype_cluster1	mSET(bridge)
!xen.groupbridge_cluster1	mSET(xenbr2)
!xen.groupscript_cluster1	mSET(my-custom-bridge)

The default values are “512” for maximum and initial memory, “1” for VCPUs and pinned physical CPUs, “bridge” for network type, “xenbr0” as the default network bridge interface and “vif-bridge” the default network script.

EDIT: Issues with the schema file. New release 1.0.3-1 solves it.

RPM: lcfg-xen-1.0.1-1.noarch.rpm lcfg-xen-1.0.3-1.noarch.rpm
SRPM: lcfg-xen-1.0.1-1.src.rpm lcfg-xen-1.0.3-1.src.rpm
Default schema file: lcfg-xen-defaults-s1-1.0.1-1.noarch.rpm lcfg-xen-defaults-s1-1.0.3-1.noarch.rpm

Fetching and saving my Twitter tweets

I have heard quite a few people complaining that tweets on Twitter are erased after some period and they can’t be saved without hassling. A quick googling revealed some options but I can’t say that there was anything satisfying, at least for what I was looking for. The way I’d like to save my tweets would be directly to my system, in a simple text file which I’ll be able to access from any other system if I need so.

To meet this “personal requirements” I implemented a short application in Java. It will fetch the specified feed and save your entries in a specified file. On the next fetch operation, if the user specifies the same destination file, it will just update the file with the new tweets from the feed. The most effective way to get that automatically done is to use it as a daily cronjob.

I have the following entry in my crontab:

$ crontab -l | grep fetch
41 23 * * * ~/xNIX/scripts/backups/fetchTweets.sh

and the scripts all it does is:

$ cat fetchTweets.sh
cd ~/xNIX/scripts/backups
java twitterFetcher http://twitter.com/statuses/user_timeline/17578627.rss ~/myTweets

I wouldn’t consider this as a proper Twitter backup application but more like a hack to get the work done without touching the Twitter API or using an external service that would require more manual work to get the tweets on my system. Also, as it uses the RSS Feed, it’s not suitable for those who tweet massively throughout the day.

Get twitterFetcher.java

Recovering messed LCFG SL5 Xen virtual machine

As there is no support at the moment for paravirtualised LCFG managed virtual machines (due to bootstrapping mechanism), all the deployed LCFG managed virtual machines are fully virtualised. That means that altering the number of CPUs for a VM requires a reboot of the VM as such changes can’t be done on-the-fly, like on paravirtualised VMs. Because of that I had to reboot one of these systems to increase the number of CPUs in use.

After rebooting the VM and loading Grub, the infamous “Error 15: File not found…” appeared. After using ‘kpartx‘ to access the partition of the disk image, I mounted the root partition under a local directory of the host system and realised that there was no vmlinuz and initrd under /boot. Pretty strange as I couldn’t recall anything that could cause that. I got the required files from another identical virtual machine (running on the same hardware, with the same specs and same OS and kernel version) and copied them over. The faulty VM was able to boot now but prompting two worrying messages:

ERROR: DM multipath kernel driver not loaded

and

tg3 device eth0 does not seem to be present, delaying initialization

I first checked the ethernet driver, as DM was not used anyway, in /etc/modprobe.conf and the entry was there, intact as it should be. I then realised that /usr/src/kernel was totally empty and most of modules missing from /lib/modules/2.6.18.123.4.1.el5/*.  The later could explain both the MD and ethernet problems. I did the same as with /boot files and copied those over from another identical virtual machine. Next reboot and there was no MD or eth0 errors, good sign. However, the lcfg components failed to start, most importantly the boot component. As an addition to that, the login prompt had as hostname ‘localhost‘ instead of the one that should really be. Having a look under/var/lcfg/log/boot revealed that the system couldn’t locate its configuration database because of the wrong hostname:

qxprof: can't tie /var/lcfg/conf/profile/dbm/localhost.DB2.db :no such file or directory

I got to have a look in /etc/sysconfig/network and the file was empty! That explained the ‘localhost’ as hostname. I edited according to the systems configuration and gave it one more kick. This time the system started up fine. The LCFG components were complaining though that couldn’t find the group ‘lcfg’, which shows that the machine was seriously messed. Nevertheless, all the components did their work as they should and all users and groups were automatically sorted, machine could bind on NIS domain, the kernel updates were applied successfully and the machine was back to normal. Despite its working status, I still haven’t figure the exact reasons the kernel packages were missing. A quick look at the updaterpms (the LCFG component that takes care of the RPMs) shows that since two days ago, the kernel packages had an ignore status. Now need to find out why that happened and if that causes automatic uninstall of these packages.