After reading this post, I’ve start having a look at a few system stressing tools. Some useful links:
openMosix stressing
Linux Test Project
stress project
IBM resource: Stress-testing the Linux kernel
There’s plenty of information on every link with uses and examples.
Following Faidon’s comment on an earlier post, I came across this informative site concerning Linux’s buffer cache state. In a nutshell, the following code will release all the cached data of a specified file.
#define _XOPEN_SOURCE 600
#include <unistd.h>
#include <fcntl.h>
int main(int argc, char *argv[]) {
int fd;
fd = open(argv[1], O_RDONLY);
fdatasync(fd);
posix_fadvise(fd, 0,0,POSIX_FADV_DONTNEED);
close(fd);
return 0;
}
There are some useful samples and examples on the mentioned web page. posix_fadvise description here.
Finally got to release 1! Not many new features but more or less stable as it seems. The only addition to this is the support for defining and configuring groups of virtual machines, forming eventually virtual clusters.
New resources are introduced that can replace the individual resources for every virtual machine that is member of the virtual group.
!xen.virtualgroup mADD(cluster1) !xen.members_cluster1 mADD(node1 node2 node3) !xen.maxmem_cluster1 mSET(2048) !xen.memory_cluster1 mSET(1024) !xen.groupvcpus_cluster1 mSET(4) !xen.groupcpus_cluster1 mSET(4)
The resource definition above will configure each member of the virtual group according to the specified resources.
Network configuration can also be performed per group basis:
!xen.groupnettype_cluster1 mSET(bridge) !xen.groupbridge_cluster1 mSET(xenbr2) !xen.groupscript_cluster1 mSET(my-custom-bridge)
The default values are “512″ for maximum and initial memory, “1″ for VCPUs and pinned physical CPUs, “bridge” for network type, “xenbr0″ as the default network bridge interface and “vif-bridge” the default network script.
EDIT: Issues with the schema file. New release 1.0.3-1 solves it.
RPM: lcfg-xen-1.0.1-1.noarch.rpm lcfg-xen-1.0.3-1.noarch.rpm
SRPM: lcfg-xen-1.0.1-1.src.rpm lcfg-xen-1.0.3-1.src.rpm
Default schema file: lcfg-xen-defaults-s1-1.0.1-1.noarch.rpm lcfg-xen-defaults-s1-1.0.3-1.noarch.rpm
I have heard quite a few people complaining that tweets on Twitter are erased after some period and they can’t be saved without hassling. A quick googling revealed some options but I can’t say that there was anything satisfying, at least for what I was looking for. The way I’d like to save my tweets would be directly to my system, in a simple text file which I’ll be able to access from any other system if I need so.
To meet this “personal requirements” I implemented a short application in Java. It will fetch the specified feed and save your entries in a specified file. On the next fetch operation, if the user specifies the same destination file, it will just update the file with the new tweets from the feed. The most effective way to get that automatically done is to use it as a daily cronjob.
I have the following entry in my crontab:
$ crontab -l | grep fetch 41 23 * * * ~/xNIX/scripts/backups/fetchTweets.sh
and the scripts all it does is:
$ cat fetchTweets.sh cd ~/xNIX/scripts/backups java twitterFetcher http://twitter.com/statuses/user_timeline/17578627.rss ~/myTweets
I wouldn’t consider this as a proper Twitter backup application but more like a hack to get the work done without touching the Twitter API or using an external service that would require more manual work to get the tweets on my system. Also, as it uses the RSS Feed, it’s not suitable for those who tweet massively throughout the day.
As there is no support at the moment for paravirtualised LCFG managed virtual machines (due to bootstrapping mechanism), all the deployed LCFG managed virtual machines are fully virtualised. That means that altering the number of CPUs for a VM requires a reboot of the VM as such changes can’t be done on-the-fly, like on paravirtualised VMs. Because of that I had to reboot one of these systems to increase the number of CPUs in use.
After rebooting the VM and loading Grub, the infamous “Error 15: File not found…” appeared. After using ‘kpartx‘ to access the partition of the disk image, I mounted the root partition under a local directory of the host system and realised that there was no vmlinuz and initrd under /boot. Pretty strange as I couldn’t recall anything that could cause that. I got the required files from another identical virtual machine (running on the same hardware, with the same specs and same OS and kernel version) and copied them over. The faulty VM was able to boot now but prompting two worrying messages:
ERROR: DM multipath kernel driver not loaded
and
tg3 device eth0 does not seem to be present, delaying initialization
I first checked the ethernet driver, as DM was not used anyway, in /etc/modprobe.conf and the entry was there, intact as it should be. I then realised that /usr/src/kernel was totally empty and most of modules missing from /lib/modules/2.6.18.123.4.1.el5/*. The later could explain both the MD and ethernet problems. I did the same as with /boot files and copied those over from another identical virtual machine. Next reboot and there was no MD or eth0 errors, good sign. However, the lcfg components failed to start, most importantly the boot component. As an addition to that, the login prompt had as hostname ‘localhost‘ instead of the one that should really be. Having a look under/var/lcfg/log/boot revealed that the system couldn’t locate its configuration database because of the wrong hostname:
qxprof: can't tie /var/lcfg/conf/profile/dbm/localhost.DB2.db :no such file or directory
I got to have a look in /etc/sysconfig/network and the file was empty! That explained the ‘localhost’ as hostname. I edited according to the systems configuration and gave it one more kick. This time the system started up fine. The LCFG components were complaining though that couldn’t find the group ‘lcfg’, which shows that the machine was seriously messed. Nevertheless, all the components did their work as they should and all users and groups were automatically sorted, machine could bind on NIS domain, the kernel updates were applied successfully and the machine was back to normal. Despite its working status, I still haven’t figure the exact reasons the kernel packages were missing. A quick look at the updaterpms (the LCFG component that takes care of the RPMs) shows that since two days ago, the kernel packages had an ignore status. Now need to find out why that happened and if that causes automatic uninstall of these packages.
I just tried to boot my FreeBSD-7.2 VM on VMware Fusion and got an error message that it will not have any network availability as “network bridge on device /dev/vmnet0 is not running“. I checked and indeed, there was no vmnet* interface up. A look in /Library/Application Support/VMware Fusion reveals the file boot.sh, which happens to be the only available script.
$ ls -lrt total 16944 drwxr-xr-x 3 root wheel 102 12 Jul 2007 licenses drwxr-xr-x 6 root wheel 204 12 Jul 2007 kexts -rw-r--r-- 1 root wheel 373 30 Oct 2007 license.fusion.site.6.0.200610 drwxr-xr-x 3 root wheel 102 2 Nov 2007 vmx -rwxr-xr-x 1 root wheel 3590412 19 Apr 2008 vmware-vdiskmanager -rwsr-xr-x 1 root wheel 366264 19 Apr 2008 vmware-authd -rwsr-xr-x 1 root wheel 3194884 19 Apr 2008 vmware-rawdiskCreator -rwxr-xr-x 1 root wheel 200968 19 Apr 2008 vmware-ntfs -rwxr-xr-x 1 root wheel 152533 19 Apr 2008 vmware-config-net.pl -rwxr-xr-x 1 root wheel 74916 19 Apr 2008 vmnet-sniffer -rwxr-xr-x 1 root wheel 61428 19 Apr 2008 vmnet-netifup -rwxr-xr-x 1 root wheel 501632 19 Apr 2008 vmnet-natd -r--r--r-- 1 root wheel 1241 19 Apr 2008 vmnet-nat.conf -r--r--r-- 1 root wheel 742 19 Apr 2008 vmnet-dhcpd.conf -rwxr-xr-x 1 root wheel 333464 19 Apr 2008 vmnet-dhcpd -rwxr-xr-x 1 root wheel 120612 19 Apr 2008 vmnet-bridge -rwxr-xr-x 1 root wheel 7932 19 Apr 2008 vm-support.tool -rwxr-xr-x 1 root wheel 17186 19 Apr 2008 boot.sh drwxr-xr-x 7 root wheel 238 15 May 2008 tools-upgraders drwxr-xr-x 6 root wheel 204 15 May 2008 messages drwxr-xr-x 13 root wheel 442 15 May 2008 isoimages drwxr-xr-x 17 root wheel 578 15 May 2008 vnckeymap drwxr-xr-x 5 root wheel 170 15 May 2008 vmnet8 drwxr-xr-x 3 root wheel 102 15 May 2008 vmnet1 -rw-r--r-- 1 root wheel 5612 15 May 2008 locations -rw-r--r-- 1 root wheel 81 15 May 2008 config drwxr-xr-x 3 root wheel 102 15 May 2008 Uninstall VMware Fusion.app
Its execution pretty simply:
$ ./boot.sh
Usage: ./boot.sh {--start|--stop|--restart}
$ sudo boot.sh --start
VMware Fusion 87978: Starting VMware Fusion:
kextload: extension /Library/Application Support/VMware Fusion/kexts/vmmon.kext is already loaded
kextload: /Library/Application Support/VMware Fusion/kexts/vmci.kext loaded successfully
kextload: /Library/Application Support/VMware Fusion/kexts/vmioplug.kext loaded successfully
kextload: extension /Library/Application Support/VMware Fusion/kexts/vmnet.kext is already loaded
Internet Software Consortium DHCP Server 2.0
Copyright 1995, 1996, 1997, 1998, 1999 The Internet Software Consortium.
All rights reserved.
Please contribute if you find this software useful.
For info, please visit http://www.isc.org/dhcp-contrib.html
Configured subnet: 172.16.17.0
Setting vmnet-dhcp IP address: 172.16.17.254
Opened: ??
Recving on VNet/vmnet8/172.16.17.0
Sending on VNet/vmnet8/172.16.17.0
Internet Software Consortium DHCP Server 2.0
Copyright 1995, 1996, 1997, 1998, 1999 The Internet Software Consortium.
All rights reserved.
Please contribute if you find this software useful.
For info, please visit http://www.isc.org/dhcp-contrib.html
Configured subnet: 172.16.123.0
Setting vmnet-dhcp IP address: 172.16.123.254
Opened: ??
Recving on VNet/vmnet1/172.16.123.0
Sending on VNet/vmnet1/172.16.123.0
Once finished, the VM could start again without complains and gets its IP via LAN’s DHCP.
Θα ξεφύγω λίγο από τη καθιερωμένη θεματολογία του blog. Μια κουβέντα που είχα τις προάλλες σχετικά με τα βυθισμένα Ιταλικά χωριά Graun και Reschen στα Ιταλο-Ελβετικά σύνορα, μου κίνησε τη περιέργεια για τυχών βυθισμένα χωριά στον ελληνικό χώρο. Προς μεγάλη μου έκπληξη διαπίστωσα ότι υπάρχουν γύρω στα 25 βυθισμένα χωριά και πολύ πιθανόν και άλλα στα όποια να μη γίνεται κάποια αναφορά. Η πλειοψηφία αυτών βρίσκεται στο τεχνιτή λίμνη των Κρεμαστών. Ο λόγος για τη βύθιση όλων των χωριών είναι η δημιουργία φραγμάτων που έχουν ως συνέπεια και σκοπό τη δημιουργία λιμνών. Κάποια από τα χωριά έχουν γρεμιστεί με τα χρόνια, άλλα με τη “βοήθεια” μηχανημάτων κατα τη δημιουργία των λιμνών. Μέσα στα κτίσματα συμπεριλαμβάνονται και οι εκκλησίες αλλά και πέτρινα παραδοσιακά γεφύρια. Ίσως το πιο αξιοσημείωτο κτίσμα να θεωρηθεί η μεσαιωνική εκκλησία της Επισκοπής χτισμένη τον 8ο αιώνα (βάση αυτού θα πρέπει να θεωρείται ενάλιος αρχαιολογικός χώρος) που βρίσκεται σε βάθος 30-40 μέτρων.
Προσπάθησα στον παρακάτω χάρτη να σημειώσω μερικά από τα χωριά αυτά, όσο μπόρεσω να βρω πάνω-κάτω. Ελπίζω να εμπλουτιστεί στο μέλλον με τις τοποθεσίες των υπόλοιπων χωριών της λίμνης των Κρεμαστών.
Βυθισμένη πολιτεία (Λίμνη Κρεμαστών)
Λίμνη Πουρναρίου… Αποκάλυψη τώρα
Κατάδυση συλλόγου ερασιτεχνών αυτοδυτών ΤΥΘΗΣ στη λίμνη Κρεμαστών και Καστρακίου: [1] και [2]
Κάλλιο, το χωριό κάτω από τη λίμνη (Μόρνου)
Μόρνος: θαυμαστός υδάτινος κόσμος
Γουλιανοί “γίγαντες” στη λίμνη Πολυφύτου Κοζάνης
Recently I got two Maxtor 80GB disks. Sometimes the existing external hard disk runs out of space but can be easily sorted by erasing unnecessary data. As this external hard disk has been on constant use for around 4 years I thought it might be a good idea to use the additional Maxtors to build a software RAID and backup the existing external disk that stores the data.
My setup is pretty simple as all of the disks are external and connected via USB with the main system. The two 80GB disks are on RAID-1 (mirroring) syncing (via rsync in a cron-job) all the required data from the existing external hard disk. To keep syncing simple, I’ve created two single partitions (/dev/sdb1 and /dev/sdc1) on each of the RAID disks and then created a RAID-1 /dev/md0
I tried two different ways configuring the RAID, one with raidtools2 and one with mdadm.
mdadm is straight forward and can be used directly from the command file as below:
mdadm --create --verbose /dev/md0 --level=raid1 --raid-devices=2 /dev/sdb1 /dev/sdc1
mdadm: chunk size defaults to 64K
mdadm: /dev/sdb1 appears to contain an ext2fs file system
size=78156192K mtime=Thu Jan 1 01:00:00 1970
mdadm: /dev/sdc1 appears to contain an ext2fs file system
size=78156192K mtime=Thu Jan 1 01:00:00 1970
Continue creating array? (y/n) y
mdadm: array /dev/md0 started.
There is no need to explain the mdadm parameters as it is pretty much obvious what is happening. A look at the man page reveals all the possible options and what they stand for.
You can also check in /proc/mdstat to see if the RAID is running:
# cat /proc/mdstat
Personalities : [raid1]
md0 : active raid1 sdc1[1] sdb1[0]
167772160 blocks 64k rounding
The other way is to use raidtools and declare the raid setup in /etc/raidtab:
$ cat /etc/raidtab
raiddev /dev/md0
raid-level 1
nr-raid-disks 2
nr-spare-disks 0
chunk-size 4
persistent-superblock 1
device /dev/sdc1
raid-disk 0
device /dev/sdd1
raid-disk 1
And then the raid can be created:
# mkraid /dev/md0 handling MD device /dev/md0 analyzing super-block disk 0: /dev/sdb1, 78156193kB, raid superblock at 78156096kB disk 1: /dev/sdc1, 78156193kB, raid superblock at 78156096kB
Either way, raidtools or mdadm, you can then create and format partitions on /dev/md0 the normal way, using fdisk and mkfs.ext*. Once done so, the partitions can be mounted as would any other partition and syncing between the external storage disk and the raid can start.
I think that I’ll stick with mdadm as it is easier and more flexible than raidtools.
Some useful links:
How to replace a failed disk on Linux software RAID-1
mdadm: A new tool for Linux software RAID management
The Software-RAID HOW-TO
New release of the component. Copying from ChangeLog;
* Adding some daemon checking on the configure method in order to reload the xend daemon when needed.
* Introducing new method, ‘ConfHostResources’ to solve minor issue on configuring host’s configuration files and resources while restaring the component.
* The Restart() method removed. On ‘om xen restart’, the execution order is: Stop() – Configure() – Start()
* The cloning method will clone the specified template only for the first disk image that is specified in the disk list. That fixes bug #153. Man page updated as well.
* New resource is introduced to help with erasing a virtual machine. The ‘erase’ resource corresponds to every single virtual machine defined in the physical hosts’s profile and if set to ‘yes’ or ‘true’ it will erase the Xen configuration file for that guest as well as all hte disk images.
* Adding more info for the ‘profileurl’ resource.
RPM: lcfg-xen-0.99.9-1.noarch.rpm
SRPM: lcfg-xen-0.99.9-1.src.rpm
Defaults schema file: lcfg-xen-defaults-s1-0.99.9-1.noarch.rpm lcfg-xen-defaults-s1-0.99.10-1.noarch.rpm
I was trying to find an easy way to count the words in LaTeX documents. I guess there must be clients that do this for you, but I don’t use such a client and many times I’m using vi to edit LaTeX documents.
An way to go with it is to use a filter to strip as much as possible the TeX/LaTeX commands from the .tex file. This can be easily done with the detex command and the wc command:
detex report.tex | wc -w
Another way to filter the .tex document is to use untex in the same way as detex. This link describes lists also an alternative way of counting the words by converting the .dvi file to a plain text file:
dvips -o - report.dvi | ps2ascii | wc -w
However, this way doesn’t seem to filter the .dvi file that good so you may count more commands as words.
RSS Feed
Recent Comments