VTK re-compile (for Python) with PowerCrust algorithm support

The following was performed with VTK 5.8 and Python 2.7.8 on Debian Wheezy x86_64.

1. Get the VTK source-code: http://vtk.org/VTK/resources/software.html
2. Get the Tim Hutton’s PowerCrust C++ source-code: https://github.com/timhutton/vtkpowercrust
3. Unzip both archives
4. Copy the C++ files (.cxx & .h) into VTK/Hybrid
5. Edit VTK/Hybrib/CMakeLists.txt and add the PowerCrust .cxx file
6. cd VTK; mkdir build; cd build; ccmake ..
7. Set desired installation directory
8. Turn on BUILD_SHARED_LIBS, VTK_USE_RENDERING, VTK_WRAP_PYTHON
9. While on ccmake first configured [c] and then generate [g]
10. make && make install
11. cd Wrapping/Python; python setup.py install
12. Assuming all go well export the VTK libraries: export LD_LIBRARY_PATH=/<installation_path>/lib/vtk-5.8

Advertisement

Remote cPanel backup via FTP

Here is the scenario:

– Domain that needs to be backed up (public_html, mail, database).
– Only access available via cPanel and FTP.
– Need automated daily full backup.

Solution:

– Take full backup on server and compress its contents (cronjob on remote server)
– Download via FTP the compressed file to the local machine (cronjob on local machine).
– Delete compressed file.

There might be tools that do this, although I didn’t come across any while looking on the web. I implemented a simple solution based on bash scripts that gets the task done.

– backup.sh: Goes on the remote server to run as a cronjob.
– ftp.sh: Runs locally (on the machine that will store the backup) as a cronjob.
– .netrc: Defines the FTP account details (stored in /home/$user/.netrc)

Gmail SMTP relay with Postfix

The following configuration has been performed in Debian but should apply in any Linux distro with Postfix installation. The task quite simple: use Gmail as SMTP relay for outgoing email traffic.

1) At first place I had to install the following packages and turn off sendmail:

apt-get install postfix mailutils libsasl2-2 ca-certificates libsasl2-modules
service sendmail stop

2) Then edit the configuration file at /etc/postfix/main.cf by adding the following options at the end of the file:

# Gmail SMTP relay
relayhost = [smtp.gmail.com]:587
smtp_use_tls=yes
smtp_sasl_auth_enable = yes 
smtp_sasl_password_maps = hash:/etc/postfix/sasl_passwd
smtp_tls_CAfile = /etc/ssl/certs/ca-certificates.crt
smtp_sasl_security_options =

3) Create the authentication file at /etc/postfix/sasl_passwd and set the right permissions:

[smtp.gmail.com]:587    <username>@gmail.com:<password>
chmod 400 /etc/postfix/sasl_passwd

Make sure the file is owned by the user who runs the Postfix daemon.
4) Reload Postfix:

service postfix reload

5) Confirm the configuration is actually working by sending a test mail:

echo "Test mail from postfix" | mail -s "Test Postfix" pkritikakos@isbs.gr

The sender of the email should be the given Gmail account. Also, in the “Sent” folder of the Gmail account you should see the email you have sent with Postfix.

If your domain is using Gmail’s infrastructure for handling emails, you can replace @gmail.com with your domain and use the corresponding account details.

NOTE: This configuration will be sending any system message as “username@gmail.com” and therefore is not advised to be used in multi-user environment.

Android SDK x86_64 – adb, libraries, emulator issues

I have installed the latest Android SDK on Fedora 17 x86_64 version. I’ve chosen the bundle for my platform, expecting it would work straight away, but that was not the case. The adb (Android Debug Brdige) needs to make use of the of libraries for the i686 version, such as glibc, ncurses and tdc++. Errors look as usual:

/lib/ld-linux.so.2: bad ELF interpreter: No such file or directory
error while loading shared libraries: libncurses.so.5: cannot open shared object file: No such file or directory

In addition to these I required to install libz for the SDK to perform correctly without any errors. Simply linking from /lib64 to /lib wouldn’t work, as adb would exit with segmentation fault, something expectable since it deals with different architecture’s libraries.  The following command will install all the required packages (with a couple of dependencies) to get that bit working:

yum install glibc.i686 ncurses0libs.i686 libstdc++.i686 libzeitgeist.i686

In addition to the libraries issues for the adb, the emulator binary that lunches the Android emulation images is also targeted for i686. The bundle comes with emulator-x86_64 but when you lunch an AVD (Android Virtual Device) through the ADT Manager, it wouldn’t start the emulator as the binary was failing to execute and wouldn’t pick automatically the x86_64 version. In this case a simple link would work. I renamed the initial i686 binary to emulator-32, and linked emulator-x86_64 to emulator. The actual AVD would then start as it should and operated normally, being able to install applications via the Android SDK.

An ARM comparison: Raspberry Pi vs OpenRD Client

Bellow there’s a comparison between Raspberry Pi (Model B) and OpenRD Client, two low-power ARM SoC (System on a Chip), running Linux (Rasbian and Fedora respectively). I used the EEMBC CoreMark benchmark with default settings for 100000, 1000000 and 2000000 iterations. For each set, each device keep its performance with the same rate of iterations per second. The difference between the devices is at, an average level of, 520 iterations per second. That translates to 1.4x faster in favour of the OpenRD Client.

While OpenRD Client performs better, in terms of efficiency, Raspberry Pi seems to doing much better. In terms of CPU, Raspbeery Pi uses a 700MHz (with the possibility of setting it up to 1GHz) ARMv11 processor while OpenRD Client an older 1.2GHz Marvell Sheeva (ARMv5TE). ARMv11 supports an FPU unlike ARMv5 and that is one of the reasons that Raspberry Pi performs quite well despite its lower specifications. In terms of memory, Raspberry Pi uses 256MB DDR2 800MHz (newer models have 512MB RAM), in share with its relatively powerful GPU, while OpenRD Client uses 512MB DDR2 800MHz. OpenRD Client also provides Gigabit Ethernet as well as many more I/O interfaces. Also, OpenRD has preloaded OS, unlike Raspberry Pi that boots from an SD card. During this benchmark, Raspberry Pi was running Rasbian “Wheezy” (unofficial Debian port to armhf) while OpenRD Client Fedora 8.

(Raspberry Pi diagram – http://www.raspberrypi.org/wp-content/uploads/2012/04/Raspi_Iso_Blue.png)

(OpenRD Client Hardware Block diagram – http://www.globalscaletechnologies.com/skins/skin_1/images/OpenRD-Client_back.PNG)

HFS+ on Linux

My MacBook Pro seems to have died. Its hard disk holds lots of data that have not been backed up anywhere else. The important data have been mostly backed up but latest changes haven’t been backed up. Other than that, I had to check the hard disk for bad sectors etc and since I don’t have another Mac to test it on I had to do it on my Fedora box. My system (Fedora 15) would automatically detect and mount the HFS+ hard disk on read only mode. It was also missing the fsck tool for HFS+ partitions. Getting the fsck tool for HFS+ required downloading and installing hfsplus-tools via yum.However, fsck.hfs will not allow you, except if you use the –force option, to scan the partition if it has journalling on, which is the case with HFS+ partitions by default.

But how would I turn off journalling when I don’t have a Mac system to attach my disk on? After a couple of minutes I came across a post on the Ubuntu forum which linked to this blog. The author has a C code that turns journalling off. A fixed version of this code can be found here. The compiled code results in an executable which gets as its only argument the partition that you want to get journalling off.

# gcc journalling_off.c -o journalling_off
# ./journalling_off /dev/sdg2

Next step was to perform the fsck check on the target disk

# fsck.hfs /dev/sdg2
** /dev/sdg2
** Checking HFS Plus volume.
** Checking Extents Overflow file.
** Checking Catalog file.
** Checking multi-linked files.
** Checking Catalog hierarchy.
** Checking Extended Attributes file.
** Checking volume bitmap.
** Checking volume information.
** The volume OS X appears to be OK.

Disk looks OK. Next step is to mount it with read-write permissions:

# mount -t hfplus -o rw,user /dev/sdg2 /mnt/osx

Next issue encountered was the different UIDs between my account on the OS X system and that on the Linux system. Therefore, next step was to change the UIDs under the whole user directory on the OS X disk so I could access without problem with write permissions from my Linux box:

# find panoskrt/ -uid 501 -exec chown panoskrt {} \;

HPC-Europa2 Virtual Cluster liveDVD – milestone6 (final release)

The JRA2 team is proud to announce the release of HPC-Europa2 live DVD milestone 6. You can download milestone 6 ISO image (compressed with size 1.7 GB) from the below links:

CINECA (Italy) – http://www.hpc-europa.eu/?q=node/142
EPCC (Scotland) – http://hpce2jra2.epcc.ed.ac.uk/milestone6.iso.gz
HLRS (Germany) – https://fs.hlrs.de/projects/live-ptd/iso/milestone6.iso.gz

Release Note for Milestone 6
============================
The Virtual Cluster liveDVD:
* provides tools and libraries that are installed and used at the partner sites.
* emaulates the environment of the platforms available at HPC-Europa centers.
* allows hands-on building of a virtual cluster.

The liveDVD contains training and dissemination material published by the
HPC-Europa2 project, including material from the partner centers such as slides,
video tutorials and example code. Optional access to a source code repository enables
the migration of code and data from the Virtual Cluster to the user account created
on the HPC-Europa machines.

The liveDVD is based on SliTaz GNU/Linux, a free operating system, working completely
in RAM and booting from a removable media. The file size of the SliTaz GNU/Linux
3.0 ISO image is 29 MB. Hence, it leaves plenty of space for adding tools, libraries
and training material into this liveDVD. The SliTaz system is quick, responsive, and clean.
A lightweight and elegant desktop is included as well in the 29MB ISO image.

The milestone 6 ISO image has roughly the size of 3.8 GB including several tools
and libraries, as well as training material. Moreover, it contains a documentation
of how to build a virtual cluster using VirtualBox, and HPC-Europa2's Science and
Supercomputing in Europe research highlights for 2009 and 2010. Finally, this milestone
release addresses several bug fixes, such as firefox with flash support and
build scripts for Torque.

Since the previous release (milestone 5), existing packages have been updated, such as:
* MPICH2 to version 1.4.1p1
* Firefox to version 9.0.1 with flash support.
* Paraver to support the updated version of MPICH2.

In addition, this release has added several new packages:
* lava-1.0.6: Platform Lava - an open source version of Platform Load Sharing Facility (LSF).
* info (texinfo-4.13a) and man pages (man-db-2.6.0.2 and man-pages-3.35) for Linux documentation.
* Abiword and Gnumeric for office word documents and spreadsheets.
* Compiler wrappers (based on gcc and gfortran) to emulate Intel, IBM and NEC SX compilers.

You can download milestone 6 ISO image (compressed - size 1.7 GB) from the below link:
    http://www.hpc-europa.eu/?q=node/142

System Requirements
===================
* A laptop or computing with a DVD drive.
* Intel or AMD processor with a hardware virtualization support when using KVM
  and libvirt. Otherwise, this liveDVD can be run under VirtualBox or VMware or similar.

Known Issues
============
* PGI compilers require a license file in order to work (not included in this milestone).
* Totalview requires a license file in order to work (not included in this milestone).
  For requiring a demo license visit:
  http://www.roguewave.com/products/totalview-family/totalview.aspx
* On a MacBook Pro laptop, the mousepad does not work. Need to use an USB mouse
* Resizing video screen on VLC causes an unexpected exit of the program.
* VLC has no sound for viewing mkv videos.
* VLC cannot stream files or view incoming streams.

Acknowledgement
===============
This work was carried out under the HPC-EUROPA2 project (project number: 228398),
with the support of the European Community - Research Infrastructure Action of the FP7.

HPC-Europa2 Virtual Cluster liveDVD – milestone 5

The fifth release of the HPC-Europa2 Virtual Cluster liveDVD is available (milestone 5) for download:

CINCENA (ITA) - http://www.hpc-europa.eu/files/iso/milestone5.iso.gz
EPCC (UK) - http://hpce2jra2.epcc.ed.ac.uk/milestone5.iso.gz

Release Note for Milestone 5
============================
The Virtual Cluster liveDVD:
* provides tools and libraries that are installed and used at the partner sites.
* emulates the environment of the platforms available at HPC-Europa centers.
* allows hands-on building of a virtual cluster.

The liveDVD contains training and dissemination material published by the HPC-Europa2 project, including material from the partner centers such as slides, video tutorials and example code. Optional access to a source code repository enables the migration of code and data from the Virtual Cluster to the user account created on the HPC-Europa machines.

The liveDVD is based on SliTaz GNU/Linux, a free operating system, working completely in RAM and booting from a removable media. The file size of the SliTaz GNU/Linux 3.0 ISO image is 29 MB. Hence, it leaves plenty of space for adding tools, libraries and training material into this liveDVD. The SliTaz system is quick, responsive, and clean. A lightweight and elegant desktop is included as well in the 29MB ISO image.

The milestone 5 ISO images has roughly the size of 3.7 GB including several tools and libraries, as well as training material.

Since the previous release (milestone 4), existing packages have been updated, such as:

* Open MPI to version 1.5.4
* Midori with flash support
* Firefox to version 7.0.1

In addition, this release has added a new package to support EVO (http://evo.caltech.edu/)

System Requirements
===================
* A laptop or computing with a DVD drive.
* Intel or AMD processor with a hardware virtualization support when using KVM and libvirt. Otherwise, this liveDVD can be run under VirtualBox or VMware or similar.

Known Issues
============
* Totalview requires a license file, which is no included, in order to work.
For requiring a demo license visit:
http://www.roguewave.com/products/totalview-family/totalview.aspx
* On a MacBook Pro laptop, the mousepad does not work. Need to use an USB mouse
* Resizing video screen on VLC causes an unexpected exit of the program.
* VLC has no sound for viewing mkv videos.
* VLC cannot stream files or view incoming streams.
* Flash cannot be viewed under Firefox. Use Midori instead.

Acknowledgement
===============
This work was carried out under the HPC-EUROPA2 project (project number: 228398),
with the support of the European Community – Research Infrastructure Action of the FP7.

Ethernet device rename

I have been looking recently to find a way of creating a bridge interface on SliTaz in order to get KVM/libVirt working with the expected br0 interface. Unfortunately, the network configuration files of SliTaz do not provide any information, neither the documentation. A post on the forum didn’t get any replies either. Rather than trying figuring how a proper bridge interface would work on SliTaz, I thought of renaming eth0 to br0, as the later would be expected by KVM/libVirt. As far as I know the following process would need to be followed for other distros as well as there is no specific way of renaming an interface other than using udev rules.

On SliTaz, the required udev rule is defined /etc/udev/rules.d/70-persistent-net.rules. The default entry looks like the following (where Xs the MAC address):

SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="XX:XX:XX:XX:XX:XX",
ATTR{dev_id}=="0x0", ATTR{type}=="1", KERNEL=="eth*", NAME="eth0"

The generated rule matches the eth0 interface with the specific network card. The required change is the renaming of the NAME field to br0. If the targeted system is a virtual machine with changeable MAC address, or most likely a live distro like in my case, the ATTR field, that specifies the MAC address, would need to be changed as well. Changing it to 0* would work and on the next boot the device should be renamed. The final udev rule:

SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="0*",
ATTR{dev_id}=="0x0", ATTR{type}=="1", KERNEL=="eth*", NAME="br0"

There is one more addition to do on SliTaz and that is to rename the default ethernet device in /etc/network.conf from eth0 to br0. That would allow higher level applications and scripts to look for br0 rather the non-existant eth0.

Size matters

…for power consumption.

OpenRD Client with Marvel Sheeva 88F6281 (7 Watt idle, single core)

Intel Atom D525 on Mini-ITX bundle (45 Watt idle, dual-core hyper-threading).

Dell PowerEdge r610 with Intel Xeon E5507 (119 Watt idle, double quad-core)