GlobusToolkit scripts

Three basic scripts for WS-GRAM, GSI-SSH and GridFTP that can made life with Globus Toolkit a bit easier.

WS-GRAM

#!/bin/bash

$HOSTNAME=

if [ "$1" == "start" ]; then

   if [ "$USER" == "globus" ]; then
     export GLOBUS_LOCATION=/usr/local/globus-4.2.1/
     export LD_LIBRARY_PATH=/usr/local/globus-4.2.1/lib
     export GLOBUS_HOSTNAME=$HOSTNAME

     cd $GLOBUS_LOCATION
     ./bin/globus-start-container &
     PID=`ps aux | grep DGLOBUS | grep -v grep | awk {'print $2'}`
     echo "WS-GRAM started with PID $PID"
   else
     echo "Only globus user can start the WS-GRAM service"
     exit 1;
   fi

elif [ "$1" == "stop" ]; then

   if [ "$USERID" == "globus" ] || [ $UID -eq 0 ]; then
     PID=`ps aux | grep DGLOBUS | grep -v grep | awk {'print $2'}`
     kill -9 $PID
     echo "WS-GRAM stopped"
   else
     echo "Only user globus and root can stop WS-GRAM"
     exit 1;
   fi

fi

GSI-SSH

#!/bin/bash

$HOSTNAME=

if [ "$1" == "start" ]; then

   if [ $UID -eq 0 ]; then
     export GLOBUS_LOCATION=/usr/local/globus-4.2.1/
     export LD_LIBRARY_PATH=/usr/local/globus-4.2.1/lib
     export GLOBUS_TCP_PORT_RANGE=20000,25000
     export GLOBUS_HOSTNAME=$HOSTNAME
     export GRID_SECURITY_DIR=/etc/grid-security/
     export X509_USER_KEY=/etc/grid-security/hostkey.pem
     export X509_USER_CERT=/etc/grid-security/hostcert.pem
     export X509_CERT_DIR=/etc/grid-security/certificates
     export GRIDMAP=/etc/grid-security/grid-mapfile

     $GLOBUS_LOCATION/sbin/sshd &
     PID=`ps u | grep $GLOBUS_LOCATION/sbin/sshd | grep -v grep | awk {'print $2 '}`
     echo "GSISSH started with PID $PID"
   else
     echo "Only root can start GSISSH service"
     exit 1;
   fi

elif [ "$1" == "stop" ]; then

   if [ $UID -eq 0 ]; then
     PID=`ps aux | grep globus | grep sshd | grep -v grep | awk {'print $2 '}`
     kill -9 $PID
     echo "GSISSH stopped"
   else
     echo "Only root can stop GSISSH"
     exit 1;
   fi

fi

GridFTP

#!/bin/bash

$HOSTNAME=

if [ "$1" == "start" ]; then

   if [ $UID -eq 0 ]; then
     export GLOBUS_LOCATION=/usr/local/globus-4.2.1/
     export LD_LIBRARY_PATH=/usr/local/globus-4.2.1/lib
     export GLOBUS_TCP_PORT_RANGE=20000,25000
     export GLOBUS_HOSTNAME=$HOSTNAME
     export GRID_SECURITY_DIR=/etc/grid-security/
     export X509_USER_KEY=/etc/grid-security/hostkey.pem
     export X509_USER_CERT=/etc/grid-security/hostcert.pem
     export X509_CERT_DIR=/etc/grid-security/certificates
     export GRIDMAP=/etc/grid-security/grid-mapfile
     export GLOBUS_CALLBACK_POLLING_THREADS=1

     $GLOBUS_LOCATION/sbin/globus-gridftp-server -p 2811 -d 255 -l $GLOBUS_LOCATION/var/gridftp.log \
     -log-transfer $GLOBUS_LOCATION/var/gridftp-transfers.log -hostname $GLOBUS_HOSTNAME &
     PID=`ps aux | grep globus-gridftp-server | grep -v grep | awk {'print $2 '}`
     echo "GridFTP started with PID $PID"
   else
     echo "Only root can start GridFTP service"
     exit 1;
   fi

elif [ "$1" == "stop" ]; then

   if [ $UID -eq 0 ]; then
     PID=`ps aux | grep globus-gridftp-server | grep -v grep | awk {'print $2 '}`
     kill -9 $PID
     echo "GridFTP stopped"
   else
     echo "Only root can stop GridFTP"
     exit 1;
   fi

fi

You could of course combine them all together in a single script but I’d rather prefer to keep them separate and “clean”, executing them depending on what services need to run/stop.

Basic HTTP authentication with Java

Opening a URL in Java is pretty much straightforward by using the java.net.URL class. If the URL trying to be accessed is password protected (by Apache’s AuthType for example), you will need to authenticate before trying to open the URL. The easiest way to authenticate is by using the Authenticator (java.net.Authenticator) class which is available on Java 1.2 and higher:

final String theUsername = username;
final String thePassword = password;

Authenticator.setDefault(new Authenticator() {
     protected PasswordAuthentication getPasswordAuthentication() {
           return new PasswordAuthentication(theUsername, thePassword.toCharArray());
      }
});

twitterFetcher 0.2

Description here. Updated with support to fetch the tweets of the users somebody follows.

java twitterFetcher <feed url> <file> <username> <password>

Finally delivered!

After almost a year, Honours Project delivered and presented!

The lcfg-xen architecture

(click to enlarge)

Some more information can be found on the poster summary published at LISA ‘09, “Integrated Configuration of Virtual Infrastructures“, as well as on this presentation from the recent LCFG Users Day ‘09, “LCFG meets Virtualisation“.

Do robots/intelligent agent need to know about ethics?

Introduction

Thousand of years have passed since the appearance of the mechanical servants of the Greek god of fire, Hephaestus, to the term robot by the Czech novelist Karel Capek, and from the novelist Isaac Asimov who places robots among humans, to the 21st century HONDA humanoid robot, ASIMO. Despite the temperament of people that changes through the ages, the technologically and the, debatable, spiritually progress of the society, there are questions that have been answered and at the same time practice shows that they remain unanswered.

Nowadays, robot-like behaviour exists within products we use everyday. Computers, mobile phones, cars and airplanes make our life easier and enjoyable, but at the same time, the same underlying technology is used by governments to control their people, to produce political, or ideological, propaganda, and if necessary turn this technology against their own people, or others.

Intelligent, and non-intelligent, machines are developed and programmed by humans, and the human nature is both good and evil. In the next few decades to follow, perhaps the novels of Isaac Asimov will be reality. Humanoid robots being among us and dealing with them as we deal with normal people. How will those humanoid machines act and “think”? Will they be able to develop their own “personality” or they will be given a fixed “personality” depending on their creator/developer/engineer believes? In the chapter of Genesis in Bible, it is said that God created man in his image and model. Keeping a clear distance from religion, are the years to follow the years that this idea will become reality? Machines being developed and acting in image and model of their creators?

Do robots/intelligent agents need to know about ethics?

Isaac Asimov defined the Three Laws of Robotics. However, these laws have been commented to be laws for slaves, while we would like robots among us to behave more human-like, and therefore being ethical in the same way as people are (James Gips, 1991). Using the same underlying idea, that we should interact with robots in a human-like way, Richard Epstein have written a book in 1997 named “The Case of the Killer Robot”. In this book, Epstein develops a fictional scenario, and he tells the story of how a programming error during the development of a robot, led to the death of the robot operator (Richard Epstein, 1997). A number of other scenarios by Epstein followed this book and they presented the “ethical and social implications of artificial intelligence and virtual reality within domains such as Business and Commerce, Human Relationships, Privacy and Personal Security, Philosophy and Thought, Medicine, Government and Law, Education, Culture and Arts, Psychology, Spirituality, and Ethics and Values” (Richard Epstein, 1997). The importance of the later writings lay to the fact that they do not investigate the “killer” behaviour of a robot, which is a very common subject in fictional concepts for novels and movies, but the way robots could affect the society, their relation with humans as members of sectors that exist and evolve within the society. In our days, we use intelligent agents everyday, from our mobile phones to our washing machines and from the airplanes to normal cars. Artificial intelligence systems are used to look into the past and re-create it in order to understand it. This is an ideal use of artificial intelligence for exploring our past, but the problem that arises is when artificial intelligence is used to replace human creativity (Richard Epstein, 2000) and human relationships (T. Bickmore & R. Picard, 2005).

Despite the negative impacts that robots may impose, well programmed intelligent agents can play a positive role is society. Researchers bring as an example the use of intelligent agents in the training and therapy procedure of autistic persons that need to follow an intensively repetitive process where “agents could be used to potentially augment such interactions”. Moreover, intelligent agents could provide help with personal networking where they could contact persons on behalf of their owner, similar to a secretary. (T. Bickmore & R. Picard, 2005).

To follow and understand the concept in answering the question if robots need to know about ethics we have to consider a very similar question: do humans need to know about ethics? Apparently yes. The issue of ethics has been discussed since ancient times by the Greek philosophers and especially Aristotle in his writing “Nicomachean Ethics”.

Aristotle differentiates the intellectual and moral virtues. He suggests that intellectual virtues can be taught while moral virtues are obtained by living a correct life, something that comes by experience. The question is how can a robot understand that. It all depends on its implementation, on the algorithms that will be used for the robot to determine if what it does is correct and ethical or not (James Gips, 1991).

Indeed, this consideration is true. We consider that every person develops his personality, his ethical and moral values, depending on its surrounding environment, be it the close one of the family, or the extended one of the society. A robot could, in the same way, decide “on its own” and understand from its surrounding what is ethical and what is not. But in order to do so the engineers and the developers that have programmed the robot, or the intelligent agent, will need to have implemented an algorithm for the system to be able to determine what is ethical and what is not. The problem that arises is that the engineer may develop an algorithm for his own purposes, according to his personal ethics and values, or of course according to his employer’s. Robots however, should act for the common good of the society. Before performing an action, the system should evaluate the current situation, the consequences of the action that will perform, how much good this action will deliver and what is the best action within a range of options that would deliver the most good. Those decisions need to be based on the implemented algorithms and the patterns that are provided to the system in order to evaluate situations and decisions. Intelligent decision making systems targeted for specific purposes do not require to consider that much ethics and moral values. Nevertheless, they need to evaluate properly the specific situation they operate on, analyse the data, simulate the operation and decide upon the probable results. Think of medical systems and weapons management systems for instance. The appliance of these systems and that of ethical robots may differ a lot in some aspects, however, both are intelligent systems that are developed to evaluate conditions based on circumstances, on data and on outcome of a specific action. In both cases, the development is solely based on their appliance. An issue that has been raised by researchers under this concept, is the use of “rational agents to provide care, understand and empathy while machines can not have emotions on their own” (Picard &Klein, 2002). While this is not something directly connected to the ethics of the intelligent agents, it is directly connected with its development and implementation. What it considers as right and wrong.

With no doubt, robots and intelligent agents must be aware of ethics and moral values. In order to get ethical robots we need to have ethical engineers and ethical scientists that will design, develop and implement ethical algorithms. Education and common understanding are the keys behind such decisions and development, which unfortunately is difficult to get at a high level in the age of intolerance and lobbies. Perhaps, the question that we should ask is not if robots and intelligent agents need to know about ethics, but how actually will scientists and engineers will be able to design and implement ethical intelligent machines that have a kind of understanding of their actions. Actions that should reside within globally accepted standards. Even then, would such systems be acceptable by those who decide for ourselves and would they find a place within the society? It certainly needs time to find out, and judging from the current situation, a pure idea can easily be converted and used for money, personal advantage and control over people.

Conclusions

Technological progress is integral connected with scientific progress. Science, and therefore technology, primarily want to model the world, understand it, help the mankind, make our lives easier and more enjoyable. We should not allow the fear of misuse to prevent technological progress but we should target at the educational concerns and purposes that lead to misuse, and how these can be decreased. The use of technology is subject to human behaviour and human ethics. These derive from within the society, the education and the environment of every single individual. As far as people and the society are ethical, their creations will also be.

Nuclear weapons are pure physics, mathematics and chemistry. Nobody though turns against the sciences themselves, leaving aside fanatical religious blindness. People forget that, in fact, technology combines these sciences together, and speaking of artificial intelligence, then cognitive science, sociology and psychology are involved as well. Technology must be seen as the mean to move forward, the “tool” that is able to combine everything in order to understand everything, following the idea of the Greek philosopher Heracleitos who once said that “Wisdom is to understand everything with the help of everything”.

References

James Gips, Towards the Ethical Robot. Android Epistemology, MIT Press, 1991

Luciano Floridi, Information Ethics, its Nature and Scope, SIGCAS Computers and Society, Volume 36, No. 3, September 2006

Picard, R. and Klein, J. 2002. Computers that recognize and respond to user emotion: theoretical and practical implications. Interact. Comput. 14, 141–169

Richard G. Epstein, 1997. The Case of the Killer Robot, New York: John Wiley and Sons

Richard G. Epstein, Stories and Plays About the Ethical and Social Implications of Artificial Intelligence, Intelligence Fall 2000, MIT Press

Ronald C. Arkin, Governing Lethal Behavior: Embedding Ethics in a Hybrid Deliberative/Reactive Robot Architecture PART I: Motivation and Philosophy, HRI’08, 2008, Amsterdam, Netherlands

Timothy W. Bickmore, Rosalind W. Picard. Establishing and Maintaining Long-Term Human-Computer Relationships. ACM Transactions on Computer-Human Interaction, Vol. 12, No. 2, June 2005, Pages 293–327

Yueh-Hsuan Weng, Chien-Hsun Chen, Chuen-Tsai Sun, The Legal Crisis of Next Generation Robots: On Safety Intelligence, ICAIL ’07 June 4-8, Palo Alto, CA US

Linux system stressing

After reading this post, I’ve start having a look at a few system stressing tools. Some useful links:

openMosix stressing
Linux Test Project
stress project

IBM resource: Stress-testing the Linux kernel

There’s plenty of information on every link with uses and examples.

Linux buffer cache state

Following Faidon’s comment on an earlier post, I came across  this informative site concerning Linux’s buffer cache state. In a nutshell, the following code will release all the cached data of a specified file.

#define _XOPEN_SOURCE 600
#include <unistd.h>
#include <fcntl.h>
int main(int argc, char *argv[]) {
    int fd;
    fd = open(argv[1], O_RDONLY);
    fdatasync(fd);
    posix_fadvise(fd, 0,0,POSIX_FADV_DONTNEED);
    close(fd);
    return 0;
}

There are some useful samples and examples on the mentioned web page. posix_fadvise description here.

lcfg-xen-1.0.1

Finally got to release 1! Not many new features but more or less stable as it seems. The only addition to this is the support for defining and configuring groups of virtual machines, forming eventually virtual clusters.
New resources are introduced that can replace the individual resources for every virtual machine that is member of the virtual group.

!xen.virtualgroup		mADD(cluster1)
!xen.members_cluster1  		mADD(node1 node2 node3)
!xen.maxmem_cluster1    	        mSET(2048)
!xen.memory_cluster1    	        mSET(1024)
!xen.groupvcpus_cluster1   	mSET(4)
!xen.groupcpus_cluster1     	mSET(4)

The resource definition above will configure each member of the virtual group according to the specified resources.

Network configuration can also be performed per group basis:

!xen.groupnettype_cluster1	mSET(bridge)
!xen.groupbridge_cluster1	mSET(xenbr2)
!xen.groupscript_cluster1	mSET(my-custom-bridge)

The default values are “512″ for maximum and initial memory, “1″ for VCPUs and pinned physical CPUs, “bridge” for network type, “xenbr0″ as the default network bridge interface and “vif-bridge” the default network script.

EDIT: Issues with the schema file. New release 1.0.3-1 solves it.

RPM: lcfg-xen-1.0.1-1.noarch.rpm lcfg-xen-1.0.3-1.noarch.rpm
SRPM: lcfg-xen-1.0.1-1.src.rpm lcfg-xen-1.0.3-1.src.rpm
Default schema file: lcfg-xen-defaults-s1-1.0.1-1.noarch.rpm lcfg-xen-defaults-s1-1.0.3-1.noarch.rpm

Fetching and saving my Twitter tweets

I have heard quite a few people complaining that tweets on Twitter are erased after some period and they can’t be saved without hassling. A quick googling revealed some options but I can’t say that there was anything satisfying, at least for what I was looking for. The way I’d like to save my tweets would be directly to my system, in a simple text file which I’ll be able to access from any other system if I need so.

To meet this “personal requirements” I implemented a short application in Java. It will fetch the specified feed and save your entries in a specified file. On the next fetch operation, if the user specifies the same destination file, it will just update the file with the new tweets from the feed. The most effective way to get that automatically done is to use it as a daily cronjob.

I have the following entry in my crontab:

$ crontab -l | grep fetch
41 23 * * * ~/xNIX/scripts/backups/fetchTweets.sh

and the scripts all it does is:

$ cat fetchTweets.sh
cd ~/xNIX/scripts/backups
java twitterFetcher http://twitter.com/statuses/user_timeline/17578627.rss ~/myTweets

I wouldn’t consider this as a proper Twitter backup application but more like a hack to get the work done without touching the Twitter API or using an external service that would require more manual work to get the tweets on my system. Also, as it uses the RSS Feed, it’s not suitable for those who tweet massively throughout the day.

Get twitterFetcher.java