dcsimg

Performing Community Service

One of the most famous stories of Linux advocate Eric S. Raymond is that hackers spontaneously pitched in and bought him a $1,000 iBook as a token of their esteem. While that story is amazing, it's not unusual or uncommon -- Raymond's geeky gift is just one of the concrete examples of the remarkable philanthropy of the hacking community, which Raymond calls a "gift culture," a society in which your reputation is made by what you give away rather than by what you keep.

One of the most famous stories of Linux advocate Eric S. Raymond is that hackers spontaneously pitched in and bought him a $1,000 iBook as a token of their esteem. While that story is amazing, it’s not unusual or uncommon — Raymond’s geeky gift is just one of the concrete examples of the remarkable philanthropy of the hacking community, which Raymond calls a “gift culture,” a society in which your reputation is made by what you give away rather than by what you keep.

This edition of Java Matters ends a remarkable year of Java programmer generosity. In 2003, this column described some truly great open source projects, including the Apache XML-RPC Web services platform; the Velocity template rendering engine; the Struts Web application framework; the JSP Standard Tag Library (JSTL), and the XOM XML processing library. The first four projects came from the Apache Software Project, whose Jakarta team has released more than 20 high-quality Java projects; the fifth was released by Java and XML book author Elliotte Rusty Harold. These five projects — and many others like them — demonstrate that open source is rapidly becoming the first place Java programmers do look and should look for class libraries, useful code, and inspiration.

It’s impossible to write about the open source Java community without noticing the altruistic nature of its programmers. People give freely of their work, time, and talent. So, in the spirit of the holidays and of open source, this month’s Java Matters is a humble attempt to give something back.

Introducing JStorageSystem

JStorageSystem is a new, open source, Java implementation of the XML Storage System, an API for exchanging files and publishing weblogs using the XML-RPC and SOAP web services protocols. Created by UserLand Software and used extensively in Radio UserLand — a commercial database, weblog publisher, and Internet development environment — the XML Storage System API is documented at http://www.soapware.org/xmlStorageSystem.

Radio UserLand publishes content by employing XML-RPC or SOAP to upload rendered web pages and other files to a Radio Community Server. (The server is free, but isn’t available for Linux.) Publishing in this manner has been given the name upstreaming. Files are uploaded without modification or are rendered into web pages dynamically during transfer using text files, XML files, and executable scripts. XML-based configuration files enable content to be routed to different XML-RPC, SOAP, and FTP servers on a per-folder and per-protocol basis. And because Radio UserLand produces static web content, Linux hosting requires only the Apache web server and an XML-RPC or SOAP server that supports the XML Storage System API.

JStorageSystem is an XML-RPC server that enables Linux to be used as a hosting platform for thousands of weblogs published with Radio UserLand and other clients that use its XML API. JStorageSystem uses Apache XML-RPC and the Log4J logging library. (Apache XML-RPC was covered in the January 2003 Java Matters column, available online at http://www.linux-mag.com/2003-01/java_01.html. Log4J was covered in the August 2002 column, available at http://www.linux-mag.com/2002-08/java_01.html.)

Due to space constraints, not all of the JStorageSystem source code appears here. However, you can download the code in its entirety from http://www.linux-mag.com/downloads/2003-12/java or from the author’s personal web site at http://www.cadenhead.org/workbench/code and follow along. The code has been released under the terms of the GNU Public License.

JStorageSystem has been implemented as two properties files and three classes, JStorageServer, JStorageHandler, and JStorageUser.

* JStorageServer is a Java application subclassed from WebServer, an XML-RPC server class in the org. apache.xmlrpc class library.

* JStorageHandler is a class that implements the methods in the XML Storage System API.

* And JStorageUser is a Java class that represents a user account on the server.

Let’s look at each of the components in some detail.

Inside JStorageServer

The JStorageServer class inherits much of its behavior from WebServer to create and start an XML-RPC web server. The process occurs in three steps:

1. Call the constructor with the server’s port number as an argument, as in:


jss = new JStorageServer(Integer.parseInt(arguments[1]));

The port number can be anything from 1024 to 49151, as long as the port isn’t already in use.

2. Add one or more handler classes with public methods that can be called remotely:


JStorageHandler handler = new JStorageHandler(jss);
jss.addHandler(“xmlStorageSystem”, handler);

Methods are called remotely using the first argument to addHandler() followed by a period and the name of the method. For example, JStorageHandler includes a registerUser() method to start an account on the server. To make this request, XML-RPC clients call xmlStorageSystem.registerUser(). (Every public method in the handler class is callable via XML-RPC, so care must be taken to ensure that other procedures aren’t made available inadvertently.)

3. Call the server’s start() method, causing it to bind to the specified port and wait for clients to make requests:


jss.start();

This last step is a recent change. Earlier versions of WebServer took requests without being started explicitly.

JStorageServer includes nine instance variables for the server and two for Log4J. Values for the variables are loaded from a properties file, a text file that contains a set of names and values, where each name and value tuple appears on its own line separated by an equal sign.

For example, the line maxFileSize=1048576 limits the size of a single file that can be uploaded to a user’s weblog.

All of the server’s instance variables are private and use accessor methods, so maxFileSize requires getMaxFile Size() to retrieve the value and setMaxFileSize(int) to set it.

The java.util.Properties class loads properties from a file into a Properties object. Each property can be loaded by calling the object’s getProperty(String, String) method. The first argument should be the name of the property; the second argument, which can be omitted, should be a default value to use if the property isn’t found.

Here’s the code in the configure() method to create a Properties object and assign a value to the maxFileSize instance variable:


// file is defined by a command-line argument
File propFile = new File(file);
FileInputStream fis = new FileInputStream(propFile);
Properties props = new Properties();
props.load(fis);
setMaxFileSize(Integer.parseInt(
props.getProperty(“maxFileSize”, “1048576″)));

The getProperty() method returns strings, so an integer variable such as maxFileSize must be converted.

By default, JStorageServer stores user information in a MySQL database, but can also be configured with the properties file to use any database with a JDBC driver.

JStorageServer also includes three methods that are used by the other classes:

* getDatabaseConnection() returns a connection to the server’s JDBC database.

* saveFile() saves a file to a user’s directory.

* deleteFile() deletes a file from a user’s directory.

saveFile() and deleteFile() are shown in Listing One. The getDatabaseConnection() method makes use of the databaseUrl and databaseClass instance variables, which are loaded from the properties file. Here’s an example for a MySQL database using the com.mysql.jdbc.Driver:




Listing One: A portion of the JStorageServer class


public void saveFile(JStorageUser user, String filename,
Object contents) throws Exception
{
File file = new File(getRootFolder() + File.separator +
user.getName() + File.separator + filename);
if (contents instanceof byte[]) {
byte[] data = (byte[]) contents;
if (data.length > getMaxFileSize()) {
throw new IOException(“File too large.”);
}
FileOutputStream fos = new FileOutputStream(file);
for (int i = 0; i < data.length; i++) {
fos.write(data[i]);
}
fos.close();
} else {
String data = contents.toString();
if (data.length() > getMaxFileSize()) {
throw new IOException(“File too large.”);
}
BufferedWriter bw = new BufferedWriter(
new FileWriter(file));
bw.write(data, 0, data.length());
bw.close();
}
}

public boolean deleteFile(JStorageUser user, String filename)throws Exception
{
File file = new File(getRootFolder() + File.separator +
user.getName() + File.separator + filename);
return file.delete();
}


databaseUrl=jdbc:mysql:
//localhost/DATABASE?user=USER&password=PASS
databaseClass=com.mysql.jdbc.Driver

When using MySQL, the uppercase text in the databaseURL value must be changed to specify the correct database name, MySQL username, and password. Other drivers can be employed by changing databaseClass to the driver’s main class and using that database’s connection URL in databaseUrl.

The server’s getDatabaseConnection() method, which requires the import of the java.sql package, can be used with little or no modification in any Java class that makes use of JDBC to connect to a database:


public Connection getDatabaseConnection()
throws Exception {
Connection conn = null;
String data = getDatabaseUrl();
Class.forName(getDatabaseClass()).newInstance();
conn = DriverManager.getConnection(data, “”, “”);
return conn;
}

The JStorageHandler

Class

The JStorageHandler class (see Listing Two) contains the four methods of XML Storage System used to publish weblogs. The methods’ signatures are directly analogous to calls in the API specification:


// request and set up a new weblog
public Hashtable registerUser(
String email, String name, String password,
int clientPort, String userAgent, String serialNumber) { }

// learn about the server
public Hashtable getServerCapabilities(String email, String password) { }
// save one or more files
public Hashtable saveMultipleFiles(String email, String password,
Vector files, Vector fileContents) { }
// delete one or more files
public Hashtable deleteMultipleFiles(String email, String password,
Vector files) { }




Listing Two: The JStorageHandler class


package org.cadenhead.jstoragesystem;
import java.io.*;
import java.util.*;

public class JStorageHandler {
JStorageServer server;

public JStorageHandler(JStorageServer server) {
this.server = server;
}

public Hashtable registerUser(
String email, String name, String password,
int clientPort, String userAgent, String serialNumber) {

server.getLogger().debug(“Registration attempt from ” +
email + ” ” + name);
Hashtable results = new Hashtable();
if (!server.getAcceptingMembers()) {
results.put(“flError”, new Boolean(true));
results.put(“message”, “Not accepting new members.”);
server.getLogger().debug(“Told ” + email +
“new registrations are blocked.”);
return results;
}
JStorageUser user = new JStorageUser(server, email, name,
password, clientPort, userAgent, serialNumber,
0);
try {
user.create();
results.put(“flError”, new Boolean(false));
results.put(“message”, “Welcome ” + user.getName());
server.getLogger().debug(“Registered new member ” + email);
} catch (Exception exc) {
results.put(“flError”, new Boolean(true));
results.put(“message”, exc.getMessage());
server.getLogger().debug(“Could not register ” + email
+ “: ” + exc.getMessage());
}
return results;
}

public Hashtable getServerCapabilities(String email, String password) {
Hashtable results = new Hashtable();
JStorageUser user = new JStorageUser(server);
server.getLogger().debug(“Server capabilities request from ” +
email);
try {
user.load(email, password);
results.put(“legalFileExtensions”,
server.getLegalFileExtensions());
results.put(“maxFileSize”,
new Integer(server.getMaxFileSize()));
results.put(“maxBytesPerUser”,
new Integer(server.getMaxBytesPerUser()));
results.put(“ctBytesInUse”,
new Integer(user.getCountBytesInUse()));
results.put(“yourUpstreamFolderUrl”,
server.getRootFolder().getPath() + File.separator +
user.getName());
results.put(“flError”, new Boolean(false));
server.getLogger().debug(“Sent capabilities to ” +
email);
} catch (Exception exc) {
results.put(“flError”, new Boolean(true));
results.put(“message”, exc.getMessage());
server.getLogger().debug(“Error sending capabilities to “+ email + “: ” + exc.getMessage());
}

return results;
}

public Hashtable saveMultipleFiles(String email,
String password, Vector files, Vector fileContents) {

Hashtable results = new Hashtable();
JStorageUser user = new JStorageUser(server);
server.getLogger().debug(“Receiving files from ” + email);
Vector urlList = new Vector();
try {
if (files.size() < 1) {
throw new IOException(“No file to save.”);
}
user.load(email, password);
for (int i = 0; i < files.size(); i++) {
server.getLogger().debug(“File #” + i +
” from ” + email + “: ” + files.get(i). toString());
server.saveFile(user, (String)files.get(i),
fileContents.get(i));
urlList.add(server.getRootUrl() + user.getName() +
“/” + files.get(i).toString());
}
results.put(“flError”, new Boolean(false));
results.put(“urlList”, urlList);
server.getLogger().debug(“Saved files for ” + email);
} catch (Exception exc) {
results.put(“flError”, new Boolean(true));
results.put(“message”, exc.getMessage());
server.getLogger().debug(“Error saving files for ” +
email + “: ” + exc.getMessage());
}

return results;
}

public Hashtable deleteMultipleFiles(String email, String password, Vector files) {

Hashtable results = new Hashtable();
JStorageUser user = new JStorageUser(server);
server.getLogger().debug(“Deleting files for ” + email);
Vector errorList = new Vector();
try {
if (files.size() < 1) {
throw new IOException(“No file to delete.”);
}
user.load(email, password);
boolean allDeleted = true;
for (int i = 0; i < files.size(); i++) {
server.getLogger().debug(“Deleting file #” + i +
” from ” + email + “: ” + files.get(i).toString());
if (!server.deleteFile(user, (String)files.get(i))) {
errorList.add(“Could not delete file.”);
allDeleted = false;
} else {
errorList.add(“”);
}
}
results.put(“flError”, new Boolean(allDeleted));
if (!allDeleted) {
results.put(“errorList”, errorList);
}
server.getLogger().debug(“Deleted files for ” + email);
} catch (Exception exc) {
results.put(“flError”, new Boolean(true));
results.put(“message”, exc.getMessage());
server.getLogger().debug(“Error deleting files for ” +
email + “: ” + exc.getMessage());
}

return results;
}
}

Methods with the same names are in the XML Storage System specification with XML-RPC data types such as number and struct as arguments and return values. Apache XML-RPC converts between these XML-RPC types and Java types automatically when methods are called and values are returned according to these rules:

* byte arrays for base64 data

* Boolean objects for boolean values

* java.util.Date objects for dateTime.iso8601 values

* Double objects for double values

* Integer objects for number values

* String objects for string values

* Hashtable objects for struct values

* Vector objects for array data

The last argument of saveMultipleFiles() demonstrates how robust the Apache XML-RPC library is. The fileContents vector holds the contents of one or more files that should be saved to the JStorageSystem server. If a file contains binary data, such as a JPEG or an MP3 file, it’s submitted as base64 data by an XML-RPC client and stored as a byte array in the vector. If a file contains a web page or other text, it’s stored as a string in the vector.

The saveMultipleFiles() method retrieves each value from the files and fileContents vectors, calling the server’s saveFile() method to store them on the server. The saveFile() method uses the instanceof operator to determine whether the file contains binary data or text, then saves it accordingly. In the method, the contents variable holds an object containing the file:


if (contents instanceof byte[]) {
// save bytes with a file output stream
byte[] data = (byte[]) contents;
} else {
// save a string with a buffered writer
String data = contents.toString();
}

Each JStorageHandler method returns a java.util. Hashtable object to the XML-RPC client that called the method. When a method fails because of an exception, two things are sent back in the hash table: a Boolean object named flError that holds true, and a string that holds the exception’s error message. Here’s an example from a catch() block with an exception named exc and a hash table named results:


results.put(“flError”, new Boolean(true));
results.put(“message”, exc.getMessage());

All four methods return flError as Boolean(false) to signify success and Boolean(true) to signify failure.

Some methods return other information when they complete without error:

* registerUser() stores a greeting in a string named message.

* saveMultipleFiles() stores a string with the URL of each saved file in a vector named urlList.

* deleteMultipleFiles() stores a string in the errorList vector for each file. If the file was deleted successfully, an empty string is added to the vector. If the file wasn’t deleted for some reason (or does not exist), an error message is added instead.

Using JStorageUser

The JStorageUser class represents individual users of the server and the methods required to create them in a JDBC database and authenticate their accounts. This class has nine instance variables that are saved to a database during account creation. The bulk of this class contains boilerplate accessor methods required to set and get each of these variables — an approach that can be extended to prohibit unacceptable values if it becomes necessary. Portions of this class are shown in Listing Three .




Listing Three: Portions of the JStorageUser class


public void create() throws Exception {
File userFolder = new File(server.getRootFolder(),
getName());
Connection connection = server.getDatabaseConnection();
PreparedStatement pstmt = null; ResultSet rec = null;
pstmt = connection.prepareStatement(
“SELECT name,password FROM jstoragesystem WHERE (NAME =?)”);
pstmt.setString(1, getName());
rec = pstmt.executeQuery();
if ( (rec.next()) | (userFolder.exists()) ) {
throw new IOException(“Username taken.”);
}
if (!userFolder.mkdir()) {
throw new IOException(“Could not create folder.”);
}
Date now = new Date(new java.util.Date().getTime());
PreparedStatement pstmt2 = connection.prepareStatement(
“INSERT INTO jstoragesystem VALUES(?, ?, ?, ?, ?, ?, ?,
?, ?, ?, ?)”);
pstmt2.setString(1, “0″);
pstmt2.setString(2, getEmail());
pstmt2.setString(3, getName());
pstmt2.setString(4, getPassword());
pstmt2.setInt(5, getClientPort());
pstmt2.setString(6, getUserAgent());
pstmt2.setString(7, getSerialNumber());
pstmt2.setInt(8, getCountBytesInUse());
pstmt2.setDate(9, now); pstmt2.setDate(10, now);
pstmt2.setInt(11, JStorageUser.OK);
pstmt2.executeUpdate();
}

public void load(String email, String password) throws Exception {
Connection connection = server.getDatabaseConnection();
Date now = new Date(new java.util.Date().getTime());
PreparedStatement pstmt = null;
ResultSet rec = null;
pstmt = connection.prepareStatement( “SELECT
name,clientport,useragent,serialnumber,countbytesinuse,”
+ “status FROM jstoragesystem WHERE (EMAIL = ?) ”
+ “AND (PASSWORD = ?)”);
pstmt.setString(1, email);
pstmt.setString(2, password);
rec = pstmt.executeQuery();
if (!rec.next()) {
throw new IOException(“Bad e-mail or password.”);
}
setEmail(email);
setName(rec.getString(1)); setPassword(password);
setClientPort(rec.getInt(2));
setUserAgent(rec.getString(3));
setSerialNumber(rec.getString(4));
setCountBytesInUse(rec.getInt(5));
setStatus(rec.getInt(6));
}
}

Implementors of XML Storage System are following the lead of the Radio UserLand client in areas outside the specification. One example is how passwords are sent in an XML-RPC call. Radio UserLand takes a password from a user and stores it as an MD5 encrypted hash. The plain-text password is never used in calls to registerUser() and the other methods.

The following static method in JStorageUser creates a hash for a specified string by using the MessageDigest class, which can be found in the java.security package:


public static String hashPassword(String password) {
String hashword = null;
try {
MessageDigest md5 = MessageDigest.getInstance(“MD5″);
md5.update(password.getBytes());
BigInteger hash = new BigInteger(1, md5.digest());
hashword = hash.toString(16);
} catch (NoSuchAlgorithmException nsae){
// ignore
}
return hashword;
}

The create() and load() methods in the JStorageUser class use the java.sql package to create and execute prepared statements in SQL. Like any SQL programming in Java, development of this class required a little hair-pulling to deal with the particulars of how SQL was supported by the desired JDBC driver and database. The biggest obstacle involved the following statement, which queries a database to retrieve a user based on an email address and password:


pstmt = connection.prepareStatement(
“SELECT name,clientport,useragent, serialnumber,countbytesinuse,”
+ “status FROM jstoragesystem WHERE (EMAIL = ‘?’) ”
+ “AND (PASSWORD = ‘?’)”);

In a prepared statement, the question marks are filled in with values by calling the statement’s setX() methods, like so:


pstmt.setString(1, email);
pstmt.setString(2, password);

However, the preceding SELECT statement contains an error — single-quotation marks around each question mark. These work correctly when a MySQL database is edited interactively. However, they cause an ArrayIndexOutOfBounds exception in Java when the first setString() method is called. Removing all four single quote marks solved the problem.

The load() method of JStorageUser retrieves a user’s information from a database and stores it in instance variables of the object using accessor methods. When the incorrect email address or password is provided, load() signals this problem by creating and throwing a new IOException:


throw new IOException(“Bad e-mail or password.”);

The application uses exceptions in a consistent manner: methods of the server and user classes declare them with throws clauses, making it the responsibility of the handler class to catch exceptions and log them.

The only place this rule does not apply is in the server’s main() method, which deals with exceptions in case a logger can’t be created to display them.

Logging

Logging is implemented in the JStorageSystem project via org.apache.log4J, which makes it easy to support variable levels of logging with a small amount of coding. A Logger instance variable named logger is set up with an accessor method in the constructor of JStorageServer:


setLogger(Logger.getLogger(“org.cadenhead.jstoragesystem”));

Log4J supports five levels of logging that express increasing amounts of concern: DEBUG, INFO, WARN, ERROR, and FATAL.

The logger is configured in a properties file rather than code, making it possible to change the behavior of the logger without recompiling the project. The file’s name and location are stored in an instance variable named loggerProperties that’s defined in the server’s main properties file.

The properties file included with JStorageSystem sets up the application to log messages to a file named JStorageSystem.log in the /usr/local/web/cadenhead/jss directory:


log4j.logger.org.cadenhead.jstoragesystem=DEBUG, APPENDER
log4j.appender.APPENDER=org.apache.log4j.FileAppender
log4j.appender.APPENDER.layout=org.apache.log4j.SimpleLayout
log4j.appender.APPENDER.File=/usr/local/ web/cadenhead/jss/JStorageSystem.log

The class method PropertyConfigurator.configure() sets up a logger according to the contents of such a properties file:


PropertyConfigurator.configure (getLoggerProperties());

Once a logger’s been configured, it has debug(), info(), warn(), error(), and fatal() methods that can be called with a string argument to log a message with each of the respective priorities.

The JStorageHandler class makes heavy use of debug() to note calls to XML Storage System methods, as in this example:


server.getLogger().debug(

“Sent capabilities to ”
+ email);

If the logger’s property file was edited to use a lower level of concern than DEBUG, these messages would not show up in the log file.

Welcome to the Gift Economy

The JStorageSystem project — around 600 lines in its first release — accomplishes a lot on the backs of Apache XML-RPC, Log4J, and the Java 2 class libraries.

Additionally, there’s a database such as MySQL involved — another piece of robust, open source software released under generous terms.

Like most open source projects, the creation of JStorageSystem scratches an itch. A Linux server equipped with Java, the Apache Web server, and MySQL is ideally suited to XML-based weblog hosting, which makes it a robust platform for the Radio UserLand community, as well as other weblog publishers.

A more grandiose goal is to encourage the kind of development that’s taken place among Python coders since the release of the Python Community Server, an open source implementation of the XML Storage System by Phillip Pearson and Georg Bauer that can be downloaded from http://www.pycs.net.

That project has become a springboard for weblog-related development ever since its original release, inspiring the creation of two new weblog publishing clients with that language.



Rogers Cadenhead is a Web application developer and lead author of the upcoming Teach Yourself Java 2 in 21 Days, Fourth Edition from Sams Publishing. To contact Cadenhead, visit his weblog at http://www.cadenhead.org/workbench. You can download the code described in this article from http://www.linux-mag.com/downloads/2003-12/java.

Comments are closed.