A Database Tool for Client-Server Systems

In my previous article, Communicating with Remote Processes without DCOM or CORBA, I introduced a new inter process communication tool, XYMessenger. Basically, XYMessenger is a binary component used to send and receive messages between processes. The Win32 version is an ActiveX control that can be used in any VB or VC++ (MFC) program. For developers working on platforms other than Win32, there is a java version which is compatible with the Win32 version. The best feature of XYMessenger is that you can easily connect a lot of processes together. Any connected process can send/receive data to/from any other connected process.

In this article, I will introduce XYDataManager, a program that uses XYMessenger to simplify database programming on the server side. This program and sample source code are included in the same download package as XYMessenger. XYDataManager runs on the Win32 platform, but its clients can be running on other platforms using the Java version of XYMessenger. I am current considering porting it to Java using JDBC.

Suppose we have multiple server programs in a client-server system. Each server provides one or more services and these services may retrieve or update data in a central database. Let's say our system is built for a bank and the central database contains all customer account information. Suppose there is a service named deductFrom provided by one server program and there is another service named depositTo provided by another server program. The deductFrom service will deduct money from a given account, the depositTo service will deposit money into a given account. How do we move money from one account to another? The simplest way would be making calls to the existing services deductFrom and depositTo. The problem is, we must make sure that both of these services succeed or both of them fail. Since these two services are provided by different server processes, we need to implement transactions across process boundaries.

This is one of the problems XYDataManager can help to solve. XYDataManager manages all database connections and transactions for you. XYDataManager has an embedded XYMessenger object. Other processes can perform database operations by sending requests to XYDataManager using XYMessenger. XYDataManager will execute these requests on the database and will return data or error messages back.

Wait a minute! Does this mean all database operations have to go through a single process, namely XYDataManager? Will XYDataManager become a bottleneck in the whole system and drag down the performance? The answer is no. First of all, XYDataManager is a multi-threaded program that manages multiple database connections (each connection in a separate thread). The number of threads dynamically increases or decreases with the number of simultaneous requests from client processes. The user can configure the number of initial threads to optimize performance. Secondly, there can be multiple instances of XYDataManager running in a system, although it is usually not necessary to run more than one instance of XYDataManager on the same machine.

Here is how XYDataManager works. Each client process of XYDataManager has an XYMessenger object embedded and is connected to the same Communication Tree as XYDataManager. Please see my previous article for the concept of Communication Tree. A client performs a database operation by sending a synchronous message to XYDataManager, which can be done by simply calling a method of XYMessenger. The message contains either a SQL statement or a call to a predefined query/procedure on the database. After executing the request, XYDataManager will return output data or an error string to the client in a reply message.

To handle database transactions, XYDataManager recognizes the following three special database requests from its clients: "begin transaction", "commit transaction", and "rollback transaction". A client starts a transaction by sending a "begin transaction" message. The reply message from XYDataManager contains a transaction ID string. This transaction ID string will be used in all other database requests that belong to the same transaction. Finally, the client can send either a "commit transaction" or a "rollback transaction" message to XYDataManager to end the current transaction.

There are two important features in XYDataManager.

The first feature is that a transaction can be started by one client and committed or rolled back by a different client. For example, client A sends a message to XYDataManager to start a transaction. Then it uses the returned transaction ID string to perform some database operations (select, insert, update, or delete). Finally, it passes the transaction ID string to client B without committing the transaction. After receiving the transaction ID string from client A, client B can use it to perform other database operations, then commits the transaction or passes the transaction ID string to a third client of XYDataManager, and so on.

The second feature is that a client of XYDataManager can manage more than one transaction at the same time. For example, process A, a client of XYDataManager, can be a server process in a client-server system. When a user (client of process A) calls a service provided by process A that requires a database transaction, process A will start a transaction by sending a "begin transaction" message to XYDataManager. While the above transaction is in progress, another user may call a service provided by process A. Process A can ask XYDataManager to start a new transaction for the new service call. This is possible because XYDataManager is multi-threaded.

Detailed documentation and sample code for XYDataManager can be found in the download package and also from my web site (which is updated frequently).

I need to point out an important advantage of using XYDataManager. It requires no special support from the database itself. The only requirements on the database are: it has to support transactions and an ODBC driver is available. The documentation is included in the download package.

XYDataManager does have its limitations, however. It cannot handle nested transactions. It cannot handle certain binary data types. Each instance of XYDataManager can only access one database. But you can use multiple instances of XYDataManager. For example, use one instance of XYDataManager to access an Oracle database and another instance to access an MS Access database.

For the latest updates of XYMessenger, XYDataManager, other related tools and documentation, please visit my web site. Thank you for reading this article.

--- Xiangyang Liu