MyThinkPond

On Java, Python, Groovy, Grails, Spring, Node.js, Linux, Arduino, ARM, Embedded Devices & Web

Archive for the ‘ZeroMQ’ Category

How to configure and install ZeroMQ (libsodium) on CentOS 6.7?

Posted by Venkatt Guhesan on September 6, 2015

zeromq_logoWhen getting started on ZeroMQ (version 4.2.0 or above) can be quite challenging especially with all the prerequisites. I’ve spent a good two days to get the process ironed out. So I’m sharing this so that others can avoid the same pitfalls and can have a good head-start with setting up their environment.

Pitfall #1: Develop for your platform. I’m accustomed to developing in Ubuntu 14.04.3 LTE but in this case my deployment environment happens to be CentOS 6.7 (minimal server). Because the dependencies such as GLIBC versions are different, it’s best to stick to a setup with the target platform in mind.

This exercise assumes that you’ve installed CentOS 6.7 (minimal server option).

Pitfall #2:

In your research you may have come across the following errors below. I’m including them in hopes that the search engine bots will bring you to this blog so that you can save some headaches.

checking whether the C compiler works… configure: error: in `/root/downloads/libzmq-master’:
configure: error: cannot run C compiled programs.
If you meant to cross compile, use `–host’.
libsodium is not installed
src/.libs/libzmq.so: undefined reference to `crypto_secretbox_open'
src/.libs/libzmq.so: undefined reference to `crypto_box_beforenm'
src/.libs/libzmq.so: undefined reference to `crypto_secretbox'
src/.libs/libzmq.so: undefined reference to `crypto_box'
src/.libs/libzmq.so: undefined reference to `crypto_box_keypair'
src/.libs/libzmq.so: undefined reference to `sodium_init'
src/.libs/libzmq.so: undefined reference to `crypto_box_open'
src/.libs/libzmq.so: undefined reference to `randombytes_close'
src/.libs/libzmq.so: undefined reference to `crypto_box_open_afternm'
src/.libs/libzmq.so: undefined reference to `randombytes'
src/.libs/libzmq.so: undefined reference to `crypto_box_afternm'
collect2: ld returned 1 exit status
make[1]: *** [tools/curve_keygen] Error 1
make[1]: Leaving directory `/root/downloads/libzmq-<wbr />master'
make: *** [all-recursive] Error 1
GLIB 2.14 not found

Solution:

The following steps provides you a step-by-step instruction to get you to a point where you can compile the standard ZeroMQ HelloWorld Server and HelloWorld Client.

# Steps to a working ZeroMQ 4+ code on CentOS67

# Login as root or make sure you have sudo access
# The following instructions assume you are logged in as "root"
# Assumes the following path /root/downloads
mkdir download
cd download

# ZeroMQ 4+ requires libsodium 
# You also need a C compiler

# Pre-requisites
yum update
# Gets your system upto date with the latest updates

yum install libtool gcc-c++ glib*
# This installs autoconf, automake, cloog-ppl, cpp, gcc, mpfr, ppl
yum groupinstall "development tools"



# Let us install and build libsodium
wget https://download.libsodium.org/libsodium/releases/libsodium-1.0.3.tar.gz
tar -xvf libsodium-1.0.3.tar.gz
cd libsodium-1.0.3
./configure
make clean
make
make install
# libsodium libraries are now installed under /usr/local/lib directory
# Next we need to tell libzmq and other development code where to find these libraries

# Add the following exports defining the following environmental libraries
# Edit the root users .bashrc file
vim /root/.bashrc
# Add the following
#-----
export sodium_CFLAGS="-I/usr/local/include"
export sodium_LIBS="-L/usr/local/lib"
export CPATH=/usr/local/include
export LIBRARY_PATH=/usr/local/lib
export LD_LIBRARY_PATH=/usr/local/lib
export LD_RUN_PATH=/usr/local/lib
export PKG_CONFIG_PATH=/usr/local/lib/pkgconfig
export CFLAGS=$(pkg-config --cflags libsodium)
export LDFLAGS=$(pkg-config --libs libsodium)
#-----

# Reinitialize settings under .bashrc
source ~/.bashrc

# Add libsodium to ldconfig 
echo '/usr/local/lib' > tee -a /etc/ld.so.conf.d/libsodium.conf

# Download the latest lizmq from the GIT repository
cd /root/downloads
wget https://github.com/zeromq/libzmq/archive/master.zip
unzip master.zip
cd libzmq-master

# Lets begin building it
# Generate the configure script from template
./autogen.sh
./configure
make clean
make
make install

# ZeroMQ libraries are installed in /usr/local/lib path
# Need to add the libraries to ldconfig so that they can be
# statically linked in C code
# Since ZMQ is installed in the same path as libsodium,
# we do not need to add another path into /etc/ld.so.conf.d/*.conf
# we just need to reload the "ldconfig"
ldconfig

# Let try compiling and testing a sample code
mkdir /root/downloads/zmqtest
cd /root/downloads/zmqtest

# vim helloserver.c
# Copy hwserver code from the following url:
wget https://github.com/imatix/zguide/raw/master/examples/C/hwserver.c

# Copy hwclient code from the following url:
wget https://github.com/imatix/zguide/raw/master/examples/C/hwclient.c

# Compile hwserver 
gcc hwserver.c -o hwserver -lzmq
gcc hwclient.c -o hwclient -lzmq

# Open two SSH terminals to /root/downloads/zmqtest
# Run hello-world server in terminal #1
# Run hello-world client in terminal #2

# If it runs, you're all set to go
# Please link me from your blog/social places so that others can easily find this article
# Also provide me with feedback so that I can ammend this guide for others

# Thank you & cheers!

 

 

Posted in C - C++ - gcc - cpp, Kernel, Linux, ZeroMQ | Tagged: , , , | 1 Comment »

Working with zeromq (0mq), Java, JZMQ on a CentOS platform

Posted by Venkatt Guhesan on June 24, 2013

Recently I decided to port some of my development using ZeroMQ onto my CentOS development machine and I ran into some challenges. I’m documenting those challenges so that if someone else runs into the same pitfalls I did, they can avoid it.

In this example today, we will work with the first “HelloWorld” examples in the ZeroMQ guide found here. I added a few modifications to the sample such as a package name and a try-catch around the Thread and an exception.tostring() to display any stack-trace.

Source code for src/zmq/hwserver.java


package zmq;

import java.io.PrintWriter;
import java.io.StringWriter;

import org.zeromq.ZMQ;

//
// Hello World server in Java
// Binds REP socket to tcp://*:5555
// Expects "Hello" from client, replies with "World"
//

public class hwserver {

	/**
	 * @param args
	 */
	public static void main(String[] args) {
		ZMQ.Context context = ZMQ.context(1);
		// Socket to talk to clients
		ZMQ.Socket socket = context.socket(ZMQ.REP);
		socket.bind ("tcp://*:5555");
		try {
			while (!Thread.currentThread ().isInterrupted ()) {
				byte[] reply = socket.recv(0);
				System.out.println("Received Hello");
				String request = "World" ;
				socket.send(request.getBytes (), 0);
				Thread.sleep(1000); // Do some 'work'
			}
		} catch(Exception e) {
			StringWriter sw = new StringWriter();
			PrintWriter pw = new PrintWriter(sw);
			e.printStackTrace(pw);
			System.out.println(sw.toString());
		}
		socket.close();
		context.term();

	}

}

Similarly, source code for the client, src/zmq/hwclient.java

package zmq;

import org.zeromq.ZMQ;

public class hwclient {

	/**
	 * @param args
	 */
	public static void main(String[] args) {
		ZMQ.Context context = ZMQ.context(1);

		// Socket to talk to server
		System.out.println("Connecting to hello world server");

		ZMQ.Socket socket = context.socket(ZMQ.REQ);
		socket.connect ("tcp://localhost:5555");

		for(int requestNbr = 0; requestNbr != 10; requestNbr++) {
			String request = "Hello" ;
			System.out.println("Sending Hello " + requestNbr );
			socket.send(request.getBytes (), 0);

			byte[] reply = socket.recv(0);
			System.out.println("Received " + new String (reply) + " " + requestNbr);
		}

		socket.close();
		context.term();

	}

}

Now that you have the sample code, how do you compile using the ZeroMQ?

Assumption: You have installed Java (1.7 or above)

Step-1: Installing ZeroMQ onto CentOS [Following steps are performed under root account]

  1. Install “Development Tools” if it’s not already installed on your CentOS as root:        yum groupinstall “Development Tools”
  2. Download the “POSIX tarball”  ZeroMQ source code onto your CentOS development machine from here. At the time of writing this article, ZeroMQ version 3.2.3 was the stable release. You might want to download the latest stable release.
  3. Unpack the .tar.gz source archive.
  4. Run ./configure, followed by “make” then “make install“.
  5. Run ldconfig after installation.

Step-2: Installing a Language Binding for Java. In this case, we will use JZMQ from https://github.com/zeromq/jzmq

  1. Download the latest stable release from GITHub link above. (git clone git://github.com/zeromq/jzmq.git)
  2. Change directory, cd jzmq
  3. Compile and Install:
    ./autogen.sh
    ./configure
    make
    make install
    
  4. Where did it install?
    # JAR is located here: /usr/local/share/java/zmq.jar
    # .so link files are located here: /usr/local/lib
    
  5. Important Step: Add /usr/local/lib to a line in /etc/ld.so.conf (here is my copy after editing)
    include ld.so.conf.d/*.conf
    /usr/local/lib
    
  6. Reload “ldconfig“. This clears the cache.

 

Step-3: Compile and run the Java examples above.

cd ~/dev/zeromq/example/
# Compile hwserver.java
javac -classpath  /usr/local/share/java/zmq.jar ./zmq/hwserver.java
# Compile hwclient.java
javac -classpath  /usr/local/share/java/zmq.jar ./zmq/hwclient.java
# Run hwserver in a separate prompt
java -classpath .: /usr/local/share/java/zmq.jar -Djava.library.path=/usr/local/lib zmq.hwserver
# Run hwclient in a seperate prompt
java -classpath .:/usr/local/share/java/zmq.jar -Djava.library.path=/usr/local/lib zmq.hwclient

Output on the hwserver console:

Received Hello
Received Hello
Received Hello
Received Hello
Received Hello
Received Hello
Received Hello
Received Hello
Received Hello
Received Hello

output on the hwclient console:

Connecting to hello world server
Sending Hello 0
Received World 0
Sending Hello 1
Received World 1
Sending Hello 2
Received World 2
Sending Hello 3
Received World 3
Sending Hello 4
Received World 4
Sending Hello 5
Received World 5
Sending Hello 6
Received World 6
Sending Hello 7
Received World 7
Sending Hello 8
Received World 8
Sending Hello 9
Received World 9

Few interesting points to note are as follows:

  • What happens if you started the client first and then the server? Well, the client waits until the server becomes available (or in other words, until some process connects to socket port 5555) and then sends the message. When you say socket.send(…), ZeroMQ actually enqueues a message to be sent later by a dedicated communication thread and this thread waits until a bind on port 5555 happens by “server”.
  • Also observe that the “server” is doing the connecting, and the “client” is doing the binding.

What is ZeroMQ (ØMQ)?

(Excerpt from the ZeroMQ website!)

ØMQ (also seen as ZeroMQ, 0MQ, zmq) looks like an embeddable networking library but acts like a concurrency framework. It gives you sockets that carry atomic messages across various transports like in-process, inter-process, TCP, and multicast. You can connect sockets N-to-N with patterns like fanout, pub-sub, task distribution, and request-reply. It’s fast enough to be the fabric for clustered products. Its asynchronous I/O model gives you scalable multicore applications, built as asynchronous message-processing tasks. It has a score of language APIs and runs on most operating systems. ØMQ is from iMatix and is LGPLv3 open source.

If you find this article useful, please subscribe to my blog and/or share my link with others.

Posted in Java, ZeroMQ | Tagged: , , , , , | 1 Comment »