MyThinkPond

On Java, Python, Groovy, Grails, Spring, Node.js, Linux, Arduino, ARM, Embedded Devices & Web

Archive for the ‘Bash’ Category

Set Up a TFTP Server on Linux | Linux.com

Posted by Venkatt Guhesan on March 13, 2017

Most users are familiar with FTP, but if you want to kickstart Red Hat installs, PXE boot systems, auto-provision VoIP phones or unbrick a Linux-based router, you want a Trivial File Transfer Protocol (TFTP) server. Setting one up on Linux is easy, and a perfect project to take on over the weekend.

Source: Weekend Project: Set Up a TFTP Server on Linux | Linux.com | The source for Linux information

Posted in Bash, CentOS 6.X, CentOS7, Hypervisior Images, Linux, Scripting | Tagged: , , | Leave a Comment »

How To Record Everything You Do In Terminal – OSTechNix

Posted by Venkatt Guhesan on March 13, 2017

Synopsis:


script -a my_history

mkdir test

cd test

touch hello_world.txt

echo 'Hello World' > ./hello_world.txt

# This closes the "script -a my_history" file
exit

Source: How To Record Everything You Do In Terminal – OSTechNix

Posted in Bash, CentOS 6.X, CentOS7, Linux, Scripting | Tagged: , , , , , | Leave a Comment »

Developing email applications on Windows, targeted for Linux environment – email, mailx, sendmail

Posted by Venkatt Guhesan on December 11, 2016

linux_centos_logoIf you’re developing on a Windows platform for an application targeted for Linux or Unix that deals with email, then this article will be useful.

Let us begin by  understand the problem.

Problem

If you are a Java/Spring developer, (developing in Java is platform independent – runs on any platform where a JVM is available) then you have two options in front of you for sending emails from a Java application:

Option #1: JavaMail API

Option #2: Linux sendmail or mailx invoked via Runtime.getRuntime().exec(invoke – linux – mail – command) [example here]

If you are a PHP or Ruby or Bash or Python developer, you can do the same using a native library available within the language or invoke a native Linux execute-command to invoke either sendmail or mailx.

What happens if you are developing a hybrid library that needs to send email both from Java and Python. Then most likely you want to leverage a common library accessible to all the various libraries. In this case, you’re going to depend on either sendmail or mailx.   Now let us throw in a additional problem into this mix. Suppose you are developing on a Windows platform but the end application is targeted to run on Linux. This now creates a new additional problem. Sendmail or Mailx do not exist for Windows. Well, this article provides a way to develop on Windows by invoking execute mailx calls from your native code to the underlying emulation of Linux on Windows.

Now that we have established the problem, let us walk through a solution.

Please note that this is not the only solution but a potential solution. If you know of other mechanisms, feel free to send me a link and I’ll add them in.

Solution

Step-1: Make sure that you have Cygwin installed (with email tool selected additionally) on your Windows computer.

# Download  setup-x86_64.exe from https://cygwin.com/setup-x86_64.exe
# and run it.

# When you run the setup, please additionally select "email" tool additionally. It is not added by default.

Once Cygwin is installed, most people traditionally use all the Cygwin tools from within a Cygwin Command-Prompt. The executables are located under “\bin\**” directory. Which means that if you run a “ls” in a DOS command-prompt, it will error saying “command not found”. There is a undocumented\not so known secret to emulating Linux commands natively in Windows. This is where your true Linux power comes in under native Windows.

Step-2: Add Cygwin\bin path natively to Windows PATH.

# In my computer, I have Cygwin installed at the following path: c:\cygwin64\

To add "c:\cygwin64\bin" the system path, perform these steps:

1. Start the System Control Panel applet (Start - Settings - Control Panel - System).
2. Select the Advanced tab.
3. Click the Environment Variables button.
4. Under System Variables, select Path, then click Edit.
5. Add c:\cygwin64\bin to the path.

Once Step-2 is completed. If you start a new DOS command-prompt and run a “ls” command, you should see the directory listing (same as a ‘dir’ command under Windows).

Step-3: The send email application uses “c:\cygwin64\etc\email\email.conf” for SMTP and other email properties.

# Edit c:\cygwin64\etc\email\email.conf and add your SMTP details (domain/server name, user-id, password)
# For this example, I have setup a *special* GMail Email account specially to test [Don't use your private GMail] so I will use the user-id and password for that Gmail account.

SMTP_SERVER = 'smtp.gmail.com'
SMTP_PORT = '587'
USE_TLS = 'true'
SMTP_AUTH = 'LOGIN'
SMTP_AUTH_USER = 'YOUR_TEST_GMAIL_ADDRESS@gmail.com'
SMTP_AUTH_PASS = 'YOUR_GMAIL_PASSWORD_GOES_HERE'
MY_NAME  = 'Venkatt Guhesan'
MY_EMAIL = 'foobar7634@gmail.com'

Step-4: Let’s test this under a DOS command-prompt.

# Open a DOS Command-Prompt and run the following:

echo "Test from Cygwin in Windows under DOS command-prompt" | email -s "Testing Email from Windows DOS" recipient_email_address@gmail.com 

# Within minutes, you should see an email in the Gmail inbox as expected.

# If you are not seeing it, then you can use the example below where you can pass in the SMTP arguments for validation. Maybe you typed in the password for the Gmail account incorrectly or you entered one of the parameters incorrectly in the c:\cygwin64\etc\email\email.conf file.

############### HERE IS AN EXAMPLE WITH SMTP PARAMETERS PASSED IN AS ARGUMENTS #######################
echo "Test from Cygwin in Windows under DOS command-prompt" | email -s "Tetsing Email" recipient_email_address@gmail.com -r smtp.gmail.com -p 587 -m login -u YOUR_TEST_GMAIL_ADDRESS@gmail.com -i YOUR_GMAIL_PASSWORD_GOES_HERE -tls
######################################################################################################

ss1

Once you have confirmed a successful email then the next step is to create a mailx application.

Step-5: Create a mailx.exe (mailx) within Cygwin’s bin directory.

For emulating mailx from Cygwin’s email.exe, you’re in luck. The format for mailx is identical to the email.exe’s arguments. All you are missing is mailx.exe. So simply copy email.exe under Cygwin’s bin directory to mailx.exe under the same Cygwin’s bin directory. And you’re all set!

# simply copy email.exe under Cygwin's bin directory to mailx.exe under the same Cygwin's bin directory

cd c:\cygwin64\bin
cp email.exe mailx.exe

# Now just like you running email (without the .exe suffix) you can run mailx (without the .exe suffix)

Step-6: Test sending an email using mailx


echo "Test from Cygwin-mailx in Windows under DOS command-prompt" | mailx -s "Testing Email from Windows DOS" recipient_email_address@gmail.com 

ss2

As a last step, now within your Java application, test Runtime.getRuntime().exec(“mailx command”);

Runtime.getRuntime().exec("echo 'Test from Cygwin-mailx in Windows under DOS command-prompt' | mailx -s 'Testing Email from Windows DOS' recipient_email_address@gmail.com");

Now your Java application as well as any Python, Bash, Linux shell applications and/or scripts are now leveraging the common “mailx” and your development is now consistently the same using one common mailx library.

Cheers!

Today’s inspirational quote:
theodore-roosevelt
  • We must all either wear out or rust out, every one of us. My choice is to wear out.
– Theodore Roosevelt, an American statesman, author, explorer, soldier, naturalist, and reformer who served as the 26th President of the United States from 1901 to 1909.

Posted in Bash, CentOS 6.X, CentOS7, Grails, Groovy, Java, Linux, Spring, Windows | Tagged: , , , , , , | Leave a Comment »

WaitUntilPortOpens() – How to wait until process A runs and binds to a port before spawning process B?

Posted by Venkatt Guhesan on July 8, 2016

Sometimes we have situations where you need to wait until one application is loaded that may bind to a certain port before kicking off a second application that may depend on that port. This process is typically described as “Wait-Until-Port-Opens” (or it could be the reverse – where you want to wait until a port closes).

Here are some use-cases for this method or script:

  • You have a Java web application (Jetty, Tomcat, WildFly, etc) that listens on port 8080 and you want your Nginx or Apache HTTPd server to start as a proxy-server once that back-end web server is bound to port 8080.
  • You want to spin off your Kong (microserver management) after, your back-end REST server is up and listening on port 123.
  • Or the reverse, where you want to kick off an email alert-notification, if you are unable to bind to a certain port where your web server runs and it has now crashed.

On a Linux system, this can be accomplished in many ways. Tools like netcat (nc), netstat, telnet makes this very simple. You can write a bash script that accomplishes this easy. But not all footprints may have those libraries installed. For example, if you’re switching from CentOS 6.8 to CentOS 7. A lot of the network tools are no longer available or their binaries are not yet ready for prime-time on the new SystemD footprint in CentOS 7. Netcat is one of those tools not yet ready in the CentOS 7. So you will need a common independent way to scan for the port.

First, let’s look at how this can be done using the above tools:

# Depends on netcat (nc)
# Run netcat and try binding to port 8080 on localhost. 
# If not sleep for five-seconds and repeat until it's available
while ! echo exit | nc localhost 8080; do sleep 5; done

# Variation of the above that depends on netstat utility
while netstat -lnt | awk '$4 ~ /:8080$/ {exit 1}'; do sleep 5; done

# If you know the process-id of your service, then you
PID=1234
while ! lsof -n -Fn -p $PID | grep -q '^n.*:8080$'; do sleep 5; done

# Variation that depends on netchat (nc)
# If the port is open, then do nothing but if the port goes down then react
while ! nc -q 1 localhost 8080 </dev/null; do sleep 5; done

As you can see all the above examples depend on other network tools to be installed on the Linux footprint. This design is good, if you have complete control of the OS and dependent RPMs and libraries that are installed.

If on the other case, if you do not have any of the above libraries such as netcat (nc), netstat, lsof then here is a Linux method to doing the same:

# Most (99.999%) Linux systems have "timeout" and "sleep" commands available so the below 
while ! timeout 1 bash -c "echo > /dev/tcp/localhost/8080"; do sleep 5; done;
# Do something here such as spawning a new service

So how does the above code work? Through the use of “Bash TCP Sockets”. You can learn more about then in the references below:

Tech Tip: TCP/IP Access Using bash

More on Using Bash’s Built-in /dev/tcp File (TCP/IP)

Cheers!

Today’s Inspirational Quote:

Doing what you love is the cornerstone of having abundance in your life.
Wayne Dyer (American philosopher, self-help author, and a motivational speaker)

-AUM

Posted in Bash, CentOS7, Linux | Tagged: , , | Leave a Comment »

bash script to generate config or property file from a template file containing variables

Posted by Venkatt Guhesan on June 23, 2016

Sometimes we have configuration or properties file (as templates) such as httpd.conf or server.conf where we want to dynamically replace $variables with values before writing the output to a new file.

Example:

# httpd.conf.tmpl
<Location $STATUS_URI>
    SetHandler server-status
    Order deny,allow
    Deny from all
    Allow from $MONITOR_IP
</Location>

We want to develop an install script in Bash such that it reads the httpd.conf.tmpl and replaces the $STATUS_URI and $MONITOR_IP with appropriate values (either passed in as script arguments or coded in the bash script) and then write out the resulting output to a new file such as /tmp/httpd.conf

Here is such as script:

#!/usr/bin/env bash

# Define the variables with values you want replaced
STATUS_URI="foobar"
MONITOR_IP="192.168.1.1"
# This could also be read in via bash arguments. 
# Google "bash getopts" for more information

# render a template configuration file
# expand variables + preserve formatting
# user="Venkatt"
# referenced inside the template.txt as $user
# render_template /path/to/template.txt > path/to/configuration_file
function render_template() {
  eval "echo \"$(cat $1)\""
}

function generate_httpd_conf {
  echo "#### Creating /tmp/httpd.conf from template ./httpd.conf.tmpl"
  render_template httpd.conf.tmpl > /tmp/httpd.conf
}

Generated output:

<Location foobar>
    SetHandler server-status
    Order deny,allow
    Deny from all
    Allow from 192.168.1.1
</Location>

This can be a useful method to generate .conf .prop (configuration or properties) files.

Cheers.

And now for today’s inspirational quote:

Let us sacrifice our today so that our children can have a better tomorrow.
Abdul Kalam
– 11th President of India from 2002 to 2007
– Chief ‘Rocket’ Scientist born from a poor family with humble beginnings.
– Lived as a “Man of simplicity”

Posted in Bash, Linux | Tagged: , , , , , | Leave a Comment »