Clean up cached memory in Linux


When we use top or free -m -h command on terminal, we may have noticed the used memory.
When Linux first read or write data, it is stored on unused areas of memory to accelerate the data access for future use. It is called caching. If this data is read again later, it can be quickly read from this cache in memory. It greatly enhance system performance. When application needs more memory, a chunk of memory is cleared from cache and provided to the application.

“The Linux disk cache is very unobtrusive. It uses spare memory to greatly increase disk access speeds, and without taking any memory away from applications. A fully used store of ram on Linux is efficient hardware use, not a warning sign.”

 
We can check current memory status by using :

watch -n 1 free -m

This command will display memory status, updating every seconds.
Memory watch on linux

watch -n 1 cat /proc/meminfo

This command will provide detail memory status and usage.

Clearing cached memory :

sudo sysctl -w vm.drop_caches=3

This command will clear all cached memory space. And will not improve nor affect system performance.

Spring part 1: Getting Started with Gradle

Table of Contents

  1. Overview
  2. Project Setup
    • Gradle Config : build.gradle
    • Configuration : web.xml
    • Context.xml
    • JSP
    • Controller


1. Overview
Spring is prominent framework developed by Rod Johnson in 2003. It is modular, light weight, open source framework and inversion of control container for the Java platform. Spring is based on POJO model, loose coupling and is non-invasive.


2. Project Setup
We’ll start by creating simple spring web application with gradle (project automation tool built upon concepts of Apache Ant and Apache Maven and is Groovy based DSL) in Eclipse IDE.

File > New > Dynamic Web Project

Provide a suitable project name and proceed, check generate web.xml and finish.


2.a. Gradle Config : build.gradle
Gradle is project automation tool built upon concepts of Apache Ant and Apache Maven and is Groovy based DSL(Domain Specific Language) instead of the more traditional XML form of declaring the project configuration. Unlike Maven, Gradle uses a directed acyclic graph (“DAG”) to determine the order in which tasks can be run.

apply plugin : 'eclipse'
apply plugin : 'java'
apply plugin : 'war'</span>

ext {
springVersion = '4.0.3.RELEASE'
logbackClassickVersion= '1.1.1'
}

repositories {
mavenLocal()
mavenCentral()
}

dependencies {
compile ("commons-codec:commons-codec:1.9")
compile ("commons-logging:commons-logging:1.1.1")
// Spring framework jars
compile ("org.springframework:spring-web:${springVersion}")
compile ("org.springframework:spring-webmvc:${springVersion}")
compile ("org.springframework:spring-context:${springVersion}")
compile ("org.springframework:spring-expression:${springVersion}")
compile ("org.springframework:spring-beans:${springVersion}")
compile ("org.springframework:spring-core:${springVersion}")
compile ("org.springframework:spring-aop:${springVersion}")
compile ("jstl:jstl:1.2")
}

task wrapper(type: Wrapper){
gradleVersion = '1.11'
}

The ‘plugins’ provide focused functionality to the Gradle.
Java Plugin – compiling, testing and publishing Java based software
Eclipse Plugin – integrating with the Eclipse IDE for development
War Plugin – project war generation

2.b. Configuration : web.xml
A deployment descriptor describes how a component, module or application (such as a web application or enterprise application) should be deployed. It directs a deployment tool to deploy a module or application with specific container options, security settings and describes specific configuration requirements. source

&lt;servlet&gt;
&lt;servlet-name&gt;appServlet&lt;/servlet-name&gt;
&lt;servlet-class&gt;org.springframework.web.servlet.DispatcherServlet&lt;/servlet-class&gt;
&lt;init-param&gt;
&lt;param-name&gt;contextConfigLocation&lt;/param-name&gt;
&lt;param-value&gt;/WEB-INF/servlet-context.xml&lt;/param-value&gt;
&lt;/init-param&gt;
&lt;load-on-startup&gt;1&lt;/load-on-startup&gt;
&lt;/servlet&gt;
&lt;context-param&gt;
&lt;param-name&gt;contextConfigLocation&lt;/param-name&gt;
&lt;param-value&gt;/WEB-INF/application-context.xml&lt;/param-value&gt;
&lt;/context-param&gt;</span>

&lt;servlet-mapping&gt;
&lt;servlet-name&gt;appServlet&lt;/servlet-name&gt;
&lt;url-pattern&gt;/&lt;/url-pattern&gt;
&lt;/servlet-mapping&gt;
&lt;listener&gt;
&lt;listener-class&gt;org.springframework.web.context.ContextLoaderListener&lt;/listener-class&gt;
&lt;/listener&gt;

The DispatcherServlet creates its own WebApplicationContext and is a central servlet that manages handlers/controllers/view-resolvers and offers functionality facilitating the development of web applications.
Basically, ContextLoaderListener creates a root web-application-context for the web-application and puts it in the ServletContext. It can be used to load and unload the spring-managed beans irrespective of what technology is being used in the controller layer.
source

2.c. Configuration: context.xml
application-context.xml

&lt;beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:tx="http://www.springframework.org/schema/tx"
xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.0.xsd http://www.springframework.org/schema/tx http://www.springframework.org/schema/tx/spring-tx.xsd"&gt;
&lt;import resource="controllers.xml" /&gt;
&lt;/beans&gt;

servlet-context.xml

&lt;beans:beans xmlns="http://www.springframework.org/schema/mvc"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:beans="http://www.springframework.org/schema/beans"
xsi:schemaLocation="
http://www.springframework.org/schema/mvc http://www.springframework.org/schema/mvc/spring-mvc-3.0.xsd
http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.0.xsd"&gt;
&lt;beans:bean
class="org.springframework.web.servlet.view.InternalResourceViewResolver"&gt;
&lt;beans:property name="prefix" value="/WEB-INF/jsp/" /&gt;
&lt;beans:property name="suffix" value=".jsp" /&gt;
&lt;/beans:bean&gt;
&lt;/beans:beans&gt;

“InternalResourceViewResolver” resolves views selected for rendering by @Controllers to .jsp resources in the /WEB-INF/jsp directory.


2.d. Pages
Create all jsp file under WEB-INF/jsp directory.
index.jsp

&lt;body&gt;
&lt;h3&gt;Howdy&lt;/h3&gt;
&lt;a href='&lt;c:out value="/nextpage"/&gt;'&gt;Next Page&lt;/a&gt;
&lt;/body&gt;

nextpage.jsp

&lt;body&gt;
&lt;h3&gt;Welcome to Spring Framework.&lt;/h3&gt;
&lt;/body&gt;


2.e. Controllers.xml
A context file for defining application controllers.

&lt;beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:mvc="http://www.springframework.org/schema/mvc"
xmlns:context="http://www.springframework.org/schema/context" xmlns:tx="http://www.springframework.org/schema/tx"
xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.0.xsd
http://www.springframework.org/schema/mvc http://www.springframework.org/schema/mvc/spring-mvc-3.0.xsd
http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context-3.0.xsd
http://www.springframework.org/schema/tx http://www.springframework.org/schema/tx/spring-tx.xsd"&gt;
&lt;context:component-scan base-package="com.pj.ci.server" /&gt;
&lt;mvc:annotation-driven /&gt;
&lt;/beans&gt;

“component-scan” scan across the package to find and register the beans in application context.
“annotation-driven” declares support for spring MVC features like @RequestMapping, @Controller as well as declrative validation with @Valid and message body conversion with @RequestBody & @ResponseBody
MainController class

@Controller
public class MainController {</span>

@RequestMapping(value = "/", method = RequestMethod.GET)
public String getMainPage(ModelMap modelMap) {
return "index";
}

@RequestMapping(value = "/nextpage", method = RequestMethod.GET)
public String getNextPage(ModelMap modelMap) {
return "nextpage";
}
}

This controller redirects request for ‘/’ and ‘/nextpage’ to ‘index’ & ‘nextpage’ view mapped in WEB-INF/jsp directory. The application default page is ‘index.jsp’

Project compile & running
In the terminal or console, browse the project directory and execute below commands. To use ‘gradle’ command in console we’ll need to set GRADLE_HOME in system.

gradle eclipse

This command will locallize the classpath of project for Eclipse IDE.

gradle build

This command will compile and generate necessary artifacts for project.
* “gradle wrapper” command can also be executed to generate wrapper file for the project. For this, we’ll need to add ‘task wrapper’ in build.gradle. With wrapper file, we’ll be able to use “gradlew eclipse” or “gradlew build”.

Reference :
1. Spring docs : 4.0.3 Release
2. Gradle user guide

Sending email using velocity template


Email (electronic mail) is an effective medium to reach out to masses. In application, a need may arise to use email as mean of communicating with customers. A notification mail has to be send when someone post comment or transaction has been successfully completed etc.


A simple email service can be prepared within application with few steps. All we need is, mail sender service and a template.


Dependencies
commons-codec-1.6.jar
commons-collections-3.2.1.jar
commons-lang-2.5.jar
mail-1.4.4.jar
spring-context-support-3.2.2.RELEASE.jar
spring-core-3.2.3.RELEASE.jar
velocity-1.7.jar


Complete Code:

final String fromEmail = "from@gmail.com";
final String toEmail = "to@gmail.com";

	private JavaMailSender javaMailSender;

	public String templateSetup(Map<String, Object> properties, String templateName){
		final VelocityEngine velocityEngine = new VelocityEngine();
		velocityEngine.setProperty(RuntimeConstants.RESOURCE_LOADER, "classpath");
		velocityEngine.setProperty("classpath.resource.loader.class", ClasspathResourceLoader.class.getName());
		velocityEngine.init();

		VelocityContext context = new VelocityContext();
		context.put("date", new Date());
		final Template template = velocityEngine.getTemplate(templateName);
		StringWriter writer = new StringWriter();
		template.merge(context, writer);

		return VelocityEngineUtils.mergeTemplateIntoString(velocityEngine, template.getName(), "UTF-8", properties);
	}

	public JavaMailSender serverSetup(String password){
		JavaMailSenderImpl impl = new JavaMailSenderImpl();
		impl.setHost("smtp.gmail.com");
		impl.setPort(587);
		impl.setUsername("from@gmail.com");
		impl.setPassword(password);

		Properties mailProp = new Properties();
		mailProp.setProperty("mail.smtp.auth", "true");
		mailProp.setProperty("mail.smtp.starttls.enable", "true");

		impl.setJavaMailProperties(mailProp);

		javaMailSender = impl;
		return javaMailSender;
	}

	public void sendMail(String password, final String templateName, final String subject) throws Exception {

		final Map<String, Object> properties = new HashMap<String, Object>();
		properties.put("message", "Hello world!");

		MimeMessagePreparator p = new MimeMessagePreparator() {

			@Override
			public void prepare(MimeMessage mimeMessage) throws Exception {
				MimeMessageHelper message = new MimeMessageHelper(mimeMessage, true);
				message.setTo(toEmail);
				message.setFrom(new InternetAddress(fromEmail));
				message.setSubject(subject);
				message.setText(templateSetup(properties, templateName), true);
			}
		};

		javaMailSender.send(p);
	}

	public static void main(String[] args) {
		MimeMailSender mail = new MimeMailSender();
		String password = "";
		try {
			System.out.print("Enter your password : ");

			BufferedReader br = new BufferedReader(new InputStreamReader(System.in));
			password = br.readLine().trim();
			mail.serverSetup(password);
			br.close();

			mail.sendMail(password, "user.vm","This is test");
		} catch (Exception e) {
			e.printStackTrace();
		}
	}

Template : user.vm

<html>
<head>
</head>
<body>
    <div>${message}</div>
</body>
</html>


Creating link in linux


In linux, there are two types of link, namely : Soft link and Hard link. These link are created by usage of ‘ln‘ command. For more on ‘ln’ command, type

$ ln --h

to get help.


1. Soft link
In linux, there’s a special file type whose data part carries a path to another file. The data in the original file can be accessed through the special file, which is called as Soft Link. Soft link is also called symbolic link or symlink.

To create a soft link :

$ ln -s {/path/to/file-name} {link-name}
$ ln -s /home/pj/Documents/abc.txt xyz.txt
$ ls -l sales.data.txt


To delete a soft link:

$ rm {link-name}
$ rm xyz.txt
$ ls -l /home/pj/Documenst/abc.txt

When a soft link is deleted, original file/directory is not deleted only link is removed, but when original file/direcltory is deleted the link is broken and data is lost.


2. Hard link
A hard link is merely an additional name for an existing file on Linux. The original name and any hard links all point to the same inode.

To create a hardlink:

$ ln {file.txt} {hard-link}
$ ln /home/pj/abc.txt xyz-link

To delete a hard link:

$ rm {hard-link}
$ rm xyz-link


Soft link refer to a symbolic path indicating the abstract location of another file.
Hard link cannot link files across drives, partition. They cannot links directrories.


References:
Linux Gazette
What is Hard link?
The Geek Stuff

Sorting Java Arraylist


The elements of List are shown in order they were added. There are times, when we need to sort the List in certain order. It can be done using Collections class. In this example, we can see use of Collections.sort() to sort the ArrayList. It sort specified list into ascending order, according to the natural ordering of its elements. The elements implements Comparable interface and performs pairwise comparison of the elements and returns 0 if it is equal, -1 if it smaller and 1 if it is larger than compared elements.

public void sort(){
		List<String> unsortedList=new ArrayList<String>();
		System.out.println("Unsorted ArrayList:");
		unsortedList.add("Mercury");
		unsortedList.add("Earth");
		unsortedList.add("Venus");
		unsortedList.add("Mars");
		for(String s : unsortedList){
			System.out.println(s);
		}
		System.out.println("");

		System.out.println("Sorted ArrayList:");
		Collections.sort(unsortedList);
		for(String s : unsortedList){
			System.out.println(s);
		}
	}


Output from above sample:

Unsorted ArrayList:
Mercury
Earth
Venus
Mars

Sorted ArrayList:
Earth
Mars
Mercury
Venus


We can also define implemention based on Comparator interface to sort ArrayList as required. In this example, we create a Java class ‘Task’ with attributes ‘title’ & ‘priority’ of type String and int respectively.

public class Task {
	private String title;
	private int priority;

	public Task(String title, int priority) {
		this.title = title;
		this.priority = priority;
	}

	// getter & setter 
}


We also define a Comparable, which compares every object by attribute priority of type int.

public class ListComparable implements Comparator<Task>{

	@Override
	public int compare(Task t1, Task t2) {
		return (t1.getPriority()<t2.getPriority() ? -1 : (t1.getPriority()==t2.getPriority() ? 0 : 1));
	}
}

Add couple of object of type Task to ArrayList and print it out in console. Without sorting, elements are displayed as they were added to ArrayList. By using own comparable, we can sort the provided the ArrayList in ascending according to their priority value.

public void sort() {
		List<Task> tasks = new ArrayList<Task>();

		tasks.add(new Task("Take out trash", 2));
		tasks.add(new Task("Go to work", 3));
		tasks.add(new Task("Clean room", 1));
		
		System.out.println("Unsorted");
		for (Task t : tasks) {
			System.out.println(t.getTitle()+" "+t.getPriority());
		}
		System.out.println("Sorted");
		Collections.sort(tasks, new ListComparable());
		for (Task t : tasks) {
			System.out.println(t.getTitle()+" "+t.getPriority());
		}
	}


Output from above sample:

Unsorted
Take out trash 2
Go to work 3
Clean room 1

Sorted
Clean room 1
Take out trash 2
Go to work 3

Adding mirrors in linux


If the mirror hostings are not available in source list, you can always add one.
First, to be on safe side, backup the sources.list before editing it.

sudo mv /etc/apt/sources.list /etc/apt/sources.list.old
sudo vi /etc/apt/sources.list


The entries in the file follows like : type, location and components of a repository,
Here is what my sources.list look like after modifying.

deb http://ubuntu.ntc.net.np/ubuntu/ quantal main restricted multiverse
deb-src http://ubuntu.ntc.net.np/ubuntu/ quantal main
deb http://ubuntu.ntc.net.np/ubuntu/ quantal-updates main restricted multiverse
deb-src http://ubuntu.ntc.net.np/ubuntu/ quantal-updates main

deb http://ubuntu.ntc.net.np/ubuntu/ quantal universe
deb-src http://ubuntu.ntc.net.np/ubuntu/ quantal universe
deb http://ubuntu.ntc.net.np/ubuntu/ quantal-updates universe
deb-src http://ubuntu.ntc.net.np/ubuntu/ quantal-updates universe

deb http://ubuntu.ntc.net.np/ubuntu/ quantal-security main restricted multiverse
deb-src http://ubuntu.ntc.net.np/ubuntu/ quantal-security main
deb http://ubuntu.ntc.net.np/ubuntu/ quantal-security universe
deb-src http://ubuntu.ntc.net.np/ubuntu/ quantal-security universe

deb or deb-src, indicates the type of archive: whether it contains binary packages (deb), that is, the pre-compiled packages that we normally use, or source packages (deb-src).
http://ubuntu.ntc.net.np/ubuntu/ is the local repository hosted by NT, where I can find lots of packages necessary for apt-get. It may differ based on your location.

Make sure to run apt-get update after editing the /etc/apt/sources.list, APT will obtain the package lists from the sources mentioned.

TeamCity setup in linux

Table of Contents

  1. Overview
  2. Setup
    • Preparation
    • Installation
    • Auto/Manual start
    • Create user and project
    • Build agent
      • General setting
      • Version Control Settings
      • Build Steps
  3. Troubleshooting


Technologies used
Linux Mint 14 64bit
JDK 1.6.0_33
TeamCity Version 7.1.4 (build 24331)


1. TeamCity
TeamCity is a user-friendly continuous integration (CI) server for developers and build engineers. It is trivial to setup, manage and use.

It scales perfectly for a company or a team of any size. And importantly it is absolutely free for small teams.

2. Setup
2.1. Preparation
Download latest TeamCity version from here. You can either choose free professional edition or enterprise edition with 60 days trial.

2.2. Installation
Unzip/extract the downloaded file to directory where TeamCity will be installed, I’ve mine extracted to “/usr/sw/TeamCity/”.

Change the owner of installation directory from root to user. And create a directory named ‘logs’ if it doesn’t exists inside TeamCity installation directory and assign appropriate permissions. The logs directory will contain all log files generated by TeamCity. These logs will be very helpful if something goes wrong.

2.3. Auto/Manual start
Next, we can create a init script that’ll start the TeamCity when system’ll boot.

The script iteself is simple and basic, contains start and stop command for TeamCity.

#!/bin/sh
# /etc/init.d/teamcity -  startup script for teamcity
export TEAMCITY_DATA_PATH="/var/TeamCity/.BuildServer"

case $1 in
start)
start-stop-daemon --start --exec /usr/sw/TeamCity/bin/teamcity-server.sh start
;;
stop)
start-stop-daemon --start --exec  /usr/sw/TeamCity/bin/teamcity-server.sh stop
;;
esac
exit 0

This init script should be updated and register to system. Now TeamCity will start automatically everytime the daemon starts.

sudo update-rc.d teamcity defaults

Or, we can run TeamCity by executing commands in terminal. By executing the command “./teamcity-server.sh start” from will start the TeamCity server.

$ /usr/sw/TeamCity/bin/./teamcity-server.sh start

2.4 Create user and project
Since, it’s first time TeamCity is running, configuration will take sometime, so have patience.

When the configuration is done, it’ll ask you to create a administrator level user. Enter a username, password and confirm it, admin user is created.

Provide project name and description to create your first project. Creating project is first step but TeamCity won’t start building the project just yet. We need to add configuration that will checkout project from VCS, manage build artifact paths, build counter, build tool etc. It is called build agent in general.

2.5 Build Agent
A Build Agent is a piece of software that actually executes a build process. TeamCity can have numbers of these build agent as per team and project requirements.
2.5.1 General setting

Name: Test Build
Build number format: {0}
Build counter: 1
Artifact paths: test-project/war/Root.war
build/build.properties

For more on build number format, look here
For more on artifact paths, look here

2.5.2 Version Control Settings
Click ‘Create and attach new VCS root’ to setup in which version control system project is hosted, where and branch name that should be checkout for build.

Type of VCS: Mercurial
VCS Root Name: Test Project
Pull Changes From: http://localhost:90/hg/test-project
Default Branch: default
Checking interval: use global server setting (60 seconds)

Test connection if the TeamCity can find the mentioned VCS and project, and save the changes.

2.5.3 Build Steps

Runner type: Maven
Execute step: Only if all previous steps were successfull
Goals: clean install compile

Save the changes.
For more on which execute steps to choose, look here

3. Troubleshooting:
1. Running TeamCity with 64 bits JVM might lead to perm gem space issue. Either increase the perm gem size or refer to here