Installing Sonar on ubuntu-14.04

SonarQube is an open platform to manage code quality. We are going install sonarqube-5.6 and sonar-scanner-2.6 on Ubuntu 14.04.

Prerequisites

Java : Oracle JRE 8 or OpenJDK 8
MySQL-5.6 or later version

install OpenJDK 8
sudo apt-get update
sudo apt-get install openjdk-8-jdk

Install MySQL Server

sudo apt-get install mysql-server-5.6

Create the MySQL database for sonar
mysql -u root -p
CREATE DATABASE sonardb;
CREATE USER ‘sonar’@’localhost’ IDENTIFIED BY ‘sonarpassword’;
GRANT ALL PRIVILEGES ON sonardb.* TO ‘sonar’@’localhost’;

Download SonarQube and Sonarscanner

cd /opt
wget https://sonarsource.bintray.com/Distribution/sonarqube/sonarqube-5.6.zip
unzip sonarqube-5.6.zip
wget https://sonarsource.bintray.com/Distribution/sonar-scanner-cli/sonar-scanner-2.6.1.zip
unzip sonar-scanner-2.6.1.zip

Setting the access to the Database for Sonar

Edit /opt/sonarqube/conf/sonar.properties and uncommet and update DB name and password,
sonar.jdbc.username=sonardb
sonar.jdbc.password=sonarpassword

Uncomment the below line which is under MySQL 5.6 section,
sonar.jdbc.url=jdbc:mysql://localhost:3306/sonar?useUnicode=true&characterEncoding=utf8&rewriteBatchedStatements=true&useConfigs=maxPerformance

Start sonar by executing the below command,

/opt/sonarqube/bin/linux-x86-64/sonar.sh start

Now, you can browser sonar in your browser http://your_ip:9000

Edit /opt/sonar-scanner/conf/sonar-scanner.properties and uncomment the below line
sonar.host.url=http://localhost:9000

create a sonar-project.properties file in your codes root directory with the below content,

sonar.projectKey=demo
sonar.projectName=demo
sonar.projectVersion=1.0.0

sonar.sources=app
sonar.language=js
sonar.dynamicAnalysis=reuseReports

Now we are ready to run sonar-scanner, go to your codes root directory and then run

/opt/sonar-scanner/bin/sonar-scanner

Once the sonar-scanner completed, you can see the report details in sonar dashboard http://your_ip:9000

Pentaho 5.4 starting issue

I had situation to install Pentaho 5.4 version in Amazon Linux for POC. The installation is straight forward, just need to download Pentaho from official website and run it. The only requirement is need to have JAVA installed.

I got the below error from pentaho.log file when I was started Pentaho,

ERROR [ContextLoader] Context initialization failed
org.springframework.beans.factory.BeanCreationException: Error creating bean with name ‘org.h2.tools.Server’ defined in file [/opt/pentaho/biserver-ce/pentaho-solutions/system/GettingStartedDB-spring.xml]: Invocation of init method failed; nested exception is org.h2.jdbc.JdbcSQLException: Exception opening port “H2 TCP Server (tcp://localhost:9092)” (port may be in use), cause: “timeout” [90061-131]
Caused by: org.h2.jdbc.JdbcSQLException: Exception opening port “H2 TCP Server (tcp://localhost:9092)” (port may be in use), cause: “timeout” [90061-131]

I noticed that the H2 was trying to bind an invalid IP. So, I added manually the binding “Dh2.bindAddress=localhost” at CATALINA_OPTS in the start-pentaho.sh. My start-pentaho.sh file CATALINA_OPTS part looks like

CATALINA_OPTS=”-Xms1024m -Xmx2048m -XX:MaxPermSize=256m -Dsun.rmi.dgc.client.gcInterval=3600000 -Dsun.rmi.dgc.server.gcInterval=3600000 -Dh2.bindAddress=localhost”

Then I got the below error when I was fixed above error and started Pentaho

ERROR [net.sf.ehcache.Cache] Unable to set localhost. This prevents creation of a GUID. Cause was: ip-172-30-4-54: ip-172-30-4-54: Name or service not known
java.net.UnknownHostException: ip-172-30-4-54: ip-172-30-4-54: Name or service not known 

Made hostname entry in /etc/hosts file to fix the issue.

Upstart script for NodeJS

Upstart is an event-based replacement for the /sbin/init daemon which handles starting of tasks and services during boot, stopping them during shutdown and supervising them while the system is running.

We can use below script to keep up and running NodeJS server,

create a file /etc/init/myapp-node.conf and add the below content to the file,

description “NodeJS Server”
author “admin”

env PORT=1338
env NODE_ENV=production

start on runlevel [2345]
stop on runlevel [016]
respawn

setuid root
chdir /projects/test_project/
exec node app.js

Now start the service, sudo service myapp-node start and it would be up and running.

apache htpasswd authentication exception for a single URL in password procted website

We may need to restrict access to all the pages in a website, except specific URL. We can do this by adding the below lines inside a virtualhost configuration and

SetEnvIf Request_URI “^/source/notification/notify$” NOPASSWD=true
Allow from env=NOPASSWD
Satisfy any

Once added the above lines, we need to reload or restart apache to reflect the changes.

For example:

This configuration should be inside the virtualhost.

<Directory “/home/website/docroot”>
SetEnvIf Request_URI “^/source/notification/notify$” NOPASSWD=true
Order allow,deny
Allow from all
AuthType Basic
AuthName “Restricted Access”
AuthBasicProvider file
AuthUserFile /etc/httpd/htpassword
Require valid-user
AllowOverride all
Allow from env=NOPASSWD
Satisfy any
</Directory>

How to setup password less SFTP login for chroot jailed sftp user

Open the sshd config file and add the AuthorizedKeysFile /sftp/%u/.ssh/authorized_keys next to ChrootDirectory /sftp/%u line.
[root@ip-10-3-0-84 ~]# vim /etc/ssh/sshd_config

Subsystem sftp /usr/libexec/openssh/sftp-server
Match Group sftpusers
ChrootDirectory /sftp/%u
AuthorizedKeysFile /sftp/%u/.ssh/authorized_keys #added line
ForceCommand internal-sftp

save and exit

Restart sshd service

[root@ip-10-3-0-84 ~]# /etc/init.d/sshd restart

Go to sftp user’s directory and perform the below steps, here sftp user is chid.

Note : sftp chroot user will not get his home directory as like normal users.

[root@ip-10-3-0-84 ~]# cd /sftp/chid/
[root@ip-10-3-0-84 chid]# mkdir .ssh
Now add user’s public key into authorized_keys file

[root@ip-10-3-0-84 chid]# vim .ssh/authorized_keys

Change the permissions,
[root@ip-10-3-0-84 chid]# chown -R chid.chid .ssh/
[root@ip-10-3-0-84 chid]# chmod 700 .ssh/
[root@ip-10-3-0-84 .ssh]# chmod 600 .ssh/authorized_keys
Now you can connect the SFTP server using the private key.

sftp -i chid-private.pem chid@sftpserverip

AWStats installation and configuration

To install on the server use below command

rpm -Uhv http://apt.sw.be/redhat/el5/en/x86_64/rpmforge/RPMS/rpmforge-release-0.5.2-2.el5.rf.x86_64.rpm

yum –enablerepo=rpmforge install awstats

or

wget http://prdownloads.sourceforge.net/awstats/awstats-7.3-1.noarch.rpm

rpm -ivh awstats-7.3-1.noarch.rpm

rpm Installation required below dependencies:

GeoIP

perl-Geo-IP

wget ftp://ftp.univie.ac.at/systems/linux/dag/redhat/el6/en/x86_64/dag/RPMS/geoip-1.4.6-1.el6.rf.x86_64.rpm

wget http://pkgs.repoforge.org/perl-Geo-IP/perl-Geo-IP-1.38-1.el6.rf.x86_64.rpm

Change the apache logs format to Combined.

Example:

CustomLog logs/http-access_log combined

By default AWStats creates an Apache configuration file /etc/httpd/conf.d/awstats.conf

Create AWStats Configuration File

cp /etc/awstats/awstats.localhost.localdomain.conf /etc/awstats/awstats.howtolinuxworks.com.conf

Edit the config file and change below options

vim /etc/awstats/awstats.howtolinuxworks.com.conf

LogFile=”/var/log/httpd/howtolinuxworks.com-access_log”

SiteDomain=”howtolinuxworks.com”

HostAliases=”howtolinuxworks.com howtolinuxworks”

Now reload apache configuration to affect the changes

/etc/init.d/httpd reload

Update AWStats Logs using below command

/usr/bin/perl /var/www/awstats/awstats.pl -config=howtolinuxworks -update

Note : above awstats directory will be vary based on installation method

Access AWStats in Browser

http://Server-IP/awstats/awstats.pl?config=howtolinuxworks

To authandicate AWStats URL, add below lines in /etc/awstats/awstats.howtolinuxworks.com.conf

vim /etc/httpd/conf.d/awstats.conf

<Files “awstats.pl”>

AuthUserFile /etc/awstats/passwd

AuthName “Restricted Access”

AuthType Basic

require valid-user

</Files>

To genarete password

htpasswd -c /etc/awstats/passwd admin

Set up cron to update awstats datas

*/30 * * * * /usr/bin/perl /var/www/awstats/awstats.pl -config=howtolinuxworks -update

Note : Some case, no need to schedule cron jobs to update logs because its depend on the installation method.

Apache error – No space left on device: Cannot create SSLMutex

Apache not running and not starting too.

Error:
[Fri Jun 28 11:56:54 2013] [error] (28)No space left on device: Cannot create SSLMutex
Configuration Failed

ipcs -s | grep apache

output:
0x00000000 196610 apache 600 1
0x00000000 7602179 apache 600 1
0x00000000 7667716 apache 600 1
0x00000000 11632645 apache 600 1

To fix the issue we need to delete all the above processes. To achieve this use below commands,

ipcs -s | grep apache | awk ‘{ print $2 }’ > apache-sem.txt

for n in `cat apache-sem.txt`; do
> ipcrm sem $n
> done

or

ipcs -s | grep apache | awk ‘ { print $2 } ‘ | xargs ipcrm sem

Now apache will start successfully.

How to get 403 code from apache access log.

To get 403 code counts from all apache access log
zcat apache_access.log-*.gz | grep ‘HTTP/1.1″ 403’ | awk -F\”GET ‘{ print $2 }’ | awk -F\HTTP\/1.1\” ‘{ print $1 }’ | sort | uniq -c | sort -n -r > output.txt

Yum repository

How to create a local YUM repository?

I am going to explain  how to create a local yum repository with ftp service.

1.First insert and mount your RHEL installation DVD

2.We are going to export the repository through ftp so we need  install vsftpd package
#  cd /media/RHEL_6.1/Packages/
# rpm -ivh vsftpd-2.2.2-6.el6_0.1.x86_64.rpm

3.Create a folder name server in /var/ftp/pub/
# mkdir -p /var/ftp/pub/server

4.Copy all the Packages from DVD to /var/ftp/pub/server/
# cp -var /media/RHEL_6.1/Packages/* /var/ftp/pub/server/

5.One all packages are copied to /var/ftp/pub/server/, we need run createrepo command for that we have to install createrepo-0.9.8-4.el6.noarch.rpm package
# rpm -ivh createrepo-0.9.8-4.el6.noarch.rpm

6.Now we going to run createrepo command, this will create a repository.
# createrepo -v /var/ftp/pub/server/
If your are successfully run this command you can see four .xml file in /var/ftp/pub/server/repodata/

7.Start ftp service
# /etc/init.d/vsftpd start
# chkconfig vsftpd on

8.Now we need to create .repo file in /etc/yum.repos.d/ and add some lines.
# vim server.repo

[server]
name=Red Hat Enterprise Linux
baseurl=ftp://10.2.2.175/pub/Packages
enabled=1
gpgcheck=0

9.Now run this commands
# yum clean all

10.To list the packages
# yum list all

That’s it now your YUM server is ready.

Few Tips

1)# service squid status
# squid pid 20169 is running….
# squid:ERROR:No runningcopy

This error message usually means that the squid.pid file is missing.Since the PID file is normally present  when squid is running, the absence of the PID file usually means Squid is not running.If you accidentally delete the PID file, Squid will continue running, and you won’t be able to send it any signals. If you accidentally removed the PID file, there are two ways to get it back.
run ps and find the Squid process id. You’ll probably see two processes, like this:
# ps ax | grep squid
# 83617 ?? Ss 0:00.00 squid -s
# 83619 ?? S 0:00.48 (squid) -s (squid)

You want the second process id, 83619 in this case. Create the PID file and put the process id number  there. For example:
# echo 83619 > To your squid.pid file

2)# /etc/init.d/httpd status
# httpd dead but pid file exists
Run following command
# rm -rf /var/lock/subsys/httpd
# service httpd start