List file names with modified time stamp customized

Commands used to list files from a directory yields results of more details and time stamp with GMT values. To customize the output to display the file names only with time stamp in custom format with sorted order, use:

$ find . -mindepth 1 -maxdepth 1 -type f -printf "%TY-%Tm-%Td %TH:%TM:%.2TS %f\n" | sort -n 

Here the mindepth and maxdepth parameters were used to use the current directory for the search. That can be modified as per need. And the output will be,

2014-04-22 19:34:36 .bash_logout
2014-04-22 19:34:36 examples.desktop
2014-04-22 19:34:36 .profile
2014-04-22 21:26:24 dropcaches.sh
2014-04-22 22:09:20 .bashrc
2014-05-04 08:41:42 .dmrc

Bulk rename using bash rename command

Rename command with RegEx pattern can be used to do the bulk rename operation. For example, to append the ip address of the machine in between the log files supplied, the below code is used.

#!/bin/bash
ip=$(hostname -I | sed 's/[^0-9.]*//g')
file="_"$ip"."
rename -v "s/(\w+).(\w+)+/\$1$file\$2/" *.log*

The output will like:

shell.log renamed as shell_192.168.1.2.log
shell.log.1 renamed as shell_192.168.1.2.log.1
shell.log.2 renamed as shell_192.168.1.2.log.2
shell.log.zip renamed as shell_192.168.1.2.log.zip

Apache authentication and authorization using LDAPS

Apache web Authentication using LDAP/LDAPS requires two steps.

1. Enabling public key at client side
2. Configure LDAPS server in Apache (assuming that your LDAPS server is already running)

To enable secure connection with LDAPS server, the public key should be installed at Apache machine server follow below steps

1. Copy the key in to /etc/ldap/cacerts or /etc/openldap/cacerts
2. Configure the key entry in /etc/ldap.conf

To configure Apache server to communicate LDAPS server for authentication, add this into Apache http.conf or default.conf file.

LDAPTrustedGlobalCert CA_BASE64 /etc/openldap/cacerts/ldap_pubkey.pem
<Directory /var/www/html/>
    AuthName "Apache authentication using Ldaps Server"
    AuthType Basic
    AuthBasicProvider ldap
    AuthzLDAPAuthoritative off
    AuthLDAPURL ldaps://ldaps.test.com:636/ou=users,dc=test,dc=com?uid
    AuthLDAPBindDN cn=manager,dc=test,dc=com
    AuthLDAPBindPassword <pwd>
    #Allowed user list
    Require ldap-user user1 user2

    Options Indexes FollowSymLinks
    AllowOverride All
    Order allow,deny
    Allow from all
</Directory>

Split PDF into multiple with n number of pages each

The following Linux shell script splits the source PDF file into multiple files with n pages of each. Totall no of pages in given input file calculated automatically and user needs to give the pages wants in each output file. This can be configured using the variable step. Default it splits the files with 5 pages.

#!/bin/bash
file="src/test.pdf";
#count total no of pages in the file
total_pages=$(pdfinfo $file | grep Pages | awk '{print $2}')
from=1
#step defines how many pages should be in output files
step=5;
while [ $from -lt $total_pages ]
do
        to=$((from+step-1));
        if [ $to -gt $total_pages ]; then
                to=$total_pages;
        fi
        echo "Generating pdf file : pages-$from-$to.pdf...";
        pdftk $file cat $from-$to output "test-$from.pdf";
        from=$((to+1));
        echo "Done.";
done

Drop virtual memory cache from Linux box to free RAM space

In performance perspective mode, Linux box is designed to keep the frequently used application's data in memory to boost up system performance. System with low RAM space will affected by opposite manner that the cache occupies RAM space mostly and system got slow even got hanged also. Apart from theory, we need to resolve that to run the machine without hang.

To free such cache from RAM, run the below command often. This will erase the cache with simple value what we give.

$ echo 3 > /proc/sys/vm/drop_caches

Remove line numbers from source code using sed

While getting sample code from internet, it may comes with line number. To simply remove those lines from the code file use the below ways.

Using sed command with RegEx pattern in Linux, that can be achieved.

To remove it from console:

$ sed -i "s/^ *[0-9.]* *//g" code.txt

To remove it from vim interface:

:%s/^ *[0-9.]* *//g

And the same RegEx pattern can be used in anywhere to remove the line numbers from the source code file.

 

Refresh system settings from sysctl.conf without restart Linux

The custom settings entered in sysctl.conf can be easily revoked without restarting the Linux box. Using sysctl command with -p argument refreshes the machine with new settings.

$ sysctl -p

Finding last occurrence of a label and its value in Spreadsheet

	A	B	C	D	E	F
1		A	11		A	19
2		B	12		B	16
3		C	13		C	15
4		D	14		D	18
5		C	15
6		B	16
7		A	17
8		D	18
9		A	19

Finding a value of label that occurs as last in a large list is always complex one. Spreadsheet's lookup functions will helps up to some level but not completely. Using the following workaround, its possible and the desired value can be populated.

Feed the below values in respective cells and press CTRL+SHIFT+ENTER to run the script.

F1 = INDEX($C$1:$C$9,MAX(ROW($A$1:$A$9)*($B$1:$B$9=$E1)))
F2 = INDEX($C$1:$C$9,MAX(ROW($A$1:$A$9)*($B$1:$B$9=$E2)))
F3 = INDEX($C$1:$C$9,MAX(ROW($A$1:$A$9)*($B$1:$B$9=$E3)))
F4 = INDEX($C$1:$C$9,MAX(ROW($A$1:$A$9)*($B$1:$B$9=$E4)))

Here Column "A" was referred just for row number reference nothing else. Column "A" may have any value or empty.

Note: Labels are A-D and values are numeric values.

Table row into Column using unnest in PostgreSQL

PostgreSQL result rows can be converted to Columns using Array function called unnest.

For example, the below query result

uid | name | mail
---------------------------------
01 | admin | admin@test.com

can be converted like below.

ColumnName | Value
---------------------------------
UID         | 01
Name | admin
Mail         | admin@test.com
----------         | ----------
Total Count | 1

Use this query to convert row values into columns:

--unnest to give array as rows
WITH
   x AS (
SELECT *
FROM   users t where uid > 0
   ),
   y AS (
SELECT ARRAY [uid::varchar, name, mail] AS val,
ARRAY ['UID','Name','Mail'] AS item
FROM   x
   )
SELECT unnest(item) AS ColumnName,
unnest(val) AS Value
FROM   x,y
UNION  ALL
SELECT '----------'::text, '----------'::text
UNION  ALL
SELECT 'Total Count'::text, count(x)::text from x

Clear sudo password remember in linux

Normally Linux will keep the session of sudo command in terminal for a while and it will not ask for password again when try to use the sudo before the expiry time. You can resolve this by clearing the session immediately by giving the below command.

sudo -k

Custom access denied error page for sites/default/files url in Apache

User will get access denied page after setting up standard codes in htaccess file of site/default/files folder. This error page can be customized by adding our used defined html page to that.

The page can be showed by redirecting from access denied default page by adding the below line in htaccess file.

ErrorDocument 404 /error/HTTP_NOT_FOUND.html.var

It redirects and shows the page from Apache's default error page where user either can modify the code of that file or add new custom html file.

Compile shell scripts to encrypted binaries using SHC

Shell scripts are the plain code files and it could be modified easily when in the case of distributing. To avoid that the scripts can be converted to some other format like binary. Using SHC library the Unix shell scritps can be converted to encrypted binaries so that reverse engineering of that is not that much easy.

The Debian or Tar source of SHC are availble in Open Source repositories. Install the tool and give the below command to make the object file.

$ shc -v -r -T -f test.sh
$ mv test.sh.x test
$ ./test 

This will output one object file and one c file. Just rename output file with the suffix ".x" (ie.test.sh.x) to some name and use it anyware.

Bulk Convert Office Documents into PDF using unoconv

There are many methods availabel to convert bulk amount of document files into PDF files. Using Using Open Source libarary/tool we can able to do it programatically. (ie. Open Office headless service, unoconv and etc.,).

The drawback of using Open Office headless service in the case of files are in sub directories too is it will output the files in current working directory instead of actual source file path. In that case we can use unoconv tool. It supports all the formats that Open Office supports.

Unoconv

       Unoconv is a command line utility that can convert any file format that LibreOffice can import, to any file format that LibreOffice is capable of exporting. To know what are the types it is supporting,

$ unoconv --show

Example

       To convert the list odt files from path and its sub directories use the below code. This can be modified to different input and output formats as required.

#!/bin/bash
#To avoid errors due to spaces in file names
IFS="$(printf '\n\t')"
#Finds odt files in current directory and its sub directories and process one by one
for file in $(find . -name "*.odt" -type f); do
  echo "Processing File : $file ...";
  unoconv -d document -f pdf "$file";
Done

Find string from list of ODT files in Linux

In Linux, a string pattern can be searched using grep command from one or more files from a directory (including its sub directories). Normally the search will be easy when in the case of text files including program source code files like c, PHP, Java and etc. At the same time a search from list of ODT files from a directory is not that easy.

This can be also possible with the use of utility command such as find, unzip and grep. The below code will done the job perfectly. The parameter can be changed upon your wish. This program will output file name(s) which contains the search string.

#content of searchodt.sh

#!/bin/bash

if [ $# -ne 1 ]; then
    echo "Usage: sh searchodt.sh <searchterm>"
    exit 1
fi

for file in $(find . -name "*.odt" -type f); do
    unzip -ca "$file" content.xml | grep -qil "$1"
    if [ $? -eq 0 ]; then
        echo "$file"
    fi
Done

Selective cell protection in Spreadsheet

Protecting sheet will locks the entire sheet and not allows users to edit any cell value. In some scenarios, some of the limited cells can be locked by using the following workaround.

• Select the cells that not to be locked.
• Right click and select "Format" and then Select "Cell Protection" Tab.
• Uncheck the protected option to skip from lock.
• Select Tools Menu → Protect Document → sheet and give password to lock entire sheet. Uncheck "Select Unprotected cells" option to prohibit user even to select the protected cells if needed. 

Here after user can able to enter or modify the values only on allowed cells.

Repeatable file changes made easy with SED command in Linux

Using sed command we can add desired text at desired line in a file. That can be a comment, description or anything.

The below code inserts the comment text at the line 2 at each PHP  file inside the current directory.

    #!/bin/bash

    for fileItem in ./*.php
    do
        echo "Updating file : $fileItem"
        sed '2i\//      Author: Senthilkumar C' $fileItem > tmp.php
        cp tmp.php "$fileItem"
    done
    #remove temp file
    rm tmp.php

Disable header and footer lines printed with web pages in Firefox

Task:
           To disable the header and footer lines getting printed with Web Pages while printing from Firefox permanently. In general, users needs to disable these settings in printer settings window each time while give print. The procedure given below will disable the options permanently.

Solution:
            Firefox itself has its own header and footer settings for this purpose and disabling of this will not fulfill the complete task what we want. For that we need to disable that settings in printer side also. To do so,

            Open Firefox and run the url 'about:config' to get the Firefox's settings page.

For Header,

1. To filter out the settings respect to our need, give '_header'
2. Unset all the values related to header such as '&T', '&D' and etc., from both Firefox and Printer that you have installed

For Footer,

1. To filter out the settings respect to our need, give 'footer'
2. Unset all the values related to header such as '&T', '&D' and etc., from both Firefox and Printer that you have installed

From now, no Header and Footer lines will be there in printouts that print.

Note : This is tested under Ubuntu 11.10.

Setting up default print page orientation in firefox

Task:
           To change the default printer page orientation setting on Firefox browser to new one instead of set by OS default. Even, if the orientation setting was set in printer driver interface or administration tool given by the respective vendor, Firefox may show different setting. To change that in Firefox use the procedure given below.
Solution:
            Orientation setting of pages to be printed from Firefox using installed printers in a system set by Operating System or the vendor driver software of the respective printers while installing or configuring of that by default. This can be reconfigured at any time by editing Firefox setting. To view and edit the settings,

1. Run  'about:config' from address bar. This will shows entire list of configurations of Firefox
2. To filter that to orientation, type 'orientation' on filter text box and you will get results given below (like similar not exactly, because printer names may vary)

   

   Orientation settings of Firefox for Printers
   
   It shows config values of two printers installed. In that, 3'rd and 4th items are denotes the numeric values of each setting (0 - portrait, 1 - Landscape)

   
   
3. Change the last two line(for two different printers), to whatever the setting you want. Here i have set 'portrait' for both printers by setting the value of '0'.

          From now, the default page orientation for the page to be printed will be changed to new setting what you set before.

Note : This is tested under Ubuntu 11.10 and Firefox version is : 7.0.1

Save entire console session actions into a file in Linux

Task:

          While installing or debugging a task in Linux, previous session history will helps a lot to review the work what we have done when if any error occurs. For this, we need to store the entire session action and its output into a file or some where.

Solution:
           This can be achieved by more commands depends of the user requirement.

1. Command wise
   
1. Using output redirect
       This is the simplest command used to save our console outputs into a file.
       $ ls -al > out.txt
       $ ls -al >> out.txt
   
   Here, '>' used to redirect the output to a file and '>>' is used to append the new output with existing one. Main drawback of this command is, the console will not display anything. All the outputs will goes to file instead of console.
   
2. Using 'tee' command
       This command overrides the drawback of previous method. This will send the copy of the output only to the file. So that the normal output is displayed in console as it is.
   
   $ ls -al |tee out.txt
   
   This will erase the previous contents while writing new one. To avoid this use the following syntax to append
   
   $ ls -al |tee -a out.txt
   Main drawbacks of previous two commands is, we need to give the tailing commands (either > or tee) with each and every commands we are executing in that console.
   
   
2. Session wise
   
   In this method, the entire session from the start to end will be save given file. This contents will be written into the file once we close the terminal.
   
1. Using 'script' command
      Run the following command before starting any execution in console to start recording.
   
   $ script -a today.txt
   
   Then do your usual work and once complete the tasks then quits the terminal and see the saved file contents like,
   
   $ more today.txt
   
   

Note: This is tested under Ubuntu 11.10

View tamil contents properly in Chrome under Ubuntu

Under Ubuntu or some other Linux flavors, Chrome browser doesn't have enough Tamil core fonts (UTF or Local) by default to render/display the contents correctly. So we need to install the required Tamil fonts to render properly.


In Ubuntu, 
         Install the font package called 'ttf-tamil-fonts' to get set of Tamil fonts. This will install the fonts in system font folder. So that this can be used by any other applications system wise not only Chrome.