Bash one-liner: Sort files by scientific number in the filename

The following code takes a list of files and sorts them by a number in the filename. For example data_0.0e0, data_1.0e3, and data_5.0e1 will be sorted to data_0.0e0, data_5.0e-1, data_1.0e3. The output could be piped into another command that needs the ordered files for whatever reason. ls data_* | sed -e 's/data_//g' | sort -g | sed -e 's/^/data_/g'

Script a script and pass inputs

I often have scripts written that do something based on user inputs.  What I want to talk about today is how to use another script to call the first script and provide the user inputs.  That way I can call the scripts a bunch of times and not have to tediously type similar things over and over.  And even better, the way to do this is pretty easy!

Ok here it goes.  Imagine you have a script called repeater that takes a user input and gives it back (not very useful but easy to test with).

Here is my script.
#!/bin/bash
# Repeater

echo "Enter something for me to repeat"
read -e text
echo "You said: $text"

If you call the script, it prompts you for an input and then repeats what you said.  For example
>>./repeater
Enter something for me to repeat
apple
You said: apple


Now here is the interesting part.  We're going to call the script and provide the input, all with one command.
>> ./repeater << EOF
> apple
> EOF


If you want to put this in a script it could look like this
#!/bin/bash

./repeater << EOF
apple
EOF

./repeater << EOF
bannana
EOF

./repeater << EOF
kiwi
EOF

If your script requires multiple user inputs, put them on different lines, e.g.,
>> ./repeater_multiInputs << EOF
> input 1
> input 2
> input 3
> EOF

Check for a slow processor on a cluster

Do you work on a cluster (large collection of computers or processors all connected and working together).  If you do, you may have come across a situation where your code seems to be crawling.  This can be caused by a lot of things and that is not the purpose of this post.  What I'm sharing today is a code I wrote to find which processor is the culprit so the node that holds the processor can be rebooted or fixed by other means.

The code is below and basically times how long it takes to do a bunch of useless work on each processor. I then sorts the times and prints the fastest and slowest results so you can compare.  Finally it tells you which node the slowest processor is on.

FYI, a node is basically a small computer that is part of the cluster.  On the node are usually multiple processors.

Ok, here is the code:


PROGRAM main
    use mpi
    implicit none
    !include 'mpif.h'

    integer :: ierr, n, i, myrank, nproc, temp_i, minIndex, slowest
    real*8    :: x, start_time, end_time, temp_r, mytime
    real*8, dimension(:), allocatable :: time
    integer, dimension(:), allocatable :: rank
    character(len=MPI_MAX_PROCESSOR_NAME) :: myname

    ! Initialize MPI environment
    call MPI_INIT(ierr)
    call MPI_COMM_SIZE(MPI_COMM_WORLD, nproc, ierr)
    call MPI_COMM_RANK(MPI_COMM_WORLD, myrank, ierr)
    call MPI_GET_PROCESSOR_NAME(myname,n,ierr)

    if (myrank.eq.0) print *,'Starting test'
    if (myrank.eq.0) print *,' results accurate to ',MPI_WTICK()

    allocate(  time(nproc));   time=real(0,8)
    allocate(  rank(nproc));

    do n = 1, 10

       ! Start timing
       call MPI_BARRIER(MPI_COMM_WORLD,ierr)
       start_time = MPI_Wtime()

       ! Do a bunch of useless work
       x=real(0,8)
       do i = 1, 100000000
          if (mod(i,2)>0) then
             x = x + real(i,8)
          else
             x = x - real(i,8)
          end if
        end do
 
        ! Stop timing
        end_time = MPI_Wtime()

        ! Collect times
        mytime = end_time - start_time
        call MPI_GATHER(mytime,1,MPI_REAL8, &
          time,1,MPI_REAL8,0,MPI_COMM_WORLD,ierr)

        ! Find slowest processors
        if (myrank.eq.0) then
         
           ! Form rank array
           do i = 1, nproc
              rank(i)=i-1
           end do
         
           ! Selection sort
           do i = 1, nproc-1
              minIndex = minloc(time(i:), 1) + i - 1
              if (time(i) > time(minIndex)) then
                 temp_r = time(i)
                 temp_i = rank(i)
                 time(i) = time(minIndex)
                 rank(i) = rank(minIndex)
                 time(minIndex) = temp_r
                 rank(minIndex) = temp_i
              end if
           end do

           ! Print fastest 5 and slowest 5 processors rank and time
           print *,''
           write(*,'(A,5I10.1,A,5I10.1)') &
            'Rank (fastest-slowest)',rank(1:5),' ...',rank(nproc-4:nproc)
           write(*,'(A,5F10.6,A,5F10.6)') &
            'Time (fastest-slowest)',time(1:5),' ...',time(nproc-4:nproc)

         end if

         ! Print name of slowest processor
         slowest=rank(nproc)
         call MPI_BCAST(slowest,1,MPI_INTEGER,0,MPI_COMM_WORLD,ierr)
         if (myrank.eq.slowest) print *, &
            'The slowest processor is on ',trim(myname)

     end do

     call MPI_BARRIER(MPI_COMM_WORLD,ierr)
     if (myrank.eq.0) print *,'Test complete'

    ! Close parallel environment
    call MPI_FINALIZE(ierr)

END PROGRAM main


Save the code to node-test.f90.  Compile the code using a command like "mpif90 -o node-test node-test.f90"  and then run the code on your cluster using your usual approach.

The output shows the 5 fastest and 5 slowest processor ranks and the time it took each of them to complete the test.  The test is repeated 10 time to see if one processor is consistently the slowest.

If you use the code and find issues or improvements please post a comment below.

High quality LaTeX figures using gnuplot - take 2

This is another way to make high quality figures for use in journal publications.  We will be using gnuplot and LaTaX to generate the figure.  Gnuplot will create an *.eps file that contains the image (lines and axes) and LaTeX will produce the text on the plot using the same font as the rest of your document, creating figures that look great.

Step 1: Create the image files using gnuplot

>> gnuplot
gnuplot>> set term epslatex
gnuplot>> set output "test_figure.tex"
gnuplot>> set xlabel '$\lambda_\theta$'
gnuplot>> set ylabel '$\omega - \mathrm{pi}$'
gnuplot>> set title 'This is a test'
gnuplot>> p sin(x) t 'sin(x)', cos(x) t 'cos(x)'
gnuplot>> exit
>>


Which should have made two files: test_figure.tex and test_figure.eps

Step 2: Compile figure in LaTeX document

\documentclass{article}
\usepackage{graphicx}
\begin{document}
\begin{figure}
\centering
\input{test_figure.tex}
\end{figure}
\end{document}

This code can be compiled using pdflatex and will produce the following figure:

LaTeX circuit diagrams

Making circuit diagrams in LaTeX documents is easy with the circuitikz package.  The following image can be created using the code below.

\documentclass{article}
\usepackage[free-standing-units]{siunitx}
\usepackage[american]{circuitikz}
\usepackage{tikz}

\begin{document}

\begin{figure}[h!]
\centering
\begin{circuitikz} \draw
(0,0) to[V, l=$9V$](0,4)
(0,4) -- (4,4)
(4,4) to[R, l=$R>470\ohm$] (4,2)
(4,2) to[led, l=LED](4,0)
(4,0) -- (0,0)
;
\end{circuitikz}
\caption{Base circuit}\label{fig:circuit1}
\end{figure}

\end{document}


The manual with complete list of symbols can be downloaded from
 http://tug.org/texlive/devsrc/Master/texmf-dist/doc/latex/circuitikz/circuitikzmanual.pdf

Gnuplot - create high quality plot for publication

Open text editor and paste the following code and save the file as plot-gnu

#!/bin/bash

# Output file.pdf
file=trig

gnuplot << TOEND
# Setup output
set term postscript eps enhanced \
dashlength 4 \
linewidth 5 \
"Helvectica" 24\
color

set output "$file.eps"
#unset key
set yrange [-1.1:1.1]

set xlabel 'x'
set ylabel 'y'

# Format plot
p sin(x) t 'sin(x)'\
, cos(x) t 'cos(x)'

TOEND
epstopdf $file.eps
rm $file.eps



Make the code executable by running
chmod +x plot-gnu Run the code with
./plot-gnu You should now have a high quality plot in a file called trig.pdf A useful plot that shows all of the available (black and white) linetypes can be created with the following code. Follow the same steps as above. This code makes a file call linetypes.pdf. #!/bin/bash

# Output file.pdf
file=linetypes

gnuplot << TOEND
# Setup output
set term postscript eps enhanced \
dashlength 4 \
linewidth 5 \
"Helvectica" 24\
#color

set output "$file.eps"
unset key
set yrange [0:10]

set xlabel 'x'
set ylabel 'Linetype'

# Format plot
p 1 lt 1\
, 2 lt 2\
, 3 lt 3\
, 4 lt 4\
, 5 lt 5\
, 6 lt 6\
, 7 lt 7\
, 8 lt 8\
, 9 lt 9

TOEND
epstopdf $file.eps
rm $file.eps