Check for a slow processor on a cluster

Do you work on a cluster (large collection of computers or processors all connected and working together).  If you do, you may have come across a situation where your code seems to be crawling.  This can be caused by a lot of things and that is not the purpose of this post.  What I'm sharing today is a code I wrote to find which processor is the culprit so the node that holds the processor can be rebooted or fixed by other means.

The code is below and basically times how long it takes to do a bunch of useless work on each processor. I then sorts the times and prints the fastest and slowest results so you can compare.  Finally it tells you which node the slowest processor is on.

FYI, a node is basically a small computer that is part of the cluster.  On the node are usually multiple processors.

Ok, here is the code:


PROGRAM main
    use mpi
    implicit none
    !include 'mpif.h'

    integer :: ierr, n, i, myrank, nproc, temp_i, minIndex, slowest
    real*8    :: x, start_time, end_time, temp_r, mytime
    real*8, dimension(:), allocatable :: time
    integer, dimension(:), allocatable :: rank
    character(len=MPI_MAX_PROCESSOR_NAME) :: myname

    ! Initialize MPI environment
    call MPI_INIT(ierr)
    call MPI_COMM_SIZE(MPI_COMM_WORLD, nproc, ierr)
    call MPI_COMM_RANK(MPI_COMM_WORLD, myrank, ierr)
    call MPI_GET_PROCESSOR_NAME(myname,n,ierr)

    if (myrank.eq.0) print *,'Starting test'
    if (myrank.eq.0) print *,' results accurate to ',MPI_WTICK()

    allocate(  time(nproc));   time=real(0,8)
    allocate(  rank(nproc));

    do n = 1, 10

       ! Start timing
       call MPI_BARRIER(MPI_COMM_WORLD,ierr)
       start_time = MPI_Wtime()

       ! Do a bunch of useless work
       x=real(0,8)
       do i = 1, 100000000
          if (mod(i,2)>0) then
             x = x + real(i,8)
          else
             x = x - real(i,8)
          end if
        end do
 
        ! Stop timing
        end_time = MPI_Wtime()

        ! Collect times
        mytime = end_time - start_time
        call MPI_GATHER(mytime,1,MPI_REAL8, &
          time,1,MPI_REAL8,0,MPI_COMM_WORLD,ierr)

        ! Find slowest processors
        if (myrank.eq.0) then
         
           ! Form rank array
           do i = 1, nproc
              rank(i)=i-1
           end do
         
           ! Selection sort
           do i = 1, nproc-1
              minIndex = minloc(time(i:), 1) + i - 1
              if (time(i) > time(minIndex)) then
                 temp_r = time(i)
                 temp_i = rank(i)
                 time(i) = time(minIndex)
                 rank(i) = rank(minIndex)
                 time(minIndex) = temp_r
                 rank(minIndex) = temp_i
              end if
           end do

           ! Print fastest 5 and slowest 5 processors rank and time
           print *,''
           write(*,'(A,5I10.1,A,5I10.1)') &
            'Rank (fastest-slowest)',rank(1:5),' ...',rank(nproc-4:nproc)
           write(*,'(A,5F10.6,A,5F10.6)') &
            'Time (fastest-slowest)',time(1:5),' ...',time(nproc-4:nproc)

         end if

         ! Print name of slowest processor
         slowest=rank(nproc)
         call MPI_BCAST(slowest,1,MPI_INTEGER,0,MPI_COMM_WORLD,ierr)
         if (myrank.eq.slowest) print *, &
            'The slowest processor is on ',trim(myname)

     end do

     call MPI_BARRIER(MPI_COMM_WORLD,ierr)
     if (myrank.eq.0) print *,'Test complete'

    ! Close parallel environment
    call MPI_FINALIZE(ierr)

END PROGRAM main


Save the code to node-test.f90.  Compile the code using a command like "mpif90 -o node-test node-test.f90"  and then run the code on your cluster using your usual approach.

The output shows the 5 fastest and 5 slowest processor ranks and the time it took each of them to complete the test.  The test is repeated 10 time to see if one processor is consistently the slowest.

If you use the code and find issues or improvements please post a comment below.

High quality LaTeX figures using gnuplot - take 2

This is another way to make high quality figures for use in journal publications.  We will be using gnuplot and LaTaX to generate the figure.  Gnuplot will create an *.eps file that contains the image (lines and axes) and LaTeX will produce the text on the plot using the same font as the rest of your document, creating figures that look great.

Step 1: Create the image files using gnuplot

>> gnuplot
gnuplot>> set term epslatex
gnuplot>> set output "test_figure.tex"
gnuplot>> set xlabel '$\lambda_\theta$'
gnuplot>> set ylabel '$\omega - \mathrm{pi}$'
gnuplot>> set title 'This is a test'
gnuplot>> p sin(x) t 'sin(x)', cos(x) t 'cos(x)'
gnuplot>> exit
>>


Which should have made two files: test_figure.tex and test_figure.eps

Step 2: Compile figure in LaTeX document

\documentclass{article}
\usepackage{graphicx}
\begin{document}
\begin{figure}
\centering
\input{test_figure.tex}
\end{figure}
\end{document}

This code can be compiled using pdflatex and will produce the following figure:

LaTeX circuit diagrams

Making circuit diagrams in LaTeX documents is easy with the circuitikz package.  The following image can be created using the code below.

\documentclass{article}
\usepackage[free-standing-units]{siunitx}
\usepackage[american]{circuitikz}
\usepackage{tikz}

\begin{document}

\begin{figure}[h!]
\centering
\begin{circuitikz} \draw
(0,0) to[V, l=$9V$](0,4)
(0,4) -- (4,4)
(4,4) to[R, l=$R>470\ohm$] (4,2)
(4,2) to[led, l=LED](4,0)
(4,0) -- (0,0)
;
\end{circuitikz}
\caption{Base circuit}\label{fig:circuit1}
\end{figure}

\end{document}


The manual with complete list of symbols can be downloaded from
 http://tug.org/texlive/devsrc/Master/texmf-dist/doc/latex/circuitikz/circuitikzmanual.pdf

Gnuplot - create high quality plot for publication

Open text editor and paste the following code and save the file as plot-gnu

#!/bin/bash

# Output file.pdf
file=trig

gnuplot << TOEND
# Setup output
set term postscript eps enhanced \
dashlength 4 \
linewidth 5 \
"Helvectica" 24\
color

set output "$file.eps"
#unset key
set yrange [-1.1:1.1]

set xlabel 'x'
set ylabel 'y'

# Format plot
p sin(x) t 'sin(x)'\
, cos(x) t 'cos(x)'

TOEND
epstopdf $file.eps
rm $file.eps



Make the code executable by running
chmod +x plot-gnu Run the code with
./plot-gnu You should now have a high quality plot in a file called trig.pdf A useful plot that shows all of the available (black and white) linetypes can be created with the following code. Follow the same steps as above. This code makes a file call linetypes.pdf. #!/bin/bash

# Output file.pdf
file=linetypes

gnuplot << TOEND
# Setup output
set term postscript eps enhanced \
dashlength 4 \
linewidth 5 \
"Helvectica" 24\
#color

set output "$file.eps"
unset key
set yrange [0:10]

set xlabel 'x'
set ylabel 'Linetype'

# Format plot
p 1 lt 1\
, 2 lt 2\
, 3 lt 3\
, 4 lt 4\
, 5 lt 5\
, 6 lt 6\
, 7 lt 7\
, 8 lt 8\
, 9 lt 9

TOEND
epstopdf $file.eps
rm $file.eps


make[3]: m2c: Command not found

make[3]: m2c: Command not found When I get this error it is usually because I used the wrong flag to put the *.mod files in the mod directory thus creating a bunch of *.mod files in the src directory. Removing the *.mod files from the source directory and using the correct compiler flags usually fixes the problem.

Making custom theme for LaTeX

I used the following approach to add a custom beamer theme to a latex presentation that looks like...

Created *.sty file that describes the theme.  Here's mine.
\ProvidesPackage{beamerthemeCTF}
[2008/11/09 v1.0 beamer 'CTF' theme]

\mode

\usebackgroundtemplate{\includegraphics[width=\paperwidth]{CTFLab.pdf}}
\setbeamertemplate{footline}{\vspace{1.2cm}}

\setbeamerfont{author}{size=\large}
\setbeamerfont{date}{size=\large}
\setbeamerfont{example text}{series=\bfseries}
\setbeamerfont{frametitle}{series=\bfseries,size=\large}
\setbeamerfont{framesubtitle}{series=\mdseries,size=\large}
\setbeamerfont{section}{series=\bfseries}
\setbeamerfont{title}{series=\bfseries}

\definecolor{CTFgrey}{RGB}{66,66,66}
\definecolor{CTFlightgrey}{RGB}{175,175,175}
\definecolor{CTFalmostwhite}{RGB}{225,225,225}
\definecolor{CTFred}{RGB}{180,28,27}
\definecolor{CTFtrans}{RGB}{0,0,0}

\setbeamercolor{alerted text}{fg=blue}
\setbeamercolor{block body alerted}{bg=red}
\setbeamercolor{block body}{bg=red}
\setbeamercolor{block body example}{bg=normal text.bg!90!black}
\setbeamercolor{block title alerted}{use={normal text,alerted text},fg=normal text.fg,bg=red}
\setbeamercolor{block title}{bg=blue}
\setbeamercolor{block title example}{use={normal text,example text},fg=example text.fg!75!normal text.fg,bg=red}
\setbeamercolor{fine separation line}{}
\setbeamercolor{frametitle}{fg=CTFred,bg=}
\setbeamercolor{item projected}{fg=black}
\setbeamercolor{normal text}{bg=black,fg=CTFlightgrey}
\setbeamercolor{palette sidebar primary}{use=normal text,fg=normal text.fg}
\setbeamercolor{palette sidebar quaternary}{use=structure,fg=structure.fg}
\setbeamercolor{palette sidebar secondary}{use=structure,fg=structure.fg}
\setbeamercolor{palette sidebar tertiary}{use=normal text,fg=normal text.fg}
\setbeamercolor{section in sidebar}{fg=black}
\setbeamercolor{section in sidebar shaded}{fg= grey}
\setbeamercolor{separation line}{}
\setbeamercolor{sidebar}{bg=red}
\setbeamercolor{sidebar}{parent=palette primary}
\setbeamercolor{structure}{bg=black, fg=green}
\setbeamercolor{subsection in sidebar}{fg=brown}
\setbeamercolor{subsection in sidebar shaded}{fg= grey}
\setbeamercolor{title}{fg=white,bg=}
\setbeamercolor{titlelike}{fg=brown}
\setbeamercolor{itemize item}{fg=CTFred}

\setbeamertemplate{navigation symbols}{}
\setbeamertemplate{headline}{\vspace{0cm}\hspace{0.95\textwidth}\insertframenumber/\inserttotalframenumber\vspace{-9cm}}

\mode

There's some extra stuff in there that I didn't use but wanted to keep all of the color options in the file so I can modify it when I need to do fancy things. :)

Save the file as beamerthemeCTF.sty in ~/Library/texmf/tex/latex/ (you might have to create the folder)

The next step is to put \usetheme{CTF} in your .tex file after \documentclass[mathserif]{beamer}

The last step is to add the background image. Save the picture below as ~/Library/texmf/tex/latex/CTFLab.pdf

Printers/Scanner on Mac

Getting an HP All-in-one printer to print and scan on a Mac caused me a lot of headaches.  I downloaded a variety of drivers and programs from HP hoping to find the magical combination.  I never did.  I did learn that the installers put the drivers in a folder /Library/Printers/hp.  Deleting this folder was useful to start from a clean slate. 

After a lot of searching I found drivers provided by Mac.  They are downloadable from http://support.apple.com/downloads/#hp.  Once I downloaded the drivers, suddenly, the printer and scanner worked at the same time!