軟件背景

雖然很早就接觸了HPC，也參與過一些項目，諸如電影動畫渲染集群以及某博導老師的基因分析計算集群，但是對于跑超算的linpack，一直沒時間上手玩。

Linpack是超算必測項目,也是考驗優(yōu)化能力的套件,很有意思，記錄下安裝過程。

要正常運行，主要需要安裝3個部分組件：

mpi ，這里使用MPICH2

BLAS庫，線性代數(shù)庫 ，intel的還有自己的MKL庫

linpack程序本身

這三個組件，除了linpack本身是標準的外，MPI和BLAS都有多種版本不同的實現(xiàn)，各有千秋和應用場景或者優(yōu)化的特點。當前主要為了跑通，所以mpi選用mipch2,BLAS選用openblas.

軟件地址如下:

?HPL - A Portable Implementation of the High-PerformanceLinpack Benchmark for Distributed-Memory Computershttps://netlib.org/benchmark/hpl/

xianyi (Zhang Xianyi) · GitHubxianyi has 36 repositories available. Follow their code on GitHub.https://github.com/xianyi/openBLAS直接下release里面最新的版本就可以。

Downloads | MPICHhttps://www.mpich.org/downloads/MPICH直接下那個體積最大的包，編譯完成后有Hydra這個程序管理可執(zhí)行程序（mpiexec/mpirun）的。

編譯安裝過程

mpich2

先安裝配置mpi，mpich2安裝配置和其他開源軟件沒啥不同就是解壓，configure make make install 幾步

tar zxf mpich-4.0.3.tar.gz cd mpich-4.0.3 ./configure --prefix=/usr/local/mpich2/ #需要提前安裝好gfortran ,關于網(wǎng)上別的文章說的要添加--with-device=ch4:ofi 參數(shù)，我的configure的時候，是默認給我配置的這個，對于其他deivce類型，有需要優(yōu)化的時候再慢慢看或者做測試吧。 make -j4 sudo make install

?安裝完成后需要配置環(huán)境變量,添加以下行變量到~/.bashrc末尾

export PATH=$PATH:/usr/local/mpich2/bin/ export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/mpich2/lib export MANPATH=$MANPATH:/usr/local/mpich2/share/man

然后拷貝/usr/local/mpich2整個目錄到集群其他節(jié)點對應的位置，如果機器太多建議還是nfs之類的共享吧，否則某個軟件改一下，所有機器又要重新拷貝一遍。

再然后就是配置ssh互信了。

ssh-keygen -t rsa ssh-copy-id uos@192.168.5.249 ssh-copy-id uos@192.168.5.230 #本機IP也要拷貝 #完成拷貝后，直接ssh 192.168.5.249 或者 ssh 192.168.5.230驗證看是否能免密登陸

配置完成后，跑個mpich2自帶的跑圓周率計算的程序

cd /home/uos/Downloads/mpich-4.0.3/examples #注意每個節(jié)點的這個mpich的源碼業(yè)要拷貝過去。 echo "192.168.5.230" >> nodes echo "192.168.5.249" >> nodes mpiexec -f nodes -n 2 ./cpi

openblas?

下載的時候看了下，剛好這最后一版本還支持FT2000了，正好。

tar zxf OpenBLAS-0.3.21.tar.gz cd OpenBLAS-0.3.21/ sudo make PREFIX=/usr/local/openblas/ install

HPL

HPL的編譯主要是mpi庫和blas庫的路徑配置一下，設置下編譯器，直接從解壓的源碼目錄setup下面拷貝一個配置文件到源碼根目錄并改名為Make.FT2000-CBLAS做修改，這個名字后面編譯需要用到，我修改后的配置如下：

# # -- High Performance Computing Linpack Benchmark (HPL) # HPL - 2.3 - December 2, 2018 # Antoine P. Petitet # University of Tennessee, Knoxville # Innovative Computing Laboratory # (C) Copyright 2000-2008 All Rights Reserved # # -- Copyright notice and Licensing terms: # # Redistribution and use in source and binary forms, with or without # modification, are permitted provided that the following conditions # are met: # # 1. Redistributions of source code must retain the above copyright # notice, this list of conditions and the following disclaimer. # # 2. Redistributions in binary form must reproduce the above copyright # notice, this list of conditions, and the following disclaimer in the # documentation and/or other materials provided with the distribution. # # 3. All advertising materials mentioning features or use of this # software must display the following acknowledgement: # This product includes software developed at the University of # Tennessee, Knoxville, Innovative Computing Laboratory. # # 4. The name of the University, the name of the Laboratory, or the # names of its contributors may not be used to endorse or promote # products derived from this software without specific written # permission. # # -- Disclaimer: # # THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS # ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT # LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR # A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE UNIVERSITY # OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, # SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT # LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, # DATA OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY # THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT # (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE # OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. # ###################################################################### # # ---------------------------------------------------------------------- # - shell -------------------------------------------------------------- # ---------------------------------------------------------------------- # SHELL = /bin/sh # CD = cd CP = cp LN_S = ln -s MKDIR = mkdir RM = /bin/rm -f TOUCH = touch # # ---------------------------------------------------------------------- # - Platform identifier ------------------------------------------------ # ---------------------------------------------------------------------- # ARCH = FT2000-CBLAS # # ---------------------------------------------------------------------- # - HPL Directory Structure / HPL library ------------------------------ # ---------------------------------------------------------------------- # TOPdir = /home/uos/Downloads/hpl-2.3 INCdir = $(TOPdir)/include BINdir = $(TOPdir)/bin/$(ARCH) LIBdir = $(TOPdir)/lib/$(ARCH) # HPLlib = $(LIBdir)/libhpl.a # # ---------------------------------------------------------------------- # - Message Passing library (MPI) -------------------------------------- # ---------------------------------------------------------------------- # MPinc tells the C compiler where to find the Message Passing library # header files, MPlib is defined to be the name of the library to be # used. The variable MPdir is only used for defining MPinc and MPlib. # MPdir = /usr/local/mpich2 MPinc = -I$(MPdir)/include MPlib = $(MPdir)/lib/libmpi.so # # ---------------------------------------------------------------------- # - Linear Algebra library (BLAS or VSIPL) ----------------------------- # ---------------------------------------------------------------------- # LAinc tells the C compiler where to find the Linear Algebra library # header files, LAlib is defined to be the name of the library to be # used. The variable LAdir is only used for defining LAinc and LAlib. # LAdir = /usr/local/openblas LAinc = -I$(LAdir)/include LAlib = $(LAdir)/lib/libopenblas_ft2000p-r0.3.21.a # # ---------------------------------------------------------------------- # - F77 / C interface -------------------------------------------------- # ---------------------------------------------------------------------- # You can skip this section if and only if you are not planning to use # a BLAS library featuring a Fortran 77 interface. Otherwise, it is # necessary to fill out the F2CDEFS variable with the appropriate # options. **One and only one** option should be chosen in **each** of # the 3 following categories: # # 1) name space (How C calls a Fortran 77 routine) # # -DAdd_ : all lower case and a suffixed underscore (Suns, # Intel, ...), [default] # -DNoChange : all lower case (IBM RS6000), # -DUpCase : all upper case (Cray), # -DAdd__ : the FORTRAN compiler in use is f2c. # # 2) C and Fortran 77 integer mapping # # -DF77_INTEGER=int : Fortran 77 INTEGER is a C int, [default] # -DF77_INTEGER=long : Fortran 77 INTEGER is a C long, # -DF77_INTEGER=short : Fortran 77 INTEGER is a C short. # # 3) Fortran 77 string handling # # -DStringSunStyle : The string address is passed at the string loca- # tion on the stack, and the string length is then # passed as an F77_INTEGER after all explicit # stack arguments, [default] # -DStringStructPtr : The address of a structure is passed by a # Fortran 77 string, and the structure is of the # form: struct {char *cp; F77_INTEGER len;}, # -DStringStructVal : A structure is passed by value for each Fortran # 77 string, and the structure is of the form: # struct {char *cp; F77_INTEGER len;}, # -DStringCrayStyle : Special option for Cray machines, which uses # Cray fcd (fortran character descriptor) for # interoperation. # F2CDEFS = # # ---------------------------------------------------------------------- # - HPL includes / libraries / specifics ------------------------------- # ---------------------------------------------------------------------- # HPL_INCLUDES = -I$(INCdir) -I$(INCdir)/$(ARCH) $(LAinc) $(MPinc) HPL_LIBS = $(HPLlib) $(LAlib) $(MPlib) # # - Compile time options ----------------------------------------------- # # -DHPL_COPY_L force the copy of the panel L before bcast; # -DHPL_CALL_CBLAS call the cblas interface; # -DHPL_CALL_VSIPL call the vsip library; # -DHPL_DETAILED_TIMING enable detailed timers; # # By default HPL will: # *) not copy L before broadcast, # *) call the BLAS Fortran 77 interface, # *) not display detailed timing information. # HPL_OPTS = -DHPL_CALL_CBLAS # # ---------------------------------------------------------------------- # HPL_DEFS = $(F2CDEFS) $(HPL_OPTS) $(HPL_INCLUDES) # # ---------------------------------------------------------------------- # - Compilers / linkers - Optimization flags --------------------------- # ---------------------------------------------------------------------- # CC = /usr/bin/gcc CCNOOPT = $(HPL_DEFS) CCFLAGS = $(HPL_DEFS) -fomit-frame-pointer -O3 -funroll-loops # # On some platforms, it is necessary to use the Fortran linker to find # the Fortran internals used in the BLAS library. # LINKER = /usr/bin/gfortran LINKFLAGS = $(CCFLAGS) -lpthread # ARCHIVER = ar ARFLAGS = r RANLIB = echo # # ----------------------------------------------------------------------

主要是修改TOPdir??MPdir??LAdir?LINKER?LINKFLAGS幾個位置，編譯的時候報錯，LINKFLAGS 后面需要添加lpthread

修改完成后直接 make arch=FT2000-CBLAS ,編譯完成后可執(zhí)行程序在bin/FT2000-CBLAS/下，主要是xhpl?HPL.dat 兩個文件。做測試的時候調整的參數(shù)都在HPL.dat 文件里面。

可以不用修改直接運行測試跑一下，

cd bin/FT2000-CBLAS/ mpirun -n 4 ./xhpl

如果沒報錯，說明安裝正常。接下來絕大部分時間都需要慢慢調HPL.dat 里面的參數(shù)了，當然MPI以及BLAS庫的優(yōu)化業(yè)是必不可少的。

調參數(shù)未完待續(xù)

HPL Tuninghttps://netlib.org/benchmark/hpl/tuning.html

總結

以上是生活随笔為你收集整理的Linpack安装测试流程记录的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網(wǎng)站內容還不錯，歡迎將生活随笔推薦給好友。