- \n
- Use Compiler Optimizations<\/a><\/li>\n
- Write Files Efficiently<\/a><\/li>\n
- Access Arrays Efficiently<\/a><\/li>\n
- Use External Libraries<\/a><\/li>\n<\/ul>\n
Advanced Options<\/h4>\n
- \n
- Don’t Guess, Just profile<\/a><\/li>\n
- Parallelize Your Code<\/a><\/li>\n<\/ul>\n
Easy Options<\/h2>\n
<\/a>Use Compiler Optimizations<\/h4>\n
Compiler flags guide the compiler to better optimization.<\/em><\/p>\n
If you are working with a compiled language, e.g. C, C++ and Fortran, then there are several optimization flags that you can pass to the compiler as you build your program. When it runs, a compiler not only translates your application to a machine-readable format, but it also adjusts your instructions in an attempt to automatically make your application faster. These compiler flags provide guidance to the compiler to help it optimize this process.\u00a0Our compilers<\/a> page documents these options for the various compilers we support. Documented there is: how to control the optimization process for each compiler; how to set these flags during the build process, e.g. what environment variables you can set to affect make; and what the trade offs are for using certain flags.<\/p>\n
<\/a>Write to Files Efficiently<\/h4>\n
Write locally, avoid writing 1000’s of files, and avoid repeatedly\u00a0opening\u00a0and closing files, and your\u00a0program will run faster*.<\/em><\/p>\n
*This advice is always good to follow, but actual performance gains really depend what you’re doing.<\/p>\n
First, if you’re going to write to a file, organize your program so that you open your file once and close it at the end. There is overhead in opening and closing a file, and it really adds up if you are doing this each time you want to save information.<\/p>\n
Next, there are two important things to know about our file system. First, most of the storage, including all types of Project Disk Space and home directories, are stored on a high performance file system and are accessed by each compute node through the network. While this involves a fast network, it is usually\u00a0<\/strong>faster to first write files to a local drive (it’s never slower, but sometimes it’s pretty much a tie). Each compute node has it’s own local storage area called
\/scratch<\/code>. Programs sometimes can benefit greatly from writing to\u00a0\/scratch<\/code>\u00a0while running, and the user can then copy the files to project disk space following their run’s completion.\u00a0\u00a0The second point is about how many files to write. \u00a0It is more efficient to write one large file versus many (thousands of) small files. If you have questions about this, please email us.<\/p>\nThe environment variable
$TMPDIR<\/code> is set each time you run a job. It represents the name of a personal, customized scratch directory that will only last as long as your job does. This is the best option when your local files are temporary, because you know<\/em> that this directory is empty and you can write to it. You can store your results locally, and copy things to your Project Disk Space at the very end of your script. <\/p>\nLocal Scratch Space<\/h5>\n
The scratch disks are available as temporary storage space. They are open to all SCF users and there is no preset quota for use. If you abuse the fair-share nature of\u00a0
\/scratch<\/code>, your files may be deleted.<\/em>\u00a0Files stored on any scratch disk are NOT BACKED UP and these files can only be kept in\/scratch<\/code> for up to 31 days. Files not removed from\/scratch<\/code> by the owner will be deleted by the system after 31 days.<\/p>\nAll of the SCC nodes have their own
\/scratch<\/code> disk. You can access a specific scratch disk via the pathname\/net\/scc-xx#\/<\/span>scratch<\/code>, where\u00a0xx<\/em>\u00a0is a two-letter string such as “ab” and\u00a0#<\/em>\u00a0is a number such as “5” or “13”. See\u00a0Technical Summary<\/a>\u00a0for the list of node names. For example,<\/p>\nscc1% cd \/net\/scc-ab5<\/span>\/scratch<\/code><\/pre>\n- \n
- The SCC login and compute nodes each have significant amounts (427+ GB+) of scratch disk space; specific\u00a0size may vary by node<\/a>.<\/li>\n
- On a compute node, a reference to\u00a0
\/scratch<\/code>\u00a0points to the local node’s\u00a0\/scratch<\/code>\u00a0at runtime.<\/li>\n- Similarly, if you are on the login node, type “
cd<\/span> \/scratch<\/code>” to access its own scratch.<\/li>\n- You can access the login node’s scratch from any compute node with
\/net\/scc\/scratch<\/code>.<\/li>\n- If you’d like to use scratch space in a batch job, please use the\u00a0scratch space of the compute nodes<\/a>\u00a0assigned to the job. (See Item 2 above.)<\/li>\n<\/ol>\n
When does it matter to write locally?<\/h5>\n
- \n
- You are compressing your results (e.g. using
gzip<\/span><\/code>). It’s fastest to compress locally and to then move the final compressed file to the networked storage than it is to write the original results on the networked spaces before compressing them.<\/li>\n- You are writing temporary files, and once completed, these files can be deleted.<\/li>\n<\/ol>\n
<\/a>Access Arrays Efficiently<\/h4>\n
When accessing arrays in a loop, keep in mind how the language actually stores multidimensional arrays.<\/em><\/p>\n
Multidimensional arrays are always stored internally as one dimensional arrays. For some languages, e.g. Fortran, R, Matlab<\/strong>, you want to loop through the first index in the inner most loop. This is called column-major<\/strong> or Fortran-order<\/strong> array access. For other languages, e.g. C, C++<\/strong>, you want to loop through the last index in the inner most loop, and correspondingly this is called row-major<\/strong> or C-order<\/strong> array access. Let’s look at an example of this.<\/p>\n
Consider a one dimensional array that you are looping through; this is like a conveyor belt of information getting ready to be processed. It is most efficient to process the values as they come:<\/p>\n
![\"array_access_conveyer_belt_good\"](\"\/tech\/files\/2014\/05\/array_access_conveyer_belt_good1.png\")
<\/a><\/p>\nFor\u00a0row-major<\/strong>\u00a0order, a 2 row by 3 column array<\/p>\n
[1 2 3 \r\n 4 5 6]<\/pre>\n
is really stored as a one dimensional array:<\/p>\n
[1 2 3 4 5 6]<\/pre>\n
and it is faster to loop through the last index in the inner most loop, like so:<\/p>\n
for (row = 0; row < 2; row++) {\r\n for (col = 0; col < 3; col++) {\r\n printf (\"%d \\n\", myarray[row][col]);\r\n }\r\n}<\/code><\/pre>\nThis way, you will access the array as if it were a one dimensional array. If you looped through the rows in the inner most loop, you will jump around the underlying one dimensional array, in an inefficient manner:<\/p>\n
![\"array_access_conveyer_belt_bad\"](\"\/tech\/files\/2014\/05\/array_access_conveyer_belt_bad.png\")
<\/a><\/p>\nThis will not matter for very small arrays, but for the large arrays commonly used in research computing, accounting for the underlying storage leads to better performance.<\/p>\n
For\u00a0column-major<\/strong> order (used by Fortran, R <\/strong>and\u00a0Matlab<\/strong>), a 2 row by 3 column array<\/p>\n
[1 2 3 \r\n 4 5 6]<\/pre>\n
is really stored as a one dimensional array:<\/p>\n
[1 4 2 5 3 6]<\/pre>\n
and it is faster to loop through the first index in the inner most loop:<\/p>\n
do col=1, 3\r\n do row=1, 2\r\n print *, myarray[row, col]\r\n end do\r\nend do<\/code><\/pre>\nThese rules hold for 3 dimensions or more.\u00a0In short, it is best to access the array in the same way it is stored internally, and this order is language dependent.<\/p>\n
<\/a>Use External Libraries<\/h4>\n
Use an external library for access to already-optimized code.<\/em><\/p>\n
For each language, there are many 3rd party libraries that save you time by providing optimized and tested code. We recommend researchers start with these to develop their code, and only once they’ve found that an alternative, unimplemented approach is needed, then they should develop their own code. In areas of parallelization, efficient IO (e.g. file access), and numerical methods, it is much faster to use code that is already available. For example, FFTW<\/a> is a library that performs Fast Fourier Transforms. It is used throughout research computing, so it has been well tested and there are abundant help resources. It also provides both serial (single processor) and parallel (cluster) versions, and it is relatively easy to update your code to use parallel resources. In short, it saves development and execution time to use this library.<\/p>\n
If you need help finding a library to help your research, feel free to contact us, and if you find can’t find a particular library on our clusters<\/a>, please let us know that we should install it<\/a>.<\/p>\n
Advanced Options<\/h2>\n
<\/a>Don’t Guess, Just Profile<\/h4>\n
Profile your code to find what functions and subroutines need to be optimized.<\/em><\/p>\n
Profiling code means measuring how long it takes to run various parts of your program. Profiling your code, either manually or by using external tools, provides empirical evidence of what part of your code needs the most work. With this evidence, you can focus your efforts in areas that actually matter. Often the trade off for faster code is more complex, i.e. harder-to-read, code. The cost of making your code hard to read should be expended only when it matters. You can email us for help with profiling your code, or you can explore your options<\/a> at your own leisure.<\/p>\n
<\/a>Parallelize your code<\/h4>\n
Run your code simultaneously on multiple CPU cores, machines or GPU cores to get work done faster.<\/em><\/p>\n
If your application involves independent tasks, then it may be advantageous to complete them simultaneously using parallelization. If you’re not sure this is the case, please contact us to review your code and talk about your options. There are three main technologies to parallelize your code, and we provide details<\/a> about programming and running these applications.<\/p>\n
OpenMP is a library that automatically generates the code to run your application on many threads. On its own, it can only be used on one machine at a time. The advantage of this library is that it is called in special comments and evoked using special compiler flags. So you can essentially leave your code unaffected as long as you don’t use the special compiler options. With this option, you can theoretically gain 16x efficiency on our systems.<\/p>\n
The next technology is MPI (Message Passing Interface). It automatically runs multiple copies of your application on multiple machines, assigns each execution a unique ID, and manages the network communications to send information between these executions. This library allows you to use many more CPU cores, because it can communicate across machines. It does require changes to your code, which then must access mpirun<\/strong> (a tool to start your new parallel application) and the MPI library.This means you will need a different copy of your code\/application. With this option, you can theoretically gain >16x efficiency on our systems.<\/p>\n
Finally, GPUs provide many cores that speed up calculations, especially those involving array manipulations. To use this option, you must use CUDA, an NVIDIA GPU specific language that comes in C, C++ and Fortran like versions. Fortunately there are many 3rd party libraries that use this technology, allowing you to use GPUs without understanding how to program CUDA. Just as OpenMP is a tool to automatically parallelize applications that use CPUs, a technology called OpenACC is an option for automatically parallelizing your code for GPUs. This might be the easiest option to get started with GPU programming, and we have notes for using OpenACC with C\/C++<\/a> and Fortran<\/a> applications, respectively. <\/p>\n","protected":false},"excerpt":{"rendered":"
Below you’ll find tips we frequently suggest to optimize “slow” code. These tips are organized into an Easy list and an Advanced list, though everything here is pretty approachable. These lists are not complete, so if you find that the advice does not help, please contact us. Easy Options Use Compiler Optimizations Write Files Efficiently…<\/p>\n","protected":false},"author":8020,"featured_media":0,"parent":64953,"menu_order":3,"comment_status":"closed","ping_status":"closed","template":"","meta":[],"_links":{"self":[{"href":"https:\/\/www.bu.edu\/tech\/wp-json\/wp\/v2\/pages\/78109"}],"collection":[{"href":"https:\/\/www.bu.edu\/tech\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/www.bu.edu\/tech\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/www.bu.edu\/tech\/wp-json\/wp\/v2\/users\/8020"}],"replies":[{"embeddable":true,"href":"https:\/\/www.bu.edu\/tech\/wp-json\/wp\/v2\/comments?post=78109"}],"version-history":[{"count":50,"href":"https:\/\/www.bu.edu\/tech\/wp-json\/wp\/v2\/pages\/78109\/revisions"}],"predecessor-version":[{"id":140699,"href":"https:\/\/www.bu.edu\/tech\/wp-json\/wp\/v2\/pages\/78109\/revisions\/140699"}],"up":[{"embeddable":true,"href":"https:\/\/www.bu.edu\/tech\/wp-json\/wp\/v2\/pages\/64953"}],"wp:attachment":[{"href":"https:\/\/www.bu.edu\/tech\/wp-json\/wp\/v2\/media?parent=78109"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}
- You are writing temporary files, and once completed, these files can be deleted.<\/li>\n<\/ol>\n
- On a compute node, a reference to\u00a0
- Parallelize Your Code<\/a><\/li>\n<\/ul>\n
- Write Files Efficiently<\/a><\/li>\n