Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Blog plugin #463

Open
wants to merge 9 commits into
base: gh-pages
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 5 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 21 additions & 0 deletions docs/blog/authors.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
authors:
ethanyoung:
name: Ethan Young
description: Author
avatar: default.png
pauldiaz:
name: Paul Diaz
description: Author
avatar: default.png
timkaiser:
name: Tim Kaiser
description: Author
avatar: default.png
aco:
name: Advanced Computing Operations
description: Author
avatar: default.png
hpcas:
name: HPC Applications Support
description: Author
avatar: default.png
Binary file added docs/blog/default.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
2 changes: 2 additions & 0 deletions docs/blog/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
# Blog

Original file line number Diff line number Diff line change
@@ -1,13 +1,15 @@
---
title: Speeding up Python Code with Numba
postdate: December 01, 2020
layout: post
author: Ethan Young
description: Numba is a just in time (JIT) compiler for Python and NumPy code. From the official website, "Numba-compiled numerical algorithms in Python can approach the speeds of C or FORTRAN."
date: 2020-12-01
authors:
- ethanyoung
---

# Speeding up python Code with Numba

Numba is a just in time (JIT) compiler for Python and NumPy code. From their official website, "Numba translates Python functions to optimized machine code at runtime using the industry-standard LLVM compiler library. Numba-compiled numerical algorithms in Python can approach the speeds of C or FORTRAN."

<!-- more -->

```python
@jit(nopython=True)
def function_to_be_compiled():
Expand All @@ -16,3 +18,5 @@ def function_to_be_compiled():
```

Importantly, many functions require *no changes or refactoring* to gain this speedup. In this [getting-started guide](https://github.com/NREL/HPC/blob/master/languages/python/numba/numba_demo.ipynb), we build an example environment on Eagle, test the performance of a Numba-compiled function using the most common implementation of the `@jit` decorator, and discuss what sorts of functions will see performance improvements when compiled.

...
14 changes: 9 additions & 5 deletions docs/blog/2021-05-06-tf.md → docs/blog/posts/2021-05-06-tf.md
Original file line number Diff line number Diff line change
@@ -1,13 +1,15 @@
---
title: Faster Machine Learning with Custom Built TensorFlow on Eagle
postdate: May 6, 2021
layout: post
author: Paul Diaz
description: Customized python installations of TensorFlow specifically optimized for the Eagle hardware (CPUs & GPUs) are now available to all Eagle users.
date: 2021-05-06
authors:
- pauldiaz
---

# Faster Machine Learning with Custom Built TensorFlow on Eagle

TensorFlow is a widely used and powerful symbolic math library commonly used for a variety of machine learning techniques. TensorFlow has built in API support for regression, clustering, classification, hidden Markov models, neural networks, reinforcement learning, as well as, a variety of activation functions, loss function, and optimizers. TensorFlow has received growing adoption among scientists, researchers, and industry professionals for its broad applicability and flexibility.

<!-- more -->

TensorFlow versions obtained from `pip` or `conda` installs may not be optimized for the CPU and GPU architectures found on Eagle. To address this, pre-compiled versions which are optimized both for the CPU and GPU architectures have been created and offer computational benefits compared to other installation approaches. These versions can easily be installed from the `wheels` provided in `/nopt/nrel/apps/wheels/` which contains different TensorFlow versions.

Here is an example of how you can install an optimized version of TensorFlow to your environment.
Expand All @@ -18,3 +20,5 @@ These builds provide a significant advantage as illustrated below over the stand
![Benchmark image](../../assets/images/gpu_ai_benchmark.png)

A recent tutorial was given on this topic, for more information see the [recording](https://web.microsoftstream.com/video/af9b54ae-9158-4075-9f36-9aa2a4412ad0) or checkout the [tutorial materials](https://github.com/NREL/HPC/tree/master/workshops/Optimized_TF)

...
terry-cao marked this conversation as resolved.
Show resolved Hide resolved
Original file line number Diff line number Diff line change
@@ -1,11 +1,14 @@
---
title: Using srun to Launch Applications Under Slurm
postdate: June 18, 2021
layout: post
author: Tim Kaiser
description: Describes options to get the desired layout of tasks to nodes/cores using srun.
date: 2021-06-18
authors:
- timkaiser
---

# Using srun to Launch Applications Under Slurm

This article primarily discusses options for the srun command to enable good parallel execution.
terry-cao marked this conversation as resolved.
Show resolved Hide resolved

<!-- more -->

## Subjects covered
1. Basics
Expand Down Expand Up @@ -575,3 +578,5 @@ r102u35
r102u35
el3:stuff>
```

...
Original file line number Diff line number Diff line change
@@ -1,9 +1,17 @@
---
date: 2022-10-04
authors:
- hpcas
---

# Running Legacy Python 2 Code on Eagle

## What is Legacy Code?

One definition of "legacy" code or software is code was written in the past using currently outdated, obsolete, or otherwise deprecated, compilers, functions, methods, or methodology.

<!-- more -->

While [Python 2 was sunset on January 1, 2020](https://www.python.org/doc/sunset-python-2/) in favor of Python 3, there is still "legacy" Python 2 software that may need to be run on Eagle.
We always encourage Eagle users to upgrade their code to Python 3.x to continue receiving official updates, bug fixes, and security patches.
But we do understand that there will always be code that is not worth porting to Python 3.
Expand Down Expand Up @@ -74,3 +82,5 @@ Additionally, depending on the needs of your project, you can also reach out to
*Remember*, since Python 2 has been officially deprecated, more and more code is either being updated or rewritten entirely in Python 3 as time passes.
Additionally, the community-maintained Python 2 packages in the default conda channels will likely disappear at some point in the future.
Keeping your code modernized to the latest standards will help ensure both the longevity and reproducibility of your software and your results.

...
Original file line number Diff line number Diff line change
@@ -1,8 +1,15 @@
---
title: Workaround for Windows SSH "Corrupted MAC on input" Error
date: 2022-12-19
authors:
- aco
---

# Workaround for Windows SSH "Corrupted MAC on input" Error

Some people who use Windows 10/11 computers to ssh to Eagle from a Windows command prompt, powershell, or via Visual Studio Code's SSH extension might receive an error message about a "Corrupted MAC on input" or "message authentication code incorrect." This error is due to an outdated OpenSSL library included in Windows and a security-mandated change to ssh on Eagle. However, there is a functional workaround for this issue. (Note: If you are not experiencing the above error, you do not need and should not use the following workaround.)

<!-- more -->

For command-line and Powershell ssh users, adding `-m hmac-sha2-512` to your ssh command will resolve the issue. For example: `ssh -m hmac-sha2-512 <username>@eagle.hpc.nrel.gov`.

For VS Code SSH extension users, you will need to create an ssh config file on your local computer (~/.ssh/config), with a host entry for Eagle that specifies a new message authentication code:
Expand All @@ -13,3 +20,5 @@ Host eagle
```

The configuration file will also apply to command-line ssh in Windows. This [Visual Studio Blog post](https://code.visualstudio.com/blogs/2019/10/03/remote-ssh-tips-and-tricks) has further instructions on how to create the ssh configuration file for Windows and VS Code.

...
15 changes: 9 additions & 6 deletions mkdocs.yml
Original file line number Diff line number Diff line change
@@ -1,11 +1,7 @@
nav:
- Home: index.md
- Blog:
- blog/2020-12-01-numba.md
- blog/2021-05-06-tf.md
- blog/2021-06-18-srun.md
- blog/2022-10-04-python2to3.md
- blog/2022-12-19-windows_ssh.md
- Blog:
- index: blog/index.md
- Documentation:
- Documentation Home: Documentation/index.md
- Getting Started: Documentation/getting_started.md
Expand Down Expand Up @@ -188,3 +184,10 @@ extra_css:
site_name: NREL HPC
repo_url: https://github.com/NREL/HPC/tree/gh-pages
repo_name: NREL/HPC
plugins:
- blog:
authors_file: blog/authors.yml
enabled: true
blog_dir: blog
post_dir: blog/posts
post_readtime: false