Tools for Manipulating NEF [NMR Exchange Format] Files and Foreign File Access
NEF-Pipelines is a set of command line (currently... there maybe a gui later!) tools for maniulating [NEF] or NMR Exchange Format files which can be used to move NMR meta data [peaks, shifts, sequences etc] between NMR data processing programs.The programs provide basic tools for manipulating nef files:
Please note NEF-Pipelines is beta software and is quite new so if you do find problems put in an issue on the issue tracker. Even better if you can help improve the software do get in touch, there is much to do!
- molecular chains: listing, renaming and cloning molecular chains
- save-frames [tables]: deleting, inserting, listing and pretty printing (tabulate)
- headers creation / updating NEF headers with correct UUIDs and history
- streaming: NEF files into a pipeline
- testing: self testing of NEF pipelines
It also provides tools for importing and exporing non NEF files from the following programs [transcoding / translators]
- fasta
- sequence [read and write]
- mars
- shifts [write]
- nmrpipe
- sequences [read]
- shifts [read]
- peaks [read]
- nmrview / nmrfx
- peaks [read and write]
- sequence [read]
- shifts [read]
- pales
- rdc restraints [write]
- rdc restraints template [write]
- pdbx
- sequence [read]
- xplor
- sequence [read]
- dihedral restraints [read]
- distance restraints 'single atom selections [read]
if you have ccpn assign its simple to install NEF-Pipelines using a supplied macro install_nef_pipelines.sh
# if assign is on your path type
install_nef_pipelines
# else if this fails and ccpnmr/3.2.1/bin/assign is where the ccpn assign program is on disk run:
ccpnmr/3.2.1/bin/install_nef_pipelinesNEF-Pipelines can also be installed using pip and pipx
Firstly install pipx if you need to, using the commands for your OS ...
python3 -m pip install --user pipx
python3 -m pipx ensurepath- make sure you have python3 installed
- check if you have pipx installed by typing
pipx --versionin a command prompt - if you don't have pipx installed, install it by typing
python3 -m pip install --user pipx
python3 -m pipx ensurepath- close your terminal window and open a new one an type
follow the instruction for linux distro of choice inside WSL2...
sudo apt install pipx python3-venvsudo dnf install pipxsudo zypper install python3-pipxpipx install nef-pipelinesafter the installation has completed you can type
nefand should see output that starts
Usage: nef [OPTIONS] COMMAND [ARGS]... ✔ ╱ took 14s ╱ nef_pipelines ╱ at 23:29:36
Options:
...
pipx upgrade nef-pipelinesAll commands also have a
help options triggered using --help or when no input is detected. So for example the nef command on its own produces
Usage: nef [OPTIONS] COMMAND [ARGS]...
Options:
--install-completion [bash|zsh|fish|powershell|pwsh]
Install completion for the specified shell.
--show-completion [bash|zsh|fish|powershell|pwsh]
Show completion for the specified shell, to
copy it or customize the installation.
--help Show this message and exit.
Commands:
chains - carry out operations on chains
fasta - read and write fasta sequences
frames - carry out operations on frames in nef frames
header - add a header to the stream
mars - export mars [shifts and sequences]
nmrpipe - read nmrpipe [peaks shifts & sequencess]
nmrview - read and write nmrview [peaks, sequences & shifts]
pales - read and write pales/dc [rdcs]
pdb - read pdb [sequences]
stream - stream a nef file
test - run the test suite
Individual nef pipeline componenets can be used standalone by using the --in or --pipe parameter which will either
read a foreign file for translators or read a NEF file for commands that manipulate NEF files. So for example to lists
the frames in NEF frame you can type
nef frames list --in test_data/pales_test_1.nefthis will produce
entry pales_test
nef_molecular_system nef_rdc_restraint_list_test_1which shows the entry name followed by the frames in the file test_data/pales_test_1.nef. However, the real
power in NEF-Pipelines comes from combining pipeline commands together or with command line tools. For example
nef header \
| nef fasta import sequence tailin1.fasta \
| src/nef nmrview import shifts tailin1.out \
| src/nef nmrview import peaks tailin1.out \
> tailin1.nefwill create a valid NEF header followed by the tailin1 sequence as a molecular system and then import shifts and peaks
for talin1 before writing a new NEF file tailin1.nef
The commands provided by NEF-Pipelines are hierarchical in nature. All commands are call by the NEF command but there
are sub commands so for example to import a sequence froma pdb file tou would type nef psb import sequence followed
by the name of the PDB file and relevant options. This can look lomng winded bu NEF-Pipelines support command completion
so typing a double tab will list all availabe sub commands.
For example
nef<tab><tab>shows
chains fasta frames header mars nmrpipe nmrview pales pdb stream test
and
nef nmrview import<tab<tab>shows
peaks sequence shifts
The origional NEF paper is not a good description of the format more an outline of ideals and needs. The CCPN website has a good guide to nef and development takes place on github via the NEF consortium repository which includes an annotated NEF example and a dictionary defining the format. NEF is a STAR format which means it has similarities to NMRStar files used by the BMRB and PDBx / mmCIF used by the RCSB / PDB because its uses the same underling format. However, its is not directly interchangeable with these file formats as it has a different syntax and intention / underlying use easy and accurate NMR data interchange] as opposed to archiving for example.