can’t create hard link

[llm98m]

Describe the bug

Hello,
I am seeking assistance with a problem I've encountered while utilizing a pipeline for analysis with a fastaq file. Once the file has been trimmed, the next step involves creating a hard link. Unfortunately, my server does not currently support the creation of hard links. Is there a method to circumvent this step, or could you guide me on how to alter the script so that it copies the file instead of creating a hard link? Enclosed is the specific error message I received when running the pipeline. I am eagerly awaiting your response.

OS/Platform

• OS/Platform:
  LSB Version: :core-4.1-amd64:core-4.1-noarch:cxx-4.1-amd64:cxx-4.1-noarch:desktop-4.1-amd64:desktop-4.1-noarch:languages-4.1-amd64:languages-4.1-noarch:printing-4.1-amd64:printing-4.1-noarch
  Distributor ID: CentOS
  Description: CentOS Linux release 7.9.2009 (Core)
  Release: 7.9.2009
  Codename: Core
• Conda version: conda 23.1.0
• Pipeline version: [e.g. v1.8.0]
• Caper version: caper --version 2.3.2

Caper configuration file

backend=local
local-loc-dir=/storage/public/home/lmli/software/Encode/ATAC
cromwell=/root/.caper/cromwell_jar/cromwell-82.jar
womtool=/root/.caper/womtool_jar/womtool-82.jar

Input JSON file

{
"atac.title" : "test (paired end)",
"atac.description" : "This is a template input JSON for paired ended sample.",

"atac.pipeline_type" : "FSH-OPU-16C-ATAC-34",
"atac.align_only" : false,
"atac.true_rep_only" : false,

"atac.genome_tsv" : "/storage/public/home/lmli/software/Encode/ATAC/genomes/bostau9/bostau9.tsv",

"atac.paired_end" : true,
"atac.fastqs_rep1_R1" : [ "/storage/public/home/lmli/Analysis/fast_slow_embro/ATAC/02.data/FSH-OPU-16C-ATAC-3/FSH-OPU-16C-ATAC-3_S1_L003_R1_001.fastq.gz"],
"atac.fastqs_rep1_R2" : [ "/storage/public/home/lmli/Analysis/fast_slow_embro/ATAC/02.data/FSH-OPU-16C-ATAC-3/FSH-OPU-16C-ATAC-3_S1_L003_R2_001.fastq.gz"],
"atac.fastqs_rep2_R1" : [ "/storage/public/home/lmli/Analysis/fast_slow_embro/ATAC/02.data/FSH-OPU-16C-ATAC-4/FSH-OPU-16C-ATAC-4_S47_L002_R1_001.fastq.gz"],
"atac.fastqs_rep2_R2" : [ "/storage/public/home/lmli/Analysis/fast_slow_embro/ATAC/02.data/FSH-OPU-16C-ATAC-4/FSH-OPU-16C-ATAC-4_S47_L002_R2_001.fastq.gz"],

"atac.auto_detect_adapter" : true,

"atac.multimapping" : 4

}

Troubleshooting result

==== NAME=atac.align_mito, STATUS=Failed, PARENT=
SHARD_IDX=0, RC=None, JOB_ID=7438
START=2024-11-09T07:37:17.498Z, END=2024-11-09T15:39:33.298Z
STDOUT=/storage/public/home/lmli/Analysis/fast_slow_embro/ATAC/10.encode/atac/fd70e000-bdda-461d-b757-03ac6cc38d28/call-align_mito/shard-0/execution/stdout
STDERR=/storage/public/home/lmli/Analysis/fast_slow_embro/ATAC/10.encode/atac/fd70e000-bdda-461d-b757-03ac6cc38d28/call-align_mito/shard-0/execution/stderr
STDERR_CONTENTS=

STDERR_BACKGROUND_CONTENTS=
ln: failed to create hard link '/cromwell-executions/atac/fd70e000-bdda-461d-b757-03ac6cc38d28/call-align_mito/shard-0/execution/glob-bc1afa799665df5c7d6afd70d2ae2cb4/FSH-OPU-16C-ATAC-3_S1_L003_R1_001.trim.srt.no_MT.samstats.qc' => '/cromwell-executions/atac/fd70e000-bdda-461d-b757-03ac6cc38d28/call-align_mito/shard-0/execution/non_mito/FSH-OPU-16C-ATAC-3_S1_L003_R1_001.trim.srt.no_MT.samstats.qc': Operation not permitted
ln: failed to create hard link '/cromwell-executions/atac/fd70e000-bdda-461d-b757-03ac6cc38d28/call-align_mito/shard-0/execution/glob-773fb92850749a2b4a829cf3c8c4de27/FSH-OPU-16C-ATAC-3_S1_L003_R1_001.trim.read_length.txt' => '/cromwell-executions/atac/fd70e000-bdda-461d-b757-03ac6cc38d28/call-align_mito/shard-0/execution/FSH-OPU-16C-ATAC-3_S1_L003_R1_001.trim.read_length.txt': Operation not permitted
ln: failed to create hard link '/cromwell-executions/atac/fd70e000-bdda-461d-b757-03ac6cc38d28/call-align_mito/shard-0/execution/glob-3bcbe4e7489c90f75e0523ac6f3a9385/FSH-OPU-16C-ATAC-3_S1_L003_R1_001.trim.srt.bam' => '/cromwell-executions/atac/fd70e000-bdda-461d-b757-03ac6cc38d28/call-align_mito/shard-0/execution/FSH-OPU-16C-ATAC-3_S1_L003_R1_001.trim.srt.bam': Operation not permitted
ln: failed to create hard link '/cromwell-executions/atac/fd70e000-bdda-461d-b757-03ac6cc38d28/call-align_mito/shard-0/execution/glob-6efbc60cb1e0959bab4e467327a9416c/FSH-OPU-16C-ATAC-3_S1_L003_R1_001.trim.srt.bam.bai' => '/cromwell-executions/atac/fd70e000-bdda-461d-b757-03ac6cc38d28/call-align_mito/shard-0/execution/FSH-OPU-16C-ATAC-3_S1_L003_R1_001.trim.srt.bam.bai': Operation not permitted
ln: failed to create hard link '/cromwell-executions/atac/fd70e000-bdda-461d-b757-03ac6cc38d28/call-align_mito/shard-0/execution/glob-7b38d9959cf6f3deb83ac2bd156d8317/FSH-OPU-16C-ATAC-3_S1_L003_R1_001.trim.srt.samstats.qc' => '/cromwell-executions/atac/fd70e000-bdda-461d-b757-03ac6cc38d28/call-align_mito/shard-0/execution/FSH-OPU-16C-ATAC-3_S1_L003_R1_001.trim.srt.samstats.qc': Operation not permitted

==== NAME=atac.align_mito, STATUS=Failed, PARENT=
SHARD_IDX=1, RC=None, JOB_ID=8941
START=2024-11-09T07:37:11.963Z, END=2024-11-09T23:15:35.005Z
STDOUT=/storage/public/home/lmli/Analysis/fast_slow_embro/ATAC/10.encode/atac/fd70e000-bdda-461d-b757-03ac6cc38d28/call-align_mito/shard-1/execution/stdout
STDERR=/storage/public/home/lmli/Analysis/fast_slow_embro/ATAC/10.encode/atac/fd70e000-bdda-461d-b757-03ac6cc38d28/call-align_mito/shard-1/execution/stderr
STDERR_CONTENTS=

STDERR_BACKGROUND_CONTENTS=
ln: failed to create hard link '/cromwell-executions/atac/fd70e000-bdda-461d-b757-03ac6cc38d28/call-align_mito/shard-1/execution/glob-bc1afa799665df5c7d6afd70d2ae2cb4/FSH-OPU-16C-ATAC-4_S47_L002_R1_001.trim.srt.no_MT.samstats.qc' => '/cromwell-executions/atac/fd70e000-bdda-461d-b757-03ac6cc38d28/call-align_mito/shard-1/execution/non_mito/FSH-OPU-16C-ATAC-4_S47_L002_R1_001.trim.srt.no_MT.samstats.qc': Operation not permitted
ln: failed to create hard link '/cromwell-executions/atac/fd70e000-bdda-461d-b757-03ac6cc38d28/call-align_mito/shard-1/execution/glob-773fb92850749a2b4a829cf3c8c4de27/FSH-OPU-16C-ATAC-4_S47_L002_R1_001.trim.read_length.txt' => '/cromwell-executions/atac/fd70e000-bdda-461d-b757-03ac6cc38d28/call-align_mito/shard-1/execution/FSH-OPU-16C-ATAC-4_S47_L002_R1_001.trim.read_length.txt': Operation not permitted
ln: failed to create hard link '/cromwell-executions/atac/fd70e000-bdda-461d-b757-03ac6cc38d28/call-align_mito/shard-1/execution/glob-3bcbe4e7489c90f75e0523ac6f3a9385/FSH-OPU-16C-ATAC-4_S47_L002_R1_001.trim.srt.bam' => '/cromwell-executions/atac/fd70e000-bdda-461d-b757-03ac6cc38d28/call-align_mito/shard-1/execution/FSH-OPU-16C-ATAC-4_S47_L002_R1_001.trim.srt.bam': Operation not permitted
ln: failed to create hard link '/cromwell-executions/atac/fd70e000-bdda-461d-b757-03ac6cc38d28/call-align_mito/shard-1/execution/glob-6efbc60cb1e0959bab4e467327a9416c/FSH-OPU-16C-ATAC-4_S47_L002_R1_001.trim.srt.bam.bai' => '/cromwell-executions/atac/fd70e000-bdda-461d-b757-03ac6cc38d28/call-align_mito/shard-1/execution/FSH-OPU-16C-ATAC-4_S47_L002_R1_001.trim.srt.bam.bai': Operation not permitted
ln: failed to create hard link '/cromwell-executions/atac/fd70e000-bdda-461d-b757-03ac6cc38d28/call-align_mito/shard-1/execution/glob-7b38d9959cf6f3deb83ac2bd156d8317/FSH-OPU-16C-ATAC-4_S47_L002_R1_001.trim.srt.samstats.qc' => '/cromwell-executions/atac/fd70e000-bdda-461d-b757-03ac6cc38d28/call-align_mito/shard-1/execution/FSH-OPU-16C-ATAC-4_S47_L002_R1_001.trim.srt.samstats.qc': Operation not permitted

==== NAME=atac.align, STATUS=Failed, PARENT=
SHARD_IDX=0, RC=None, JOB_ID=8455
START=2024-11-09T07:37:11.961Z, END=2024-11-10T01:01:13.459Z
STDOUT=/storage/public/home/lmli/Analysis/fast_slow_embro/ATAC/10.encode/atac/fd70e000-bdda-461d-b757-03ac6cc38d28/call-align/shard-0/execution/stdout
STDERR=/storage/public/home/lmli/Analysis/fast_slow_embro/ATAC/10.encode/atac/fd70e000-bdda-461d-b757-03ac6cc38d28/call-align/shard-0/execution/stderr
STDERR_CONTENTS=

STDERR_BACKGROUND_CONTENTS=
ln: failed to create hard link '/cromwell-executions/atac/fd70e000-bdda-461d-b757-03ac6cc38d28/call-align/shard-0/execution/glob-bc1afa799665df5c7d6afd70d2ae2cb4/FSH-OPU-16C-ATAC-3_S1_L003_R1_001.trim.srt.no_MT.samstats.qc' => '/cromwell-executions/atac/fd70e000-bdda-461d-b757-03ac6cc38d28/call-align/shard-0/execution/non_mito/FSH-OPU-16C-ATAC-3_S1_L003_R1_001.trim.srt.no_MT.samstats.qc': Operation not permitted
ln: failed to create hard link '/cromwell-executions/atac/fd70e000-bdda-461d-b757-03ac6cc38d28/call-align/shard-0/execution/glob-773fb92850749a2b4a829cf3c8c4de27/FSH-OPU-16C-ATAC-3_S1_L003_R1_001.trim.read_length.txt' => '/cromwell-executions/atac/fd70e000-bdda-461d-b757-03ac6cc38d28/call-align/shard-0/execution/FSH-OPU-16C-ATAC-3_S1_L003_R1_001.trim.read_length.txt': Operation not permitted
ln: failed to create hard link '/cromwell-executions/atac/fd70e000-bdda-461d-b757-03ac6cc38d28/call-align/shard-0/execution/glob-3bcbe4e7489c90f75e0523ac6f3a9385/FSH-OPU-16C-ATAC-3_S1_L003_R1_001.trim.srt.bam' => '/cromwell-executions/atac/fd70e000-bdda-461d-b757-03ac6cc38d28/call-align/shard-0/execution/FSH-OPU-16C-ATAC-3_S1_L003_R1_001.trim.srt.bam': Operation not permitted
ln: failed to create hard link '/cromwell-executions/atac/fd70e000-bdda-461d-b757-03ac6cc38d28/call-align/shard-0/execution/glob-6efbc60cb1e0959bab4e467327a9416c/FSH-OPU-16C-ATAC-3_S1_L003_R1_001.trim.srt.bam.bai' => '/cromwell-executions/atac/fd70e000-bdda-461d-b757-03ac6cc38d28/call-align/shard-0/execution/FSH-OPU-16C-ATAC-3_S1_L003_R1_001.trim.srt.bam.bai': Operation not permitted
ln: failed to create hard link '/cromwell-executions/atac/fd70e000-bdda-461d-b757-03ac6cc38d28/call-align/shard-0/execution/glob-7b38d9959cf6f3deb83ac2bd156d8317/FSH-OPU-16C-ATAC-3_S1_L003_R1_001.trim.srt.samstats.qc' => '/cromwell-executions/atac/fd70e000-bdda-461d-b757-03ac6cc38d28/call-align/shard-0/execution/FSH-OPU-16C-ATAC-3_S1_L003_R1_001.trim.srt.samstats.qc': Operation not permitted

==== NAME=atac.align, STATUS=Failed, PARENT=
SHARD_IDX=1, RC=None, JOB_ID=9115
START=2024-11-09T07:37:11.959Z, END=2024-11-10T02:49:55.187Z
STDOUT=/storage/public/home/lmli/Analysis/fast_slow_embro/ATAC/10.encode/atac/fd70e000-bdda-461d-b757-03ac6cc38d28/call-align/shard-1/execution/stdout
STDERR=/storage/public/home/lmli/Analysis/fast_slow_embro/ATAC/10.encode/atac/fd70e000-bdda-461d-b757-03ac6cc38d28/call-align/shard-1/execution/stderr
STDERR_CONTENTS=

STDERR_BACKGROUND_CONTENTS=
ln: failed to create hard link '/cromwell-executions/atac/fd70e000-bdda-461d-b757-03ac6cc38d28/call-align/shard-1/execution/glob-bc1afa799665df5c7d6afd70d2ae2cb4/FSH-OPU-16C-ATAC-4_S47_L002_R1_001.trim.srt.no_MT.samstats.qc' => '/cromwell-executions/atac/fd70e000-bdda-461d-b757-03ac6cc38d28/call-align/shard-1/execution/non_mito/FSH-OPU-16C-ATAC-4_S47_L002_R1_001.trim.srt.no_MT.samstats.qc': Operation not permitted
ln: failed to create hard link '/cromwell-executions/atac/fd70e000-bdda-461d-b757-03ac6cc38d28/call-align/shard-1/execution/glob-773fb92850749a2b4a829cf3c8c4de27/FSH-OPU-16C-ATAC-4_S47_L002_R1_001.trim.read_length.txt' => '/cromwell-executions/atac/fd70e000-bdda-461d-b757-03ac6cc38d28/call-align/shard-1/execution/FSH-OPU-16C-ATAC-4_S47_L002_R1_001.trim.read_length.txt': Operation not permitted
ln: failed to create hard link '/cromwell-executions/atac/fd70e000-bdda-461d-b757-03ac6cc38d28/call-align/shard-1/execution/glob-3bcbe4e7489c90f75e0523ac6f3a9385/FSH-OPU-16C-ATAC-4_S47_L002_R1_001.trim.srt.bam' => '/cromwell-executions/atac/fd70e000-bdda-461d-b757-03ac6cc38d28/call-align/shard-1/execution/FSH-OPU-16C-ATAC-4_S47_L002_R1_001.trim.srt.bam': Operation not permitted
ln: failed to create hard link '/cromwell-executions/atac/fd70e000-bdda-461d-b757-03ac6cc38d28/call-align/shard-1/execution/glob-6efbc60cb1e0959bab4e467327a9416c/FSH-OPU-16C-ATAC-4_S47_L002_R1_001.trim.srt.bam.bai' => '/cromwell-executions/atac/fd70e000-bdda-461d-b757-03ac6cc38d28/call-align/shard-1/execution/FSH-OPU-16C-ATAC-4_S47_L002_R1_001.trim.srt.bam.bai': Operation not permitted
ln: failed to create hard link '/cromwell-executions/atac/fd70e000-bdda-461d-b757-03ac6cc38d28/call-align/shard-1/execution/glob-7b38d9959cf6f3deb83ac2bd156d8317/FSH-OPU-16C-ATAC-4_S47_L002_R1_001.trim.srt.samstats.qc' => '/cromwell-executions/atac/fd70e000-bdda-461d-b757-03ac6cc38d28/call-align/shard-1/execution/FSH-OPU-16C-ATAC-4_S47_L002_R1_001.trim.srt.samstats.qc': Operation not permitted