Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Run i ifinal #19

Open
wants to merge 135 commits into
base: master
Choose a base branch
from
Open
Changes from 1 commit
Commits
Show all changes
135 commits
Select commit Hold shift + click to select a range
fa42829
2018 branch to run on 102x mc
lsoffi Sep 25, 2018
13e4889
cut by cut bool
swagata87 Oct 17, 2018
4b82aaf
cut-by-cut ele branches
swagata87 Oct 17, 2018
a252248
changes
swagata87 Oct 19, 2018
dea118f
update
swagata87 Dec 10, 2018
278bc29
100X to 94X name change
swagata87 Feb 28, 2019
4a806f2
Do not track *.pyc, *.root or temporary crab files
Aug 29, 2019
c801ab0
Update README.md
Aug 29, 2019
1cbf97c
Comment added
Aug 29, 2019
5dcd9f7
Pile-up for 2018
Aug 29, 2019
f64d32d
Clean-up of unneeded old commented code
Aug 29, 2019
3272736
Make it possible to add multiple HLT filters to measure
Aug 29, 2019
67f9bba
Update README.md
tomcornelis Sep 4, 2019
269f9e2
2018 files
swagata87 Sep 9, 2019
d850acf
Merge pull request #1 from tomcornelis/Nm1_proposedUpdates
swagata87 Oct 2, 2019
1b58ef3
first commit
lfinco Dec 4, 2019
3bde22e
trying to match L1 objects
lfinco Dec 4, 2019
2e8b722
working L1 matching for Leg1
lfinco Dec 5, 2019
41465c3
Attempt to have the L1 leg thresholds based on prescale
Dec 6, 2019
784560a
Merge pull request #1 from tomcornelis/lfinco
lfinco Dec 9, 2019
386ab20
modified version of submitting script
lfinco Dec 16, 2019
449397d
Subtracting lumis from lower thresholds + merging files with the same…
Dec 16, 2019
3b0ceab
Merge pull request #2 from tomcornelis/lfinco
lfinco Dec 16, 2019
c5d46e3
Skipping empty jsons
Dec 17, 2019
04cf172
Merge pull request #3 from tomcornelis/lfinco
lfinco Dec 17, 2019
bd4b94e
correction of the allProbes collection for the case with L1 matching
lfinco Jan 16, 2020
9f926d8
few small corrections and tests
lfinco Jan 17, 2020
3119dd0
Simplifying/generalization of crab submission script
Feb 18, 2020
bf02db6
README: recommended release to include V2 ID's
Feb 18, 2020
b81fb62
Storing more ID's in the trigger tree
Feb 18, 2020
77d822a
Test directory
Feb 18, 2020
a1caf80
Move crab directory one level up
Feb 18, 2020
5f06b00
Quick way to test all data/MC and year combinations
Feb 19, 2020
b52ec39
Run trigger by default
Feb 19, 2020
4f19787
implement el_relPfLepIso03 and passingDoubleHLTsafe
guitargeek Feb 19, 2020
b8acc18
Merge pull request #1 from guitargeek/VVV_patch
tomcornelis Feb 19, 2020
fd7160a
Adding check for single values in the test runs
Feb 20, 2020
ab6f418
Big clean-up/refactoring, all ID's now added to the trigger tree
Feb 20, 2020
b722d8b
Merge remote-tracking branch 'myfork/RunIIfinal' into RunIIfinal
Feb 20, 2020
0bb0766
Print out the error when test fails
Feb 20, 2020
d47a635
Reference files uploaded to git
Feb 20, 2020
be188eb
In the process of adding 2016/2017
Feb 20, 2020
afc9ac0
Updating DY samples
Feb 21, 2020
7fa4c3b
README updated
Feb 21, 2020
2f46c2c
L1 matching requirement for leg23 flag, added flags for HLTDoubleEle33
lfinco Feb 21, 2020
7b17490
Merge remote-tracking branch 'lfinco/lfinco' into RunIIfinal_lfincoMerge
Feb 21, 2020
f14dc6a
L1 matching clean-up
Feb 21, 2020
2a8c7b5
Apply L1 matching based on the "L1match" occuring in the flag name
Feb 26, 2020
eebab97
Small print statement
Feb 27, 2020
9124096
Adding photon variables
Feb 27, 2020
6b9870a
Updating test files
Feb 27, 2020
045921b
Merge pull request #2 from tomcornelis/RunIIfinal_lfincoMerge
tomcornelis Feb 27, 2020
7c2f2cd
Adding two more photon variables
Feb 27, 2020
5b682a3
Correct HLT settings for all years
Feb 28, 2020
fab75d0
Print statement
Feb 28, 2020
c878088
print statement
Feb 28, 2020
bf03941
Update reference files
Feb 28, 2020
837ff10
Merge pull request #3 from tomcornelis/RunIIfinal_lfincoMerge
tomcornelis Feb 28, 2020
790ae04
Improving crab submission
Feb 28, 2020
88ed42e
typo solved
Feb 28, 2020
8482262
Make sure they make separate crab directories for different years
Feb 28, 2020
1d98bbc
Allow to submit from slc7
Feb 28, 2020
1d0c267
Use wrapper to avoid crazy crab errors
Feb 28, 2020
3c42d67
Make it work on lxplus
Mar 2, 2020
b98c2ec
Needs to differentiate between era=2016 and is80X, because 2016 MiniA…
Mar 2, 2020
be26178
Adding variable which seem to be in there in some other branches
Mar 3, 2020
5083d29
Implementing the possibility to submit with different L1 thresholds
Mar 3, 2020
dcc0f6d
Little script to merge the output files
Mar 5, 2020
c404f12
Automatic check if it was already merged correctly
Mar 5, 2020
debf6b6
To mix the prescales in monte carlo as they are in data
Mar 6, 2020
50aed17
Update README
Mar 9, 2020
3e5f9fc
Small fixes
Mar 14, 2020
0606440
Implementation of lepton mva's
Mar 20, 2020
bbe7ab9
Switch of debug output
Mar 20, 2020
b77a6dc
Refactoring the probe modules
Mar 23, 2020
1ff90cd
Removing duplicate code, use writeValueMap() everywhere
Mar 25, 2020
f249a2c
Adding the new TOP leptonMva
May 8, 2020
0247ea2
Update testing
May 8, 2020
1a17dbb
Updating README
May 8, 2020
2b85a53
Update README
May 8, 2020
b277d2b
Update README
May 8, 2020
f37c63e
Make sure there is a log directory
May 8, 2020
eab98da
Make sure ElectronVariableHelper.h can work on AOD
May 12, 2020
aa13dc9
Fix AOD running
May 12, 2020
b6458a6
Save mc_probe variables only for MC
May 12, 2020
43b5ecb
Do not ignore exceptions
May 12, 2020
d054740
Fix/improve compareTrees script
May 12, 2020
f3754d6
New test run finished
May 12, 2020
fa23503
update .gitignore
May 12, 2020
18be326
To quickly commit updated rootfiles and logs from test
May 12, 2020
4d12423
Putting GT's in TnPTreeProducer_cfg
May 14, 2020
fb1ba8e
New test run finished
May 14, 2020
2fb6e1a
Fixing variable input to lepton mva's
May 14, 2020
b530134
New test run finished
May 14, 2020
9636b75
Quick change in what to run for tests
May 15, 2020
87f7f2a
Quick change in what to run for tests
May 15, 2020
bcc1e0a
Typo and update README
May 15, 2020
8bb7fdf
Add DY NLO and reduce the units per job to avoid walltime limits
May 29, 2020
f3d79eb
Remove tabs from python code
Jun 2, 2020
571fabd
Avoid to run the try catch block for every event
Jun 3, 2020
73108b3
Adding location of tuples to README
Jun 5, 2020
35a7cc6
Use own test files
Jun 5, 2020
57a2bac
Adding a few more variables
Jun 8, 2020
2f8d7b3
Working towards a CMSSW-release indepedent branch
Jun 8, 2020
2524b1a
New test run finished
Jun 8, 2020
60787da
Combining branch RunIIfinal_UL into RunIIfinal
Jun 8, 2020
7c4a054
New test run finished
Jun 8, 2020
52334fa
Needed for compilation in CMSSW_10_6_X
Jun 9, 2020
d41c3ca
Small fix to find xml for leptonMvaTOP in UL
Jun 9, 2020
91a15ec
Also having the UL test files stored at eos
Jun 9, 2020
85b0783
runTests.py does UL tests when in CMSSW_10_6
Jun 9, 2020
8e972bf
Update README: RunIIfinal branch works now in both CMSSW_10_2 and CMS…
Jun 9, 2020
0932585
New test run finished (UL tests added)
Jun 9, 2020
eedea55
Get rereco and UL crab submission in same file
Jun 9, 2020
83ff108
Make it CMSSW_11_X compatible
Jun 9, 2020
6cda14b
Merge branch 'RunIIfinal' of https://github.com/tomcornelis/EgammaAna…
Jun 9, 2020
2069ed6
Update README
Jun 9, 2020
68690fd
Fix isReleaseAbove function
Jun 9, 2020
b7f4de3
Update GT for 2018
Jun 9, 2020
3f34119
We still had this old 80X-style 2016 data here, corrected now
Jun 10, 2020
bbe62cb
Typo in README
Jun 11, 2020
8382a15
Update README: new tuples location
Jun 15, 2020
e18c34b
Adding extension for 2018 DY NLO sample
Jun 20, 2020
901502b
Add matched conversion
Jul 8, 2020
cf79a2a
Don't fail when temp files already exist
Jul 16, 2020
de3b6c1
Updated general info for UL2016
ram1123 Feb 10, 2021
74d6b35
Merge pull request #5 from ram1123/dev/RunIIfinal_UL2016
swagata87 Jun 27, 2022
a378e17
change made
Aug 1, 2022
ac31b25
change made
Aug 1, 2022
c9acfdf
Update README.md
yiruole Aug 1, 2022
d23131f
Update TnPTreeProducer_cfg.py
yiruole Aug 2, 2022
df9fd9e
change made
Mar 20, 2023
e1b8d5f
change made
Mar 20, 2023
0e0da1f
change made
Apr 3, 2023
ac7d575
change made
Apr 3, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
few small corrections and tests
  • Loading branch information
lfinco committed Jan 17, 2020
commit 9f926d8ed626d90004b7f677354bbb3e155e399f
6 changes: 0 additions & 6 deletions python/TnPTreeProducer_cfg.py
Original file line number Diff line number Diff line change
@@ -87,12 +87,6 @@
"Threshold for L1 matched objects"
)

#varOptions.register(
# "leg2Threshold", 0,
# VarParsing.multiplicity.singleton,
# VarParsing.varType.int,
# "Threshold for L1 seed leg2"
# )

varOptions.parseArguments()

6 changes: 5 additions & 1 deletion python/egmTreesSetup_cff.py
Original file line number Diff line number Diff line change
@@ -66,13 +66,16 @@ def setTagsProbes(process, options):
process.probeElePassHLT.inputs = cms.InputTag("probeEle")
process.probeElePassHLT.isAND = cms.bool(False)

#not needed
#probably not needed
#process.probeElePassHLTL1matched = process.tagEle.clone()
#process.probeElePassHLTL1matched.inputs = cms.InputTag("probeEleL1matched")
#process.probeElePassHLTL1matched.isAND = cms.bool(False)

for flag, filterNames in options['HLTFILTERSTOMEASURE'].iteritems():
setattr(process, flag, process.probeElePassHLT.clone(filterNames=filterNames))
#probably not needed:
#setattr(process, flag + "L1matched", process.probeElePassHLTL1matched.clone(filterNames=filterNames))

#setattr(process, flag, process.probeElePassHLTL1matched.clone(filterNames=filterNames)) #if I add this, I guess I'm introducing on all probes (not only the ones I called probeElepassHLTL1matched) the L1 matching requirement (but only in the numerator, so the efficiency goes down)


@@ -348,6 +351,7 @@ def setSequences(process, options):
process.hlt_sequence = cms.Sequence()
for flag in options['HLTFILTERSTOMEASURE']:
process.hlt_sequence += getattr(process, flag)
#process.hlt_sequence += getattr(process, flag + "L1matched")

if options['isMC'] :
process.tag_sequence += process.genEle + process.genTagEle
91 changes: 91 additions & 0 deletions scripts/crab/submitWithVariableThresholds_MC.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,91 @@
#!/bin/env python

### Version of submission ###
submitVersion = "test"



defaultArgs = ['isMC=True','doEleID=False','doPhoID=False','doTrigger=True', 'GT=102X_upgrade2018_realistic_v12']
#mainOutputDir = '/store/user/tomc/tnpTuples/%s' % submitVersion # Change to your own
mainOutputDir = '/store/group/phys_egamma/lfinco/2018/L1Matching/%s' % submitVersion
from WMCore.Configuration import Configuration
from CRABClient.UserUtilities import config
config = config()

config.General.requestName = ''
config.General.transferLogs = False
config.JobType.pluginName = 'Analysis'

config.JobType.psetName = '../../python/TnPTreeProducer_cfg.py'
config.JobType.sendExternalFolder = True

config.Data.inputDataset = ''
config.Data.inputDBS = 'global'
config.Data.publication = False
config.Data.allowNonValidInputDataset = True
config.Site.storageSite = 'T2_CH_CERN'#'T2_BE_IIHE'



if __name__ == '__main__':

import urllib, os, glob
def download(url, destination):
try: os.makedirs(destination)
except: pass
urllib.urlretrieve(url, os.path.join(destination, url.split('/')[-1]))

def getSeedsForDoubleEle(year): # note: only getting DoubleEle seeds here
prescalePage = 'https://tomc.web.cern.ch/tomc/triggerPrescales/%s/' % year
hltTrigger = 'HLT_Ele23_Ele12_CaloIdL_TrackIdL_IsoVL'
dirToStore = os.path.join('prescaleInformation', year, hltTrigger)
download(prescalePage + hltTrigger + '.php', dirToStore)
with open(os.path.join(dirToStore, hltTrigger + '.php')) as f:
for line in f:
if 'prescale1' in line and 'L1_DoubleEG' in line:
download(prescalePage + line.split('>')[0].split('=')[-1], dirToStore)

for json in glob.glob(os.path.join(dirToStore, '*.json')):
leg1 = int(json.split('L1_DoubleEG_')[-1].split('_')[0].replace('LooseIso', ''))
leg2 = int(json.split('L1_DoubleEG_')[-1].split('_')[1].replace('LooseIso', ''))
yield leg1, leg2, json


from CRABAPI.RawCommand import crabCommand
from CRABClient.ClientExceptions import ClientException
from httplib import HTTPException

# We want to put all the CRAB project directories from the tasks we submit here into one common directory.
# That's why we need to set this parameter (here or above in the configuration file, it does not matter, we will not overwrite it).
config.General.workArea = 'crab_%s' % submitVersion

def submit(config, sample, leg1threshold, leg2threshold):
config.General.requestName = sample.split('/')[-3] + '_%s_%s' % (leg1threshold, leg2threshold)
config.Data.inputDataset = sample#'/EGamma/Run2018A-PromptReco-v1/MINIAOD'
config.Data.outLFNDirBase = '%s/%s/' % (mainOutputDir,'mc')
config.Data.splitting = 'FileBased'
#config.Data.lumiMask = json
config.Data.unitsPerJob = 5
config.JobType.pyCfgParams = defaultArgs + ['L1Threshold=%s' % leg1threshold]
#config.JobType.pyCfgParams = defaultArgs + ['leg1Threshold=%s' % 22, 'leg2Threshold=%s' % 0]

try:
crabCommand('submit', config = config)
except HTTPException as hte:
print "Failed submitting task: %s" % (hte.headers)
except ClientException as cle:
print "Failed submitting task: %s" % (cle)

#for leg1, leg2, json in getSeedsForDoubleEle('2018'):
# Crab fails on this on second iteration, of course with only a very cryptic error message
# Not sure how to workaround this
#submit(config, '/EGamma/Run2018A-PromptReco-v1/MINIAOD', leg1, leg2, json)
#submit(config, '/EGamma/Run2018A-17Sep2018/MINIAOD', leg1, leg2, json)
submit(config, '/DYJetsToLL_M-50_TuneCP5_13TeV-madgraphMLM-pythia8/RunIIAutumn18MiniAOD-102X_upgrade2018_realistic_v15-v1/MINIAODSIM', 22, 0)
#submit(config, '/DYToEE_M-50_NNPDF31_TuneCP5_13TeV-powheg-pythia8/RunIIAutumn18MiniAOD-102X_upgrade2018_realistic_v15-v1/MINIAODSIM', 22, 0)
#submit(config, '/DYJetsToLL_M-50_TuneCP5_13TeV-madgraphMLM-pythia8/RunIIAutumn18MiniAOD-102X_upgrade2018_realistic_v15-v1/MINIAODSIM', 25, 0)
#submit(config, '/DYToEE_M-50_NNPDF31_TuneCP5_13TeV-powheg-pythia8/RunIIAutumn18MiniAOD-102X_upgrade2018_realistic_v15-v1/MINIAODSIM', 25, 0)