Skip to content
This repository has been archived by the owner on Apr 11, 2019. It is now read-only.

Hannah Weekly Updates

Jump to bottom
hdean83 edited this page Mar 31, 2014 · 14 revisions

Week 1
Week 2
Week 3
Week 4
Week 5
Week 6
Week 7

# Week 1 I met with Derrick, went over data structures and the larger systems integration test, and set up an office space at IOOS as well as a connection to the shared drive. I worked independently on familiarizing myself with the Wakari environment and the workflow of the office. I gathered CMSP materials and literature to discuss a possible scenario involving oil spill/energy siting in order to flesh out a theme. I completed 30% of an online Python tutorial to familiarize myself with the language used to call and examine data registries. # Week 2 I continued to review the full sweet of systems integration documentation and worked on creating a notebook within the wakari environment. I successfully created a sample notebook using climate data and a sample csv file in order to familiarize myself with the notebook environment. I summarized my past work on CMSP within the Wiki environment for the systems integration test and linked to documentation that could inform the team on ways to frame the scenario in terms of data layers. I familiarized myself with the OSWLibrary and the language required for specific types of commands used for registries and the services that these registries provide (Web Mapping Services, etc.). Due to a snowstorm, I was unable to meet with Derrick. # Week 3 I drafted a baseline theme notebook that runs through a series of elements for each data registry and developed the language needed to test a web mapped services. This will provide a ground level means and processes for testing particular data layers required during a test scenario. I presented the notebook during my weekly meeting with the IOOS office, and got valuable feedback on how to structure the notebook and next steps in terms of how to handle the output of each notebook so that the information is presentable to a wider audience. During the meeting, we also clarified next steps for finalizing a scenario and creating corresponding notebooks and outputs. It was decided that rather than create standalone notebooks, it would be valuable to create bundles with notebooks and output files so that these could be gathered into a report at the completion of the project.

I made some progress on the master notebook, and just wanted to share that. I'm basically working through gathering different types of portals from the OWSLib and going through what can be looked at for each type, keeping a list at the top that is hashed, and coming up with some simple loops that will feed out the results. Here's the notebook so far: Testing Catalog Interfaces.

# Week 4

This week I continued to work on the IPython Baseline notebook, including calling of services from particular registries and determining the best way to develop metrics for testing registries in the context of the particular Themes outlined for the Project.

I also worked on articulating the third theme focused on marine planning/energy siting. To that end, I developed a visualization approach to mapping data sets to regulations/laws/policy documents that aims to tie the need and use of particular data sets to specific directives at the legislative and executive levels. Utilizing Lucidcharts, I put together a few sample diagrams that aim to draw a clearer line between policy and data. These visuals would provide a starting point for a set of notebooks aimed at testing registries and determining if the planning objectives are possible given the current resources.

I met with Derrick on Friday to discuss next steps in terms of summarizing the approach for Theme 3, determining approaches to quantifying outputs of the IPython notebooks, and approaching how to organize and quantify outputs from the collection of IPython notebooks that have been developed for the project so far.

# Week 5 This week I attended the Regional IOOS Directors meeting which was valuable both in terms of understanding modeling and data perspectives from the regional viewpoint, and in terms of hearing a bit more from the regions about their priorities in terms of ocean and coastal planning (conflict resolution), as well as their particular approaches to managing their data. In addition, the meeting resulted in gathering together documents that will help inform the direction that the themes go for the DATE project, and make sure to align the project with regional needs.

I also worked to catalogue existing iPython notebooks, and provide that catalogue online, which will serve to provide a resource both for the project and for the public by creating a one stop clear outline of current testing activities. I met with Derrick to talk about next steps, and determined that the final theme needs to be reworked over the next week. We also developed a preliminary approach to standardizing and editing iPython notebooks so that they can serve as a more polished product in terms of the outcomes of the project.

# Week 6

I completed cataloging Rich's notebooks and other resources on the References page on the Wiki, and also added additional references and resources after review. I also completed refining the development of test themes page so that we can start building the final products for the project. In addition, I outlined a standard format for the development of IPython notebooks for the project so that those on the project as well as the wider community could start with questions posed under the extreme even themes and then develop notebook products which could then be reviewed and edited by the program office or by COL so that we end up creating a library of products which will serve as modular examples of systematic testing of data integration issues.

I also started working on developing an environment that all the notebooks can be run in so that this can be added as a resource. During the COL Policy Forum, I also did some outreach on the extreme events themes and made some contacts to discuss data catalogs that could be tested under additional extreme event scenarios, including invasive species events.

# Week 7 After meeting with Derrick and John this morning, we've laid out a clear plan for developing a page that will map notebooks:questions:scenarios:themes within the extreme events project I'm working on. This week, I continued to work on the baseline notebook and the oil tanker spill notebook. During the meeting this morning, we determined that the best approach to developing additional notebooks will be to continue to develop my own, and to pick out those notebooks that Rich has developed and determine how they can fit into the scenarios. I'll be working on that further this week, and developing to do lists for those notebooks that require additional aspects in order to answer the questions posed within scenarios.

I prepared a presentation on my work so far on the project which I'll be presenting on Monday to the IOOC. I'll provide a link to the presentation, which goes over work flow and approach to developing notebooks, via the Wiki by Monday.

# Week 8 I continued working on a notebook for the Theme 2 and tried to think about best ways to summarize the outputs in a result section. I also met with Hassan to learn more about ERDDAP and Ecosystem based management as well as ecological forecasting and observing. The meeting was valuable in that it helped solidify and explain some of the larger issues with Biological Data.

I talked to the Bio TT about the DATE Project and set up a briefing with the head of the IOOC Bio TT to review the purpose of the DATE project and talk about ways that the information from the Bio TT survey might feed into additional DATE Projects. I also began to catalog Existing notebooks into the Extreme Even Themes.

# Week 9 # Week 10

###NOTES

Proposed Policy Analysis Tasks for DMAC DATE Project: TASK 1: Review and Provide Summary of Policy Documents, with focus on Data Integration Objectives in order to 1. Summarize Policy; 2. Identify Areas of Policy into which the IOOS Enterprise could fit PPD-21 - Presidential Policy Directive -- Critical Infrastructure Security and Resilience (http://www.whitehouse.gov/the-press-office/2013/02/12/presidential-policy-directive-critical-infrastructure-security-and-resil) PPD-21 takes a whole community approach [Determine if approach differs for coastal town, if approach focuses on data integration and whether it accommodates for data across jurisdictional boundaries - coastal boundaries, town boundaries, state waters, federal waters] National Security Strategy (NSS) Presidential Policy Directive 8 (PPD-8): National Preparedness DHS Strategic Plan 2012-2016 DHS National Response Framework (NRF) DHS Directive 8 (HSPD-8): National Preparedness Goal (NPG) DHS Annual National Preparedness Report (NPR) DHS/FEMA Crisis Response and Disaster Relilience 2030 White House National Strategy for Physical Protection of CIKR National Strategy for Global Supply Chain Security (GSCS)
Stafford Act National Response Framework Economy Act Post Katrina Emergency Management Reform Act Oil Pollution Act Disaster Relief and Emergency Assistance Act (2007) National Preparedness Report - 2012 National Infrastructure Protection Plan (NIPP) 2010 DHS Private Sector Preparedness Accreditation and Certification Program (PS-Prep), Implementing the Recommendations of the 9/11 Commission Act of 2007 National Exercise Program (NEP) supported by interagency, 37 core capabilities National Response Framework - 15 National Planning Scenarios [Query: should DATE project tie into one of these scenarios so as to create synergies with ongoing considerations? - Scenarios 9 and 10, earthquake and hurricane, would fit well if placed in coastal setting] Daily Open Resource Infrastructure Reports (See, http://www.dhs.gov/dhs-daily-open-source-infrastructure-report) (determine if this kind of reporting system could be or is being utilized in order to report on ocean observing infrastructure?] DHS CIP risk assessment program across private sector working through voluntary program - Infrastructure Survey Tool indexes and analyzes risk - goal is to create a practical dashboard tool that can compare index of security and resilience across facilities at local level and sectors at macro level to strengthen management skills

TASK 2: Review PPP policy in regards to disaster/resilience with focus on ocean and coastal data integration in the context of PPP Activity/Regional Assessment Activities in order to 1. Summarize Policy; 2. Identify whether and how data management and integration PPPs are currently part of resiliency policy

DHS National Protection and Programs Directorate, Office of Infrastructure Protection - 16 infrastructure sectors, Protective Security Coordination Division National Infrastructure Simulation and Analysis Center - DHS Regional Resiliency Assessment Program (RRAP) see page 153 for details on the program - attempts to standardize how resiliency is evaluated across varioussectors and facilities. RRAP is an interagency assessment of critical infrastructure and key resources (CIKR) combined with a regional analysis of the surrounding infrastructure - evaluates based on clusters and systems - participation is voluntary and information is protected under the Protected Critical Infrastructure Information Program PCIIP [Determine if IOOS Data is part of simulations and which core variables, determine whether and how private data and information could be integrated into IOOS system or if it is already in a protected way…] Formation of PPPs are highlighted in National Security Strategy, Presidential Policy Directive 8, and PPD-21, and National Response Framework [Determine if these partnerships highlight or emphasize data management or access - how do we incentivise the access of privately held/managed/formatted data in times of disaster without threatening proprietary value of the data? Rolling access? Limited Access? System of protected, event specific access?] Ostrom/PPPs - role in homeland security - this could be extended to data commons. Query: Could Ostrom’s approach to resources be applied to data or information tools? → Ostrom identifies a series of policies and rules that can be applied to common goods, including that there are clear boundaries, that users of property have to be able to monitor use and punish those who exploit it, and provide incentives to self regulate. DHS CIP Databases - Infrastructure Information Partnerships - Appendix C identifies a series of database integration and systems improvement tasks within the Department of Homeland Security, Infrastructure Protection Improvements are needed in the Automated Critical Asset Management System (http://dem.nv.gov/homeland_security/ACAMS_-_Automated_Critical_Asset_Management_System/) “ACAMS is a secure, online database and database management platform that allows for the collection and management of CIKR asset data; the cataloguing, screening and sorting of this data; the production of tailored infrastructure reports; and the development of a variety of pre- and post-incident response plans useful to strategic and operational planners and tactical commanders” This youtube explains the value of the platform in terms of protecting private information: http://www.youtube.com/watch?v=T2rHOy7ebXY

TASK 3: Address identified legal and policy analysis in the context of ocean and coastal data integration In Chapter 7, Egli identifies major findings which include the following needs for further analysis: a comprehensive review of legislation and statutes relative to resilience policy [Driving query could be, does current law and statutes include data integration, management, and information access within the context of resiliency? I.e. does the current framework addressing resiliency at multiple scales acknowledge the importance of access to information on a continuous basis so that at the time of the disaster, that information is ingrained in operations and available to responders?] 7.52: “a comprehensive decision support tool is needed to work across organizations and data boundaries. With the current processing tools, sought-after information is eventually found in the system, but the ability to collect, sort, and analyze infrastructure information in a systematic and automated manner is lacking.” See also operational recommendations, p. 85, 8.15: “Devlop an analytically sound enterprise platform that provides CIKR and preparedness planners with an effective collaborative tool, leveraging data fusion and automated protocols to collect and process disparate time-critical data from homeland security information sources.” 7.53: “We need a systematic functional resilience framework that is scalable and can be generalized to various operational and geographic areas….” 7.56: “The private sector often views the flow of information as one way - from them to the government….is not motivated to share information due to 1. regulatory repercussions...the risk of proprietary information being exposed to market competitors.” 7.77: “the current approach to risk calculations being used by government and industry is flawed and works only if there are reliable data regarding threats, vulnerabilities and consequences. Such data is seldom available and in some cases is impossible to acquire therefore, subjective information is often utilized to support decisions and grant proposals.” 7.92: “There should be more analytical rigor and quantitative information used in preparedness planning to reduce the amount of guesswork. Everyone is surprised by natural disasters….FEMA research indicates that there are disaster cycles and geographic trends that can allow better preparedness decisions at the regional level” CERTs - Community Emergency Response Teams - FEMA [Analysis could involve determining if the team can or should develop best practices, or whether there is an information officer role/training component in these scenarios which would serve to build information and community skill side of the emergency response] TASK 4: Based on Tasks 1 - 3, develop approach for further outreach to GIS departments in states, partner with http://www.nsgic.org/ in order to conduct survey that would specifically target the topic of data integration and preparedness in regards to natural disasters Egli’s Methodology included surveys, case studies, interviews of practitioners [NSGIC has engaged in state GIS surveys geared towards general information, DMAC ST could develop a targeted survey that would focus on ocean and coastal data - this might limit the number of states surveyed] Egli relied on agency interviewees open source public documents private sector subject matter experts (members of planning and operational staffs; representatives from private industry; members of academic community)

Notes on PPD-21 Policy: “Three strategic imperatives shall drive the Federal approach to strengthen critical infrastructure security and resilience: 1) Refine and clarify functional relationships across the Federal Government to advance the national unity of effort to strengthen critical infrastructure security and resilience; 2) Enable effective information exchange by identifying baseline data and systems requirements for the Federal Government; and 3) Implement an integration and analysis function to inform planning and operations decisions regarding critical infrastructure.” ...2) Enable Efficient Information Exchange by Identifying Baseline Data and Systems Requirements for the Federal Government A secure, functioning, and resilient critical infrastructure requires the efficient exchange of information, including intelligence, between all levels of governments and critical infrastructure owners and operators. This must facilitate the timely exchange of threat and vulnerability information as well as information that allows for the development of a situational awareness capability during incidents. The goal is to enable efficient information exchange through the identification of requirements for data and information formats and accessibility, system interoperability, and redundant systems and alternate capabilities should there be a disruption in the primary systems. Greater information sharing within the government and with the private sector can and must be done while respecting privacy and civil liberties. Federal departments and agencies shall ensure that all existing privacy principles, policies, and procedures are implemented consistent with applicable law and policy and shall include senior agency officials for privacy in their efforts to govern and oversee information sharing properly. ….3) Identification of Baseline Data and Systems Requirements for the Federal Government to Enable Efficient Information Exchange. Within 180 days of the date of this directive, the Secretary of Homeland Security, in coordination with the SSAs and other Federal departments and agencies, shall convene a team of experts to identify baseline data and systems requirements to enable the efficient exchange of information and intelligence relevant to strengthening the security and resilience of critical infrastructure. The experts should include representatives from those entities that routinely possess information important to critical infrastructure security and resilience; those that determine and manage information technology systems used to exchange information; and those responsible for the security of information being exchanged. Interoperability with critical infrastructure partners; identification of key data and the information requirements of key Federal, SLTT, and private sector entities; availability, accessibility, and formats of data; the ability to exchange various classifications of information; and the security of those systems to be used; and appropriate protections for individual privacy and civil liberties should be included in the analysis. The analysis should result in baseline requirements for sharing of data and interoperability of systems to enable the timely exchange of data and information to secure critical infrastructure and make it more resilient. The Secretary shall provide that analysis to the President through the Assistant to the President for Homeland Security and Counterterrorism.

####Marine Energy Planning The following notes are from Hannah Dean, and were originally written as presenter's notes for the March 2014 RA Directors briefing. There were a bit too detailed for that setting, but should be preserved for reference, so they have been copied here.

Questions for Theme 3: Marine Energy Planning

  1. What data is needed for particular energy planning policies (Outer Continental Shelf Lands Act, National Environmental Policy Act, etc)?
  2. Can that data be easily accessed from the set of registries and catalogues that the IOOS Systems Test is examining? If so, which registries can each dataset be accessed through? And, using baseline metrics developed under Theme 1 for registries, are there certain registries that are better or worse for accessing particular data sets?

Example: A. Specific Questions using OCSLA as a starting point:

  1. Can we plot bottom character/bathymetry against historical primary productivity (chlorophyll a, gC.m^2/year, phytoplankton abundance and species, zooplankton abundance and species) for a particular planning area (e.g. Gulf of Mexico) as might be required in the context of planning for mineral extraction under the OCSLA and corresponding policy?
  2. Can we plot jurisdictional boundaries (fisheries management, leasing blocks, marine sanctuaries, marine protected areas, etc.) against data sets for the IOOS core variables, and in particular data sets that might be used to estimate primary productivity (chlorophyll a, gC.m^2/year, phytoplankton abundance and species, zooplankton abundance and species)?

B. Projected outcomes:

  1. For each question listed under part A, the IPython notebooks would result in a set of metrics for each registry assessing whether or not the plot could be completed
  2. Aggregating the notebooks developed for this example would result in a summary of those registries where metadata, particular variables, or particular formats make it hard to overlay and integrate information needed to meeting the planning objectives of the particular policy
Clone this wiki locally